Unsupervised Intuitive Physics from Past Experiences
Authors: Sebastien Ehrhardt
Aron Monszpart
Niloy Mitra
Andrea Vedaldi
Venue:  (2020)
Keywords: Intuitive physics
Deep learning

Abstract: We consider the problem of learning models of intuitive physics from raw, unlabelled visual input. Differently from prior work, in addition to learning general physical principles, we are also interested in learning ``on the fly'' physical properties specific to new environments, based on a small number of environment-specific experiences. We do all this in an unsupervised manner, using a meta-learning formulation where the goal is to predict videos containing demonstrations of physical phenomena, such as objects moving and colliding with a complex background. We introduce the idea of summarizing past experiences in a very compact manner, in our case using dynamic images, and show that this can be used to solve the problem well and efficiently. Empirically, we show, via extensive experiments and ablation studies, that our model learns to perform physical predictions that generalize well in time and space, as well as to a variable number of interacting physical objects.

Under review as a conference paper at ICLR 2020
UNSUPERVISED INTUITIVE PHYSICS FROM PAST Ex- PERIENCES
Anonymous authors Paper under double-bling review
ABSTRACT
We consider the problem of learning models of intuitive physics from raw, unla- belled visual input. Differently from prior work, in addition to learning general physical principles, we are also interested in learning "on the fly" physical proper- ties specific new environments, based small number of environment-speciff experiences. We do all this in an unsupervised manner, using meta-learning for- mulation where the goal phenomena, such as objects moving and colliding with a complex background. We introduce the idea summarizing past experiences our case using dynamic images, and show that this can be used to solve the problem well and efficiently. Empirically, we show, via extensive experiments and ablation studies, that our model learns to perform physical predictions that generalize well in time and space, as well as to variable number of interacting physical objects.
predict videos containing demonstrations of physical
very compact manner,
INTRODUCTION
Many animals possess an intuitive understanding of the physical world. They use this understanding to accurately and rapidly predict events from sparse sensory inputs. In addition to general physical principles, many animals also learn specific models of new environments as they experience them over time. For example, they can explore environment to determine which parts of it can be navigated safely and remember this knowledge for later reuse.
Authors have looked at equipping artificial intelligences (Als) with analogous capabilities, but focusing mostly on performing predictions from instantaneous observations of an environment, such a few frames in video. However, such predictions can be successful only if observations combined with sufficient prior knowledge about the environment. For example, consider predicting the motion of bouncing ball. Unless key parameters such as the ball's elasticity are known priori, is impossible to predict the ball's trajectory accurately. However, after observing at least one bounce, is possible infer some of the parameters and eventually perform much better predictions.
In this paper, we are interested in learning intuitive physics in an entirely unsupervised manner, by passively watching videos. We consider situations in which objects interact with scenarios that can only be partially inferred from their appearance, but that also contain objects whose parameters cannot be confidently predicted from appearance alone (fig. D. Then, we consider learning a system that can observe few physical experiments to infer such parameters, and use this knowledge to perform better predictions in the future.
Our model has three goals. First, it must learn without the use of any external or ad-hoc supervision. We achieve this by training our model from raw videos, using a video prediction error as a loss.
Second, our model must be able to extract information about a new scenario by observing a few experiments, which we formulate as meta-learning. We also propose a simple representation of the experiments based the concept of "dynamic image" that allows to process long experiments more efficiently than using a conventional recurrent network.
Third, our model must learn good representation of physics without access to any explicit external supervision. Instead, we propose three tests to support this hypothesis. (i) We show that the model can predict far in the future, which is proxy temporal invariance. (ii) We further show that the model can extend to scenarios that are geometrically much larger than the ones used for training, which is proxy spatial invariance. (iii) Finally, we show that the model can generalize several

---

Under review as a conference paper at ICLR 2020
what happens next?
(A)
(U)
(B)
Figure I: Probing a new environment. Given the input showing (purple) ball moving from right to left, three possible future outcomes are possible: the ball passes above (A), passes under (U), or bounces (B) off the object. This cannot be predicted from appearance alone. Instead, our model learns the property of the object by probing each new environment via a small set of experiments.
moving objects, which is a proxy to locality. Locality and time-space invariance are of course three key properties of physical laws and thus we should expect any good intuitive model of physics to possess them.
In order to support these claims, we conduct extensive experiments in simulated scenarios, including lesting the ability of the model to cope with non non-trivial visual variations of the inputs. While the data simpler than a real-world application, we nevertheless make substantial progress compared to previous work, as discussed in section 2 We do so by learning from passive, raw video data good model of dynamics and collisions that generalizes well spatially, temporally, and to variable number of objects. The scalability of our approach, via the use of the dynamic image, is also unique. Finally, we investigate the problem of learning the parameters of new scenarios on the fly via experiences and we propose an effective solution to do so.
2 RELATED WORK
natural way to represent physics is to manually encode every object parameter and physical property
(e-g., mass, velocity, positions) and supervision make predictions. This has been widely used
to represent and propagate physics ( (Wu et al., 2015 2016 Battaglia et al.] [2066] Chang et al. 2017] al. 2018). If models like Wuu al. (2015 2016) also
Mrowca al., 2018; estimate environment parameters, these works rely on physics engine that assumes strong priors about the scenario, while our approach does not require such constraint.
Inspired by the recent successes of Convolutional Neural Networks (CNNs (Krizhevsky et al. 2012 and their application implicit representation dynamics (Ondruska & Posner 2016; Oh et al. 2015, Chiappa et al. 2017 Bhattacharyya al. 2018), researchers (Watters et al. 2017 Ehrhardt et al. 2019 Kansky al.) 2017) have tried base their approaches on visual inputs. They learn from several frames of a scene to regress the next physical state of system. In general these approaches
learn an implicit representation of physics (Ehrhardt et al. 2019 Watters et al. (011)) as a tensor state from recurrent deep networks.
Such models are mostly supervised using ground-truth information about key physical parameters (e.g.. positions, velocities, density) during training. While these approaches require an expensive annotation of data, other works have tried to learn from unsupervised data as well. Researchers have successfully learned unsupervised models either through active manipulation (Agrawal al. 2016
Denil et al. 2016 Finn et af.] 2016a), using the laws of physics (Stewart & Ermon 2017). using dynamic clues and invariances (Greff et al.) 2017 van Steenkiste et al. | or features extracted from unsupervised methods (Finn et al. 2016b] Ehrhardt et al. (2018). Fragkiadaki et al. (2016) also used an unsupervised system like ours, however, they assumed the rendering system and the number of objects to be known, which we do not, which prevented the internal representation to fail over time. Perhaps most related to our approach is the work of Wang et al.) al. (2018), where the model is learnt using future image prediction on simple synthetic task and then transferred to real world scenarios. They also demonstrate long-term dynamic predictions, however they did not generalize to different background/number of balls.
In other works, models are taught to answer simple qualitative questions about physical setup, such as: the stability of stacks of objects (Battaglia et al. 2013 3: Lerer et al. 2016 Li et al. 2017 Groth
ett al. 2018). the likelihood of a scenario (Riochet et al. 2018). the forces acting behind scene (Wu
et al.
2017 Mottaghi et al. 2016) properties of objects through manipulation (Agrawal et al.
2016 Denil et al. (0116). Other papers compromise between qualitative and quantitative predictions
and focus on plausibility (Tompson et al. 2016: Ladický al. 2015 Monszpart et al. 2016).
2

---

Under review as a conference paper at ICLR 2020
I(R)
median
7
S
loss
Lo:To
X0:To
Xt To:t
X
It
Io:T
m
a
f ; 151 f2l
dynamic image median
V
I(R1)
1(22)
I(Rx)
1ood
2
bood
experience
8 (R1...., Rx)
V B ( NxW
Figure 2: Overview our method. (bottom right block) acts as a meta-learning module. It takes as input past experiments compressed into dynamic images alongside with the median image and learn to optimise m and a to minimise the final loss. 7 (top-left yellow block) extracts states Xo:T, which are carried forward with the auto-regressive predictor o. Finally renders frame I, from x,-
Finally similar problems can be found in online learning settings (Nagabandi et al. 2019a b). These works also use meta-learning but, differently from ours, allow their models learn and adapt at test time from feedbacks.
METHOD
describe our model, illustrated fig. starting from the input and output data.
A scenario S is physical environment that supports moving objects interacting with it. In this paper, we take as scenarios 2.1D environments containing obstacles and we consider rolling balls as moving objects. Hence, interactions are in the form of bounces. Formally, a scenario is defined over a lattice Q= (O...., H 1} x {0, {O..... W 1} and specified by list of obstacles = {(O,,b)) by), j 1, ... K). Here O; C Ω is the shape of an obstacle and bj E {B, A, U} is flag that tells whether the ball bounces against it (B), passes above it (A), or passes under it (U). Obstacles do not overlap.
A run is tuple R yS associating scenario S with trajectory y (ye Ωt ttt= 0.... - for the ball (this is trivially extended to multiple balls). Scenarios and runs are sensed visually. The symbol I(S) Ω >> R3xHxW denotes the image generated by observing a scenario with no ball and [ = I(S, yt) the image generated by observing the ball at time t in a run. The symbol I(R) (L,(R), O....,7 T 1) denotes the collection of all frames run, which can be thought of as a video.
We are interested in learning intuitive physies with no explicit supervision on the object trajectories, nor an explicit encoding of the laws of mechanics that govern the motion of the objects and their inter- actions and collisions with the environment. Hence, we cast this problem as predicting the video I(R) of the trajectory of the objects given only a few initial frames Io.ra (R) = where To < T.
(R)),
We are not the first to consider a similar learning problem, although most prior works do require some form of external supervision, which we do not use. Here, however, we consider an additional key challenge that the images In.r, (R) do not contain sufficient information to successfully predict the long-term objects' motion. This because these few frames tell us nothing about the nature of the obstacles in the scenario. In fact, under the assumptions that obstacles of type B, A and U have similar or even the identical appearance, is not possible to predict whether the ball will bounce, move above, move under any such obstacle. This situation is representative of agent that needs to operate in a new complex environment and must learn more about it, from past experiences, before it can do reliably.
Thus, we consider a modification of the setup described above in which the model can experience each scenario for while, by observing the motion of the ball, before making its own predictions.
3

---

Under review as a conference paper at ICLR 2020
Formally, an experience is collection of N runs E (R1...., Rx) all relative to the same scenario with each a different trajectories y1..... .... By observing such examples, the model must determine the nature of the obstacles and then this information to correctly predict the motion of the ball in the future. We cast this as the problem of learning a mapping
Ω (To:ro (R), I(E)
I(R),
where I(E) = (I(R)).... I(Rw)) are the videos corresponding to the runs in the experience. We will call R. the prediction run distinguish it from the experience runs E.
3.1 A META-LEARNING PERSPECTIVE
The setup we have described above can be naturally described as meta-learning, Namely, eq. ( can be thought of as incorporating "local" learning rule M that maps the experience
scenario- specific predictor Ω the fly: (2.) S2(- [()) M[][ Hence M must extract from the experience as much information as possible about the underlying scenario and transfer it to the scenario-specific predictor Ω. In order to learn M, we consider meta-samples of the type (S, R, E) comprising a scenario S, a prediction run R and experience runs E. Given a dataset DD of such meta-samples, meta-learning searches for the mapping M that minimizes the error on the prediction runs:
M argmin M
1
Σ
((I(R), ((Jo:T((R))) Ω 2= =[](E]
D
(S.R.E)ED
3.2 COMPRESSED DYNAMIC EXPERIENCES
Concretely, we parameterise the scenario-specific predictor 92 w. m) using parameters w. the weights a deep neural network, which is fixed and scenario-independen and m, which is scenario- specific. The latter extracted by a network m v (I()) from the experience videos I(E). Since we expect m to provide information about the nature of the obstacles in the scenario, we let m E x be a tensor with the same spatial resolution as the scenario and interpret it as an "obstacle mask".
Given that the function v takes as input a number of video sequences, one may think to implement it recurrent neural network; however, recurrent networks scale badly, especially in meta- learning context to the point that we had difficulties even running such a model at non-trivia scale. Instead, we propose construct V based on a compact representation of the experience which leads to a much more efficient design.
the concept of dynamic image (Bilen al. 2016), which encodes a video weighted average of its frames: [Io,,. I--1) Σi= IrI, with a; = -n' 1(=+ Since the dynamic image is only sensitive changes in the video, we complement it by computing also the median image fo 1) 1)= = median I, and combine the two functions fi and f2 in a single representation f by stacking the respective outputs along the channel dimension.
For this,
i+1
With this design, we can rewrite the map V as follows:
U (I(E))
maxpool U ((((RR)).))
n= n=I....N
Here the map V _6xHXx + xxxHXW which can be implemented as a standard CNN, takes as input the dynamic/median image and produces as output the obstacle mask m. The pooling operator summarizes the information extracted from multiple runs into single mask m.
We also consider a second similar map 12 H6 + RC*HXW to extract an "appearance" tensor a. The latter helps the generator render obstacles above or underneath the moving ob- jects as needed. The response of this function is also max-pooled over runs, but channel-wise: [pool axjomu = [axce] where k(c) = argmax Evn(as] [as]
In practice, V and Vن are implemented as single neural network v xxxHxW - ((+C))xXX where v is the first output channel and 1 the others.
Optional obstacle mask supervision. In the experiments, show that the map i can be learned automatically without any external supervision. We contrast this with supervising V with oracle
4

---

Under review as a conference paper at ICLR 2020
rendition of the obstacle map. To this end, define the tensor Mg. for scenario S to be the indicator mask of whether a pixel contains a solid obstacles (including the perimetral walls) and then minimize the auxiliary loss f(mg, v (I(E)) mmg v (I(E)))
3.3
AUTO-REGRESSIIE PREDICTOR
The predictor & is designed as an RNN that takes as input To past states X and outputs a new state X,. Each state variable is turn distributed representation of the physical state of the system, in the form of a "heatmap" E R R#xW Considering To past states allows the model to auto-regressively represent the dynamics of the system if so learning chooses to do.
(Xi-To..... xt-1)
The predictor also takes as input the scenario representation given by eq. 3 The first To state variables are initialized from observations To:Ta (R) via an initialization function 1j R3x Hxx "fxW,, We thus have
Xo:T, = [[)()(())]ss
(initialization)
(4)
X = [xxxtToot: m)
(auto-regressive prediction)
(5)
In short, this model estimates recursively the evolution of the system dynamics from the visual observations of the first To samples.
Conditional generator. Variable x, contains information about the state of the moving objects (balls). This is then converted into prediction of the corresponding video frame, combining also the appearance tensor and the median of the first To images in the sequence I = median: To L.- This is formulated as a conditional generator network I, g(x,, a)
Video reconstruction loss. Next, we discuss the nature of the loss equation 2 Owing also to the static background, the conditional video generation task is relatively simple provided that the dynamics are estimated correctly. As consequence, the generated videos are likely to closely approximate the ground truth ones, so the loss function É in equation does not require to be complex. In our experiments, we combine the L image loss with the perceptual loss of (Johnson et al. 2016). The latter obtained by passing the ground-truth and predicted images through the first few layers of a pre-trained deep neural network VGG-16 (Simonyan & Zisserman 2015) in our case, and then comparing the resulting encodings in L distance. The overall loss is then given by [,), = >rI I, (L) - - e(I,)] (details in the sup. mat.). The perceptual loss is robust to small shifts in the image reconstructions that may arise from imperfect physics predictions and favors reconstructing sharp image; by comparison, the 2 loss alone would result in blurrier images by regressing the mean. In practice, trading-off the _ and perceptual losses results in qualitatively better images as well as in better trajectory prediction.
State space integrity. An issue with the recursion equation iis that the state x, may, in the long run, falls outside the space of meaningful states. leading to a catastrophic failure of the predictor. This is especially true for predictions longer than the ones used for training the model. In order to encourage the recursion to maintain the integrity of the state over time, we add a self-consistency loss |x,- (R)) ? Here the network 17 is the same that is used to extract the state space from the initial frames of the video sequence in equation 4 which weights are fixed. In practice, we find that, by using this additional loss, our recursion learns to maintain the integrity of the state nearly indefinitely.
4 EXPERIMENTAL SETUP
Data. A scenario S (see embedded figure) is generated by sampling certain number of obsta- cles of different types m; and shapes O (either rectangular or custom), placing them at random locations and orientations stacles (denoted R2), a random number from 3 to 4 rectangular obstacles (R4) or two curved obstacles (C). Scenarios are rendered by painting the background and wall around it. Then, all obstacle are painted in a solid colour, randomly picked from a fixed palette to ensure suffi- cient contrast. The background is also painted in solid color (general case) or by using a texture image (only curved shapes, denoted C+T). Crucially, there is no correlation between an obsta- cle's type and its shape, location and color, so its type cannot be inferred by appearance alone.
2D board. We consider boards with either two rectangular ob-
5

---

Under review as a conference paper at ICLR 2020
Runs R are generated by placing one more dynamic ob- jects ("balls") with a random initial location and a random momentum oriented towards the obstacles, simulating the 2.1D physics, and rendering the balls as circles with a fixed color (see fig. Al in supp. mat. for more examples).
For each scenario S, we sample N + 1 runs, using the first as a prediction run and the others as experience runs, forming triplets (S, R. E). Unless otherwise stated, we set N and let each run evolve for 60 frames, including for the experience runs. Thanks to our compact representation, we only use 200MB for experiences for backpropagation during training. Using a recurrent network requires an amount of memory larger in proportion with the length of an experience, i.e. 60-fold, so 12GB for 64x6e 64 boards, which wouldn't 't allow the receurrent architecture to fit into memory. Using shorter runs is also not possible, as they tend to contain no collisions.
For training, samples (S, R. E) are drawn in an on-line fashion and are thus unlimited. For testing, we use a fixed set of 200 samples. We generate boards of size 64 x 64 and 128 x 128, but use the latter only for testing purposes to evaluate generalization. Unless otherwise stated, the board size is 64 x 64.
Evaluation metrics. We report the video prediction error as average T,2 image loss. In order to assess the ability of the method to predict good trajectories, we note that blobs tend to emerge in the heatmaps X2 in correspondence of the tracked objects. Hence, we handcrafted a simple blob detector h(x)) that detects the blobs contained in the heatmaps x, (see sup. mat.). We then report the number of detected blobs vs the actual number of moving objects and, for each, the distance in pixel space of the predicted blob center and ground-truth object center, averaged over the different scenarios. For each experiment we report mean and standard deviation across all sampled scenarios.
Baselines. We compare with three baselines. The first is version of the Interaction Net- works (Battaglia et al.) 2016), trained with perfect knowledge of the object locations, background and object generation. The baseline works directly on the object positions and regresses object positions. The ground truth obstacle map is given as input and transformed into a vector thanks to a pre-trained VGG- 16 architecture. We also reimplemented the RLVN network of Wang al. (2018). which has a vector representation bottleneck, and compared video prediction error results. Note that this baseline wouldn't 't adapt to bigger board. Our last baseline amounts to running the ground-truth physics simulator after removing the obstacles from the board (and thus results in perfect predictions for trajectories that do not hit solid obstacles).
Implementation details. Networks & and 1) share very similar auto-encoder type architecture, the network & uses a U-Net-like architecture (Ronneberger & Fischer (015). As stated in Wang et al. (2018) intuitively V would better preserve appearance while the structure of t would loose some appearance structure and put an inductive bias on dynamic predictions. g is a fully-convolutionn 6-layers stack. Our implementation uses TensorFlow 1.9 (Abadi et al. 2015) and the Adam opti- mizer (Kingma & Ba 2014) with learning rate 10-4 and Xavier initialization (Glorot & Bengio 2010). We used batch size of 10 samples (S, R. E). Models are first trained for 110,000 iterations using only the 2 and self-consistency losses for eq. (2). and then optionally fine-tuning for further 1,000 iterations using the perceptual loss as well. All models are trained on R2 with board size 642 and tested without any fine-tuming on different board types and sizes (see Tab. 1,2.3.A.3). For the C+T scenarios, we first train the model using flat colour R2 scenarios and then fine-tune for 55,000 iterations using the textured data. Unless otherwise specified, models are trained in fully unsupervised fashion. Full details can found in the sup. mat.
RESULTS
Full system. Table I rows 1-4 report the prediction accuracy of our system on the 200 simulated test sequences using one moving object. In the table, we distinguish three durations: To 4 is the number of frames to bootstrap the prediction of a test runs, Train = 20 is the length of prediction runs observed during meta-training and Tiss = KTrain is the length of the runs observed during meta-testing, where k- = 1,3,5 1.3.5. 5. We test both 64 x 64 boards (which is the same size used for training) and larger 128 x 128 boards to test spatial generalization. We also compare training using the L or the L +perceptull loss for video prediction supervision. We report the number of detected objects (which should be 1 in this experiment), the video prediction error, and the position prediction error.

---

Under review as a conference paper at ICLR 2020
Table 1: Predicting one moving object. Obst. the obstacle type (R2, R4, C). The test board size
can be either 64 x 64 (same as in training) or 128 x 128 and we consider using either supervision or obstacle supervision. We test the average prediction error at Twu = 20, 60, 100 well above the duration Tuain 20 observed during training. Position errors were normalized by the board size
diagonal. Video errors were normalized to board size 64 where larger. L loss on positions used for (14)-(15).
Tine Tini 20
Tiee 3xTmm
Time 5 x 3xTmm
Sup.
size
Train loss
obbj.
er
Abb..
L2
obj.
Pos. etr.
None
64
L.
1.0002
2.5226
0361:096
0.3105
4.6133
241
0.1203
4.7-33
.2622.180
(2)
R2
None
1282
LA
0.740044
1.7413
.1794-226
0.1403
1.7411
4844-228
0.0:0.11
1.7=1.1
.531
(3)
R2
None
61
Percop.
100102
2.5:299
02881:073
10000
5.3140
14541.148
100022
5,5=40
.2862.168
(4)
R2
None
128²
Percep.
0.940444
1.741.
.1044.170
10404
2.0+144
2274.1199
100404
2.0-1.4
.320-.172
(5)
R2
Obsl,
64
Percep.
1.000
3.0.333
.218
10402
5.7444
11124
1.040.3
5.8-42
.231
(6)
R2
Obst.
128²
Percep.
8,3433
2.44
.2234.064
26.54664
5.6:4400
354
25,8463
588-4.1
.3634.133
None
61
Percop.
100101
2.91355
0401:006
L1103
5.4144.1
.1551.160
10:03
5.4240
.2651.115
(8)
None
128a
Percep.
0.9204
1.7214
174
1.0104
2.0214
227:1:199
1.0004
5.6446
.320= 172
Obst
61
Percop.
1.0:0.11
2.913.1
0241:.02
101011
5.7142
.1281.114
100:0.
5.5:4.1
.247:1144
(10)
Nome
64
Percep.
1.00.1
3.44229
.017:004
6.1434
10-022
6.2=29
.222-.147
(11)
C+T None
128*
Percep. 1.0+01
675
.022
1.0002
144
065
0.9203
1.4
2121
Ablation: removing the median and dynamic images, respestively (scc text)
(12)
R2 None
64
Percep.
1.0203
4.0.1:0
0074
0.8405
5.1240
3022 202
0.6205
5.0=18
.357=2212
(13) None
64
Percep. 14106
3.213.4
.0441.062
1.8008
5.5:122
151 1.7208
5.62422
.32622.136
Comparison: Interaction Network (14.15), RIVN (16) and Ground Trith Simulater minus Ohstacles (17-18)
(14)
Full
128
L
038
1864
.31-21122
(15) C
Full
128*
L Pos.
003
.349 #.03
(16)
R2
642
3.642
5.3+33
5,5=3,5
(17)
R2
612
060-+-099
229- 224
.224 224-4.209
(18)
612
.55
2194:217
.232:1.228
We find that switching from the L to the L +pperceptlal loss for training (rows I vs 3 and 2 vs 4) reduces significantly the trajectory prediction error and prevents the object from disappearing. This, however, results to slightly larger L2 video prediction error compared to empty boards predicted in the long run by models trained with L2 loss only. We also find that generalization through time is excellent: prediction errors accumulate over time (as it is unavoidable), but the recursion evolves without collapsing well beyond Tirain (qualitatively we found that the recursion can be applied indefinitely.. This should be contrasted with prior work on intuitive physics (Fragkiadaki et al. 2016) where recursion was found to be unstable in the long term.
Spatial generalization to larger boards (rows 2,4,6) is also satisfactory if the perceptual loss is used. However, we did notice the emergence of a few artifacts in this case.
In rows 5-6 use external supervision train the obstacle map predictor suggested in section 3.2 (the rest of the system is still unsupervised). Supervision improves the trajectory prediction accuracy for boards of the same size the ones seen in training. However, generalization larger boards is poorer, suggesting that explicit supervision causes the model to overfit somewhat.
Rows 7-11 show results for more complex shapes (C) and appearances (C+T). Prediction accuracy is only slightly worse than with the simpler obstacles (e.g. rows 3 vs 7 or 9 vs 10).
Table 2: Importance of experience. For the unsupervised model, we report the trajectory prediction error at T = 40 (rows 1-3) and the obstacle mask L2 prediction error (rows 4-6) for different obstacle types. The number of runs N in the experiences is varied from to 50. For the obstacle mask prediction error, we also report the trivial baseline
|| ||mg m ll as 'AlI-ON" (see text). Position
errors were normalized by the board size (642) diagonal. Tuain 20.
No Fr:: Obst.
AIL-N
N
7
10
20
50
(I)
Pus.
132
.144 157
.108-136 136
.0834.128
.077..120
.071+.114
.0702.116
.0702.104
(2) Pos. R4
152
.194
169
1658_164
1152
110
.114 130
.1102.15
.1142:152
(3)
Pos. C
A591.124
.6611.172
.125_.188
.0911.19
.0911.166
.0911.1
.1012.144
.1062.143
(4) Ohs R2
13.22111
24.61164
10.3995
7.6284
42454
3.8247
3.523.
3.6235
3.8.3377
(6) Obs.
16.81133
23.6115.1
14.111109
11.5299
7.7880
7,4:7.3
7.311.2
7,4172
8.418.11
(5) Obs. C
9.4899
27.8
10.14482
7.0
5.44:5666
5.3-55.2
5.14:5.2
5.74:5.6
6.045599
7

---

Under review as a conference paper at ICLR 2020
(U1)
(U2)
(U3)
(B1)
(B2)
(B3)
Figure 3: Qualitative results. We show time lapse images for the trajectory predicted for the ball
colliding with obstacles of type U (permeable; left) and B (solid; right). We test three variants of
the model: (1) the model supervised with ground-truth obstacle masks, (2) the unsupervised model
and (3) the model after suppressing the dynamic image that summarizes the experience (ablation). Qualitatively, (1,2) produce plausible predictions but (3) does not.
Ablation. We suppress respectively the median image (row 12) and dynamic image (row 13) in the feature extractor f summarizing the experiences. We notice a sensible deterioration of performance compared to row 3, suggesting that both components are important.
Comparison. Compared to the fully-supervised IN (rows 14-15), our unsupervised method is highly competitive, especially for longer simulation times (rows 4,8). The method also outperforms (except for very long simulations where drift prevails) the ground-trute truuth simulator that ignores the obstacles (rows 16, 16,17). Finally we show that our method surpasses RLVN (Wang et al. (0ow) 16), qualitatively, this method produces similar results to (13) and the object tends to disappear when encounting an object.
The importance of experience. We investigate how much information the model extracts from the experiences E by ablating the number of runs N in it. For the special case N 0, since there are no runs, we generate an pseudo experience run by copying the first frame To (R) of the prediction run R.
In table 2 we report the trajectory prediction accuracy for different obstacle types (row 1-3). We also test the ability to predict the obstacle mask mgg (defined in section 3). Since the model is unsupervised, the learned mask m will not in general be an exact match mgg (e.g. the mask range arbitrary); hence, after fixing all the parameters the model, we train simple regression network that maps m to mgg and report the prediction accuracy of the latter on the test set (see sup. mat. for details). As a point of comparison,
also report the error || mail-on ll2 of the trivial baseline that predict a mask Mall-on where all objects are highlighted, regardless of their flag.
We note that there is very large improvement when we move from zero to one experience runs, and a smaller but non-negligible improvement until N = 10. Furthermore, comparing the results with predicting mall-on shows that the system can tell from the experiences which obstacles are solid and which are not.
Table 3: Predicting the obstacles: supervision vs no supervision. The table reports the obstacle mask prediction error (T22) for network trained with or without supervision for the obstacles (see text). The last column shows a multi-ball predictor operated with 3 balls each run. Turain = 20.
Supervision R2
R4
C+T R2 balls) C+T R2
Obstacle 2.7 6,347s 3,5 2.6
2.6439
7.1
None 3,8 7.4172 5.3152 4.0
3.9415
7.6
Supervised obstacle regression. Table 3 compares our unsupervised method to using full obstacle map supervision in order predict the obstacle map mgg from the experiences E. For the unsupervised system, the obstacle map is estimated as explained in the paragraph above. As expected, supervised learning achieves a lower error, but the unsupervised method is still much better than the trivial baseline of table 2 Finally, we observe that for network fine-tuned on C+T and evaluated back on R2 (C+T~RR2 in Table 3) obstacle predictions results remain competitive with results on R2 only while object appearance and backgrounds drastically changed in the new setting.
Multiple moving objects. We also test whether the system trained with a single moving object can generalize to multiple ones. In table A.3 (see sup. mat.) we show that the network can simulate the motion properly until balls collide, after which they merge (see videos in the sup. mat.). This indicates that, just as physical laws, the rules learned by the model are local. We also train model (scenario R2) showing it from I to 3 balls in each run. This model is able not only to correctly handle ball collisions, but is also able generalize correctly to several more objects the board. Furthermore,
8

---

Under review as a conference paper at ICLR 2020
shown in table 3 this model still predicts correctly the obstacle masks from experiences, despite the act that the latter are much more crowded.
CONCLUSIONS
We have demonstrated system that can learn an intuitive model of physics in an unsupervised manner. Differently from most prior works, our system is also able to learn on-the-fly and very efficiently some physical parameters of new scenarios from few experiences. Prediction results are strong, competitive with fully-supervised models, and predictors generalize well over time space, and an arbitrary number of moving objects. Our next challenge is to apply the system more difficult scenarios, including non- pronfrlntal views of the physical worlds. For this, we plan to learn a function to summarize past experiences in more general manner than the dynamic image could.
REFERENCES
Abadi al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflo..org
P. Agrawal et al. Learning to Poke by Poking: Experiential Learning of Intuitive Physics. In Proc. NeurIPS, pp. 5074-5082, 2016.
Battaglia al. Interaction networks for learning about objects, relations and physics. Proc. NeurlPS, pp. 4502-4510, 2016.
P.W. Battaglia, J. Hamrick, and J.B. Tenenbaum. Simulation engine of physical scene under- standing. PNAS, 188745331332--18333,, 2013.
A. Bhattacharyya et Long-term image boundary prediction. In Thirty-Second AAAI Conference Artificial Intelligence. AAAI, 2018.
H. Bilen et al. Dynamic image networks for action recognition. In IEEE CVPR, 2016.
M. B. Chang et al. A compositional object-based approach to learning physical dynamics. In Proc. ICLR, 2017.
Chiappa et al. Recurrent environment simulators. In Proc. ICLR, 2017.
M. Denil al. Learning to perform physics experiments via deep reinforcement learning. Deep Reinforcement Learning Workshop, NIPS, 2016.
S. Ehrhardt et al. Unsupervised intuitive physics from visual observations. Proc. 2CC8,2018.
S. Ehrhardt et al. Taking visual motion prediction to new heightfields. Computer Vision and Image Understanding (CVIU), 2019.
C. Finn, Goodfellow, and Levine. Unsupervised learning for physical interaction through video prediction. In Advances in neural information processing systems, 2016a.
C. Finn et al. Deep spatial autoencoders for visuomotor learning. In Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp. 512-519. IEEE, 2016b.
K. Fragkiadaki et al. Learning visual predictive models physics for playing billiards. Proc. NeurIPS, 2016.
X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256. 2010.
K. Greff et al. Neural expectation maximization. Proc. NeurIPS, 2017.
O. Groth al. Shapestacks: Learning vision-based physical intuition for generalised object stacking. In eccv, 2018.
Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. Proc. ECCV, 2016.

---

Under review as a conference paper at ICLR 2020
K. Kansky al. Schema networks: Zero-shot transfer with generative causal model of intuitive physics. In International Conference on Machine Learning, pp. 1809-1818, 2017.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. Proc. ICLR, 2014.
A. Krizhevsky, Sutskever, and Hinton. Imagenet classification with deep convolutional neural networks. In Proc. NeurIPS, pp. 1097-1105, 2012.
Ladický al. Data-driven fluid simulations using regression forests. ACM Trans. on Graphics (TOG), 34(6):199, 2015.
A. Lerer, S. Gross, and Fergus. Learning physical intuition of block towers by example. In Pro- ceedings of the 33rd International Conference on International Conference on Machine Learning- Volume 48. pp. 430 438, 2016.
W. Li, A. Leonardis, and M. Fritz. Visual stability prediction and its application to manipulation. AAAI, 2017.
A. Monszpart, N. Thuerey and N. Mitra. SMASH: Physics-guided Reconstruction of Collisions from Videos. ACM Trans. on Graphics (TOG), 2016.
R. Mottaghi et al. Newtonian scene understanding: Unfolding the dynamics of objects in static images. In IEEE CVPR, 2016.
D. Mrowca et al. Flexible Neural Representation for Physics Prediction. Proc. NeurlPS, 2018.
Anusha Nagabandi, Chelsea Finn, and Sergey Levine. Deep online learning via meta-learning: Continual adaptation for model-based iclr, 2019a.
Anusha Nagabandi et al. Learning to adapt in dynamic, real- world environments through meta- reinforcement learning. iclr, 2019b.
J. Oh et al. Action-conditional video prediction using deep networks in atari games. In Advances in Neural Information Processing Systems, pp. 2863-2871, 2015.
P. Ondruska and I. Posner. Deep tracking: Seeing beyond seeing using recurrent neural networks. In Proc. AAAI, 2016.
R. Riochet et al. IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning. ArXiv e-prints, 2018.
O. Ronneberger and T. Fischer, Pann Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, 2015.
A. Sanchez-Gonzalez al. Graph networks as learnable physics engines for inference and con- trol. Proceedings of the 35th International Conference on International Conference on Machine Learning, 2018
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
R. Stewart and S. Ermon. Label-free supervision of neural networks with physics and domain knowledge. In AAAI, pp. 2576-2582, 2017.
J. Tompson et al. Accelerating Fulerian Fluid Simulation With Convolutional Networks. ArXiv e-print 1XXi:16677003977 2016.
S. van Steenkiste et al. Relational neural expectation maximization: Unsupervised discovery of objects and their interactions. Proc. ICLR, 2018.
Z. Wang et al. Neural allocentric intuitive physics prediction from real videos. arXiv preprint arXiv: 1809 03330, 2018.
10

---

Under review as a conference paper at ICLR 2020
N. Watters et al. Visual interaction networks: Learning physics simulator from video. In Proc. NeurIPS, 2017.
J. Wu et al. Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. In Proc. NeurIPS, pp. 127-135, 2015.
Wu al. Physies 101: Learning physical object properties from unlabeled videos. Proc. BMVC, 2016.
J. Wu et al. Learning to physics via visual de-animation In Proc, NeurIPS, 2017.

---

Under review as a conference paper at ICLR 2020
A1 DATA GENERATION
In fig. Al and fig. A2 we provide more details on the scene aspect. For solid background color, the objects and background colours are sampled to be different. Every rectangular object have height and width randomly sample in between 10 and pixels. Custom object are loaded from template images and randomly scaled from to 2.
Figure A1: Dataset samples. Pairs of sample data from left right: R2, C, C+T.
2013
Figure A2: Additional material used for data generation. From left right: color palette for solid object/background color (R2.R4,C), texture used to increase background complexity in (C+T), set of custom objects for (C) data generation.
A2
IMPLEMENTATION DETAILS
A2.1
NETWORK ARCHITECTURE
In fig. A3 show the details of all the networks of our pipeline. The regression network men- tionned in 5ec.5 is simple 3 layers fully convolutional network with [3x3] kernel convolution and intermediate channel of size 64.
A2.2 TRAINING DETAILS
In the overall loss is:
E(t) = AI - ii,1 + Ax|ix; (I) Aplle(I,) - =(f.))12
With t ranging from 0 to Trrain For = {o..., To) we used \ 1 and X = Ap = 0. In such case i, g(x,, a, f) a,,
For > To. when training with L only we used A = Ax = and A 0. When finetuning
with perceptual weighted the perceputal loss with coefficient A 10 and the other losses with Ar = 2x = 0.01. Finally, perceptual loss uses feature extracted from conv3 of VGG-16.
The Interaction Network baseline is simple Interaction Network which state concatenated the background feature extracted from a pre-trained VGG-16 network. The state propagator uses the last 4 states. The network then trained over T Train 40 time steps with Adam optimizer with learning rate 10 and batch size 50. We found that 50 000 iterations were sufficient to reach convergence.
All experiments are run using single NVIDIA-GPU Titan
A2.3 EVALUATION DETAILS
To detect blob create a binary image by manually thresholding the intermediate heatmap x; and using simple Hough circle detector. Every threshold coefficient hand picked for every model but kept constant across scenarios and different data.
For position errors used Lg loss, when more than object was detected took the distance to the closest detected object and applied fix penalty to the loss for all other detection (half the board size).
12

---

Under review as a conference paper at ICLR 2020
Con
7
3x3x64
3x3x64
3x3x120
3x3x128
3x3x256
3x3x256
3x3x256
4x4x120
Sx3x
3x3x120
4x4x64
3x3x64
3x3x64
3x3x
Conv Layenla
Upcone-Layer
Gons+ Rali-l ayerhoomm stride
MaaPool
m, x T.d
6x8x64
3x3x64
3x3x64
3x3x126
3x3x 128
3x3x256
3x3x256
3x3x256
4x4x129
3x3x128
3x3x128
4x4x644
3x3x64
3x3v64
[,
3x3x32
3x3v32
3x3v64
3x3x64
3x3x64
3x3r3
5x5x18
3x3x
3x3xd2
3x3x32
3x3x64
3x3x64
3x3x64
3x3x 128
3x3x
4x4x64
3x3x04
3x3x64
4x4x32
3Y3x32
3x3x64
4x4x15
3x3x1
Figure A3: Networks architecture detail. All the networks generally uses layer normalization and & are simple auto-encoder type architecture while V/ shares similarity with U-Net.
Table A1:
Predicting moving object. Obstacle type R2. The test board size is 64 x 64 We test the average prediction error at Tiea 20, 60, 100 well above the duration Tizin = 20 observed during training. Position errors were normalized by the board size diagonal.
Tiza
Tuan
Tis x Tuii
Taa: Tum
No.
Sup.
Truin
loss
#obj.
Vid. Lg
Pos. em.
# obj
Vid. Lg
Pos. em.
* obj.
Vid. Li
Pro.
2.5429 0013-012
1.00011
588444
0955.095
.201::1 6.2445
None
Perc.
1.020.1
322311
.030-068 068
1.000.
5,5240
.3362.111 09404 5,5200 .2624 180
A3 ADDITIONAL RESULTS
Table AI we report two more results on one moving object prediction. (1) an upper bound of system had full knowledge of {B, U} obstacle locations at training time. We simply replaced with the binary map B obstacles and a with the binary map of obstacles where Is indicate the presence of the obstacle type. In this scenario the results introduced in Table our method still remains competitive with this upper bound. (2) reports results of the network trained on C+T and evaluated back on R2. As in Table we see that the network still manage to make accurate predictions. Table A2 is an extension of Table 2 in the paper for the supervised model and Table A3 introduces results multiple balls prediction.
note although that obtain upper bound of
13

---

Under review as a conference paper at ICLR 2020
Table A2: Importance of experience. For the supervised model, we report the trajectory prediction eπror at T' 40 (rows 1-3) and the obstacle map prediction error (rows 4-6) for different obstacle types. The number of runs N in the experiences is varied from to 50. Position errors were normalized by the board size diagonal. Tuain 20.
Eir.
N
50
(1) Pos. R2
181114]
9411.975
531.11
.151041
08::21
07:.11
.10.221
.94.38
Pos. R4
3.2821944
2.221167
1.56
631
.34
372
44
831.97
(3) Pos.
1.051091
581.61
362.49
.151:29
.13.:26
.12..22
161:26
241.36
(4) Ohs. R2
4.1240
36238
3.1334
2.3233
2.3224
2,3224
2.3277
2.6330
Obs. R4
6.1122
4.14388
3.1334
2.8:33
2.74331
2.63300
2.7.19
3.3136
Obs.
39439
3.3463.7
2.84:311
2.442227
2.4228
2.3426
24426
2,5229
Table A3: Predicting multiple moving object. Obstacle type is R2, all network were trained on the fully unsupervised with perceptual loss. Num. the maxmium number of balls per run during training. For instance Num=3 means that every runs sampled contained one to three balls. The test board is size 64 We test the average prediction error at Tiett 20, 60, 100 well above the duration Train 20 observed during training.
Tixx
Trai 20 Tws 3 Tirain Tiest x Tirain
Num. obj. Vid. La
# obj.
Vid. Le
# obj. Vid. La
3 Balls
1
2.6
6.5
1.7=09 8.5459
1.64ax 8.5
3,1
4.6148
2.9-08 10.4176
2,910s 11.0is0
5 Balls
1
3.3
8.9
2,0=10
10.2
3,1406 1.8
5.5
6.6262
5.12ta
13.0295
5.1410 13.5
14


Reviews:
[{"review_id": "r1ewN1AsYH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "UPDATE: I appreciate the authors' discussions and qualitative results. My main original concern was that the empirical evaluation only studies a single type of situation of inferring physical parameters. Given that the authors claim that the proposed method infers \"on the fly\" physical properties, I would expect that the authors demonstrate the the method works on other physical properties as well beyond just one setting. Because this concern was not sufficiently addressed, I maintain my original rating.\n\n----\nSummary: This paper tackle the question of learning to infer physical parameters of a novel physical scenario with which to predict future motion of objects. In particular, the authors propose a meta-learning framework for inferring parameters of physical objects from a few video observations of physical scenario, with the constraint that physical parameters are not apparent from appearance alone. The authors evaluate the approach on a domain where the learner needs to infer whether balls should pass above, pass below, or bounce off obstacles.\n\nRecommendation: Borderline. The main limitation of the paper is that the empirical evaluation only studies a single type of situation of inferring physical parameters (whether balls should pass above, pass below, or bounce off obstacles) but does not consider other situations for inferring physical parameters for prediction in a new domain. This paper has great potential, but the lack of evaluation on a wider variety of physical phenomena makes it difficult to evaluate the generality of the claims made by this paper. I would highly consider increasing my score if the authors would be able to provide a thorough evaluation of two more types of physical phenomena.\n\nResearch Problem: The research problem this paper tackles is to learn a model of intuitive physics that can learn \"on the fly\" physical properties specific to new environments.\n\nApproach: A meta-network compresses a set of video observations of a scenario using the dynamic image encoding. The meta-network produces context parameters m and a that parameterize the scenario-specific predictor \\hat{Phi} and conditional generator g respectively. The state x is represented with a two-dimensional array. The authors use a perceptual loss as an image reconstruction metric. The author also enforce that the predictions of the state-space predictor and the encoder eta should provide close predictions.\n\nStrengths\n- The meta-learning formalization is an interesting and intuitive contribution.\n- The experiments provide a proof-of-concept of the efficacy of the framework with thorough analysis.\n\nWeaknesses\n- While the work is well-motivated and the experiments provided show a proof-of-concept of the approach with thorough analysis, the paper could be significantly strengthened by considering more domains for inferring physical parameters - simply inferring whether balls will pass above, pass below, or bounce off obstacles is a good first step, but does not provide enough evidence to evaluate the generality of the proposed method to intuitive physics tasks. This I believe is the main limitation of the paper. \n\nQuestions\n- Would the authors be able to provide an empirical study (with qualitative analysis) or theoretical justification to explain the reason why the method can generalize to scenarios with more objects than observed during training? Prior work that does show generalization to more objects explicitly build in the locality constraint (e.g. van Steenkiste 2018) through the network architecture, so it is interesting to see that the proposed method also can generalize similarly.\n\nVan Steenkiste, S., Chang, M., Greff, K., & Schmidhuber, J. (2018). Relational neural expectation maximization: Unsupervised discovery of objects and their interactions. arXiv preprint arXiv:1802.10353.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SygSlxlNtr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary \nThe paper proposes an architecture for few-shot video prediction in which a number of videos are summarized through global pooling operations and passed into a video predictor that learns to leverage them for adaptation, similar in spirit to the RNN-based meta-learning approaches such as Santoro’16, Duan’16. Due to global image and feature pooling operations, the proposed approach is computationally efficient. Proof-of-concept experiments are presented in a simple simulated physical prediction setting. It is claimed that the proposed model achieves generalization to longer sequences and larger board sizes, as well as a larger number of objects.\n\nDecision\nThe work considers an interesting task and shows promise, but the paper lacks a substantial contribution at this time. I recommend weak reject. To improve the paper, I recommend considering harder prediction tasks and lifting some of the restrictions of the method.\n\nPros\nThe paper proposes and evaluates a model for few-shot video prediction, which is an interesting and well-motivated task. \n\nCons\n1. The method is only evaluated on a toy task. While a model is proposed that can successfully solve the presented toy task, certain assumptions, such as averaging directly in the image space, may prevent the findings of this paper to scale to more complex tasks, which demands further investigation.\n\n2. It is claimed that the method generalizes to longer sequences and larger board sizes, however, the performance of the method seems to quickly deteriorate in both settings. Authors present anecdotal evidence that the method generalizes to larger number of objects, however, quantitatively the results again are much inferior to the method trained on a larger number of objects. While the presented qualitative generalization findings are interesting, they seem limited in scope, especially when taking into account the simplicity of the considered data.\n\nMinor comments\n1. On page 3, the paper states “We are not the first to consider a similar learning problem, although most prior works do require some form of external supervision.” While there is a substantial body of literature that uses extra supervision, the following foundational works do not: Finn et al, 2016a,b; Fragkiadaki, 2016. In addition, “a similar learning problem” is considered without supervision in many papers on video prediction, including: Ranzato et al., 2014; Srivastava et al., 2015; Mathieu et al., 2015; Denton et al., 2017, 2018; Villegas et al., 2017, 2019; Wichers et al., 2018; Castrejon et al., 2019. While it might not be necessary to cite all of these as they do not necessarily apply their models to intuitive physics, the statement in the paper seems rather unsubstantiated.\n2. A similar meta-learning prediction problem is addressed in Nagabandi’19a,b, which are not cited.\n3. The paper is somewhat cluttered with notation, which overall hampers the flow for the reader. The paper would benefit from slight reorganization to improve the flow. This could be done e.g. be structuring the paper to first present the computational task that it addresses, namely few-shot video prediction, and how each component of the model works to address that problem, and only later go into the detail of how the data were collected and how each individual module was implemented\n\n\nSantoro et al, Meta-learning with memory-augmented neural networks.\nDuan et al, RL2: Fast reinforcement learning via slow reinforcement learning.\nNagabandi et al, LEARNING TO ADAPT IN DYNAMIC, REAL-WORLD ENVIRONMENTS THROUGH META-REINFORCEMENT LEARNING,\nNagabandi et al, DEEP ONLINE LEARNING VIA META-LEARNING: CONTINUAL ADAPTATION FOR MODEL-BASED RL\n\n\n--------------------- Update 11.19 -----------------------\n\nI appreciate the newly provided qualitative generalization results (in response to R3). However, it is clear that the network has trouble maintaining the exact number of spheres in the scene, especially after collisions or interactions with obstacles. While this generalization finding is undoubtedly interesting, I am not convinced it is enough for a publication in ICLR.\n\nFurthermore, to support the authors' claim that the prediction performance on long sequences is satisfactory, I suggest comparison with a hand-crafted baseline, such as one suggested by R1, or one that makes use of explicit physics.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "Hke-9ZE-tH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nPaper summary:\n\nThe paper proposes a method to predict the future trajectory of a ball (or a set of balls) given the first few frames of the trajectory and a set of experience runs in the same environment. The model first learns to convert the set of images to a set of corresponding heatmaps that encode the location of the ball. Then, a recurrent network generates the future states given the first few locations and the output of a network that learns abstract information from the experience runs. The network is trained using reconstruction and perceptual similarity loss.\n\nPaper strengths:\n\nIt is interesting that the model is able to learn from a set of experience runs. Learning abstractions from a set of experience runs is an interesting direction.\n\nPaper weaknesses:\n\nThe paper has two main problems. (1) The model and the experiments are designed for a very simplistic scenario. I do not think that machinery is really needed to perform prediction in this simple setting. (2) The paper needs major re-rewriting. Some of the main parts of the paper are not clear. Please refer to the comments below for more details.\n\n- It is unclear how the initialization function (Equation 4) is learned. It is very unlikely that the auto-encoder produces a heatmap of the object locations only based on the reconstruction loss that it receives at the end. This should be clarified.\n\n- Equation 2 is very confusing. Is \\phi-hat a function of experience I(E) or I_{0:T}(R). It seems \\phi-hat produces an image that is compared with I(R), but it doesn't seem that M produces an image.\n\n- The proposed machinery is overkill for the very simple experimental setup. A nearest neighbor baseline would probably perform as well. We can extract a set of frames in the experience runs that are closest to the first few frames using some simple distance functions. Then, we can predict the future movements by copying the rest of the trajectory from the experience run.\n\n- The only part that seems unsupervised is the location of the ball, which can be easily obtained by image subtraction in this simple setting. What else is unsupervised in the proposed approach?\n\n- The beginning of section 3.2 mentions w, but there is no mention of it afterwards.\n\n- Why does the backprop for sixty 64x64 images require 12Gb of memory? It would be good to explain that.\n\n- Regarding the evaluation, how does it know which prediction corresponds to the which groundtruth to compute the distance that is mentioned? The prediction can correspond to any of the balls since there is no explicit notion of object in the model.\n\nDue to the issues mentioned above, especially extremely simple experimental setup and lack of clarity, I chose \"Weak Reject\". \n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: While the reviewers found the paper interesting, all the reviewers raised concerns about the fairly simple experimental settings, which makes it hard to appreciate the strengths of the proposed method. During rebuttal phase, the reviewers still felt this weakness was not sufficiently addressed.
Decision: Reject