Learning Underlying Physical Properties From Observations For Trajectory Prediction
Authors: Ekaterina Nikonova
Jochen Renz
Venue:  (2020)
Keywords: Physical Games
Deep Learning
Physical Reasoning
Transfer of Knowledge

Abstract: In this work we present an approach that combines deep learning together with
laws of Newton’s physics for accurate trajectory predictions in physical games.
Our model learns to estimate physical properties and forces that generated given
observations, learns the relationships between available player’s actions and estimated
physical properties and uses these extracted forces for predictions. We
show the advantages of using physical laws together with deep learning by evaluating
it against two baseline models that automatically discover features from
the data without such a knowledge. We evaluate our model abilities to extract
physical properties and to generalize to unseen trajectories in two games with a
shooting mechanism. We also evaluate our model capabilities to transfer learned
knowledge from a 2D game for predictions in a 3D game with a similar physics.
We show that by using physical laws together with deep learning we achieve a better
human-interpretability of learned physical properties, transfer of knowledge to
a game with similar physics and very accurate predictions for previously unseen
data.

Under review as a conference paper at ICLR 2020
LEARNING UNDERLYING PHYSICAL PROPERTIES FROM OBSERVATIONS FOR TRAJECTORY PREDIC-
TION
Anonymous authors Paper under double-blind review
ABSTRACT
In this work we present an approach that combines deep learning together with laws Newton's physics for accurate trajectory predictions physical games. Our model learns to estimate physical properties and forces that generated given observations, learns the relationships between available player's 's actions and es- timated physical properties and uses these extracted forces for predictions. We show the advantages of using physical laws together with deep learning by eval- uating it against two baseline models that automatically discover features from the data without such knowledge. We evaluate our model abilities to extract physical properties and generalize unseen trajectories in two games with a shooting mechanism. We also evaluate our model capabilities to transfer learned knowledge from a 2D game for predictions in 3D game with a similar physics. We show that by using physical laws together with deep learning we achieve bet- ter human-inte interprttttttrt of learned physical properties, transfer knowledge game with similar physics and very accurate predictions for previously unseen data.
INTRODUCTION
Games that follow Newton's laws of physics despite being a relatively easy task for humans, remain to be a challenging task for artificially intelligent agents due to the requirements for an agent to understand underlying physical laws and relationships between available player's actions and their effect in the environment. In order to predict the trajectory of physical object that was shot using some shooting mechanism, one needs to understand the relationship between initial force that was applied to the object by mechanism and its initial velocity, have knowledge of hidden physical forces of the environment such as gravity and be able to use basic physical laws for the prediction. Humans, have the ability quickly learn such physical laws and properties of objects in given physical task from pure observations and experience with the environment. In addition, humans tend to use previously learned knowledge in similar tasks. was found by researchers human psychology, humans can transfer previously acquired abilities and knowledge to a new task if the domain of the original learning task overlaps with the novel one (Council 2000).
The problem of learning properties of the physical environment and its objects directly from ob- servations and the problem of using previously acquired knowledge similar task are important to solve in Al as this is one of the basic abilities of human intelligence that humans learn during infancy (Baillargeon) 1995). Solving these two problems can bring AI research one step closer to achieving human-like like superhuman results in physical games.
In this work we explore one of the possible approaches to these two problems by proposing a model that is able to learn underlying physical properties of objects and forces of the environment directly from observations and use the extracted physical properties in order to build a relationships between available in game variables and related physical forces. Furthermore, physical knowledge in order to accurately predict unseen objects trajectories in games that follow Newtonian physics and contain some shooting mechanism. We also explore the ability of model
model then uses learned

---

Under review as a conference paper at ICLR 2020
to transfer learned knowledge by training model in 2D game and testing it in 3D game that follows similar physics with no further training.
Our approach combines modern deep learning techniques (LeCun et al. 2015) and well-known physics laws that were discovered by physicists hundreds of years ago. We show that our model automatically learns underlying physical forces directly from the small amount of observations, learns the relationships between learned physical forces with available in-game variables and uses them for prediction of unseen object's trajectories. Moreover, we also show that our model allows us to easily transfer learned physical forces and knowledge to the game with similar task.
In order to evaluate model abilities infer physical properties from observations and predict unseen trajectories, we use two different games that follow Newtonian Physics. The first game that we use testing environment for our model is Science Birds. Science Birds a clone of Angry Birds a popular video game where the objective is to destroy all green pigs by shooting birds from a slingshot. The game proven to be difficult for artificially intelligent playing agents that use deep learning and many agents have failed to solve the game in the past (Renz et al.) 2019). The second
game that we are using as our testing environment is Basketball 3D shooter game. In this game the objective of a player is to shot ball into basket.
In order to test the abilities of our model transfer knowledge to a different game we first train our model on small amount of shot trajectories from Science Birds game and then test trained model for predictions the ball trajectory in the Basketball 3D shooting game.
We compare the results of our proposed model that is augmented with physical laws against a two baseline models. The first baseline model learns to automatically extract features from observations without knowledge of physical laws, whereas the second baseline model learns to directly predict trajectories from the given in-game variables.
2 RELATED WORK
Previous AI work in predicting future dynamics of objects has involved using deep learning ap- proaches such as: graph neural networks for prediction of interactions between objects and their future dynamics (Battaglia et al. (2016); Watters et al.] (2017); (Sanchez-Gonzzlez et al. (2018). Bidirectional LSTM and Mixture Density network for Basketball Trajectory Prediction (Zhao et al. (017 and Neural Physies Engine (Chang al. 2016)). Some of the researchers also tried to com- bine actual physical laws with deep learning. In one of such works, researchers propose a model that learns physical properties of the objects from observations (Wu et al. (2016)). Another work proposes to integrate a physics engine together with deep learning to infer physical properties (Wul al. (2155)
However, most of the work on predicting future objects dynamics is focused on learning physics from scratch uses some known physical properties in order to train a model. This could be problem as in most real-world physical games the underlying physical properties are not known to the player unless one has an access the source code of physics engine. Because of that, these properties have to be learned directly from experience with the environment without any supervision on the actual values of physical properties. Another important point, is that instead of learning physics from scratch we can use to our benefit already discovered and well-established laws of physics.
In this work we propose an approach that combines classical feedforward networks with well-known physical laws in order to guide our model learning process. By doing so, our model learns physical properties directly from observations without any direct supervision on actual values of these proper- ties. Another contribution is that our model leams from a very small training dataset and generalizes well to the entire space. Furthermore, learned values can be easily interpreted by humans which allows us to use them in any other task in the presented test domains and can be easily transferred to other games with similar physics.
2

---

Under review as a conference paper at ICLR 2020
3 APPROACH
3.1
BASELINE MODELS
In order measure the advantages of combining classical physical laws together with deep learning we are comparing our model against pure deep learning approaches with similar architectures.
3.1.1 ENCODER BASELINE MODEL
Besetiee NooM Without Encode
Decide
Trajedbriiss
Figure 1: Left: First baseline model. The first neural network consists of encoder and decoder. En- coder takes in trajectories and encodes them into latent space. Decoder then takes the latent feature representation and decodes them back trajectories. Loss calculated using MSE and propagated back to both encoder and decoder. The second neural network receives in-game variables such as relative position values then serve input the decoder which predicts the actual trajectories. For the prediction of unseen trajectories only the second neural network and the decoder are used. Right: Second baseline model. This model does not observed trajectories and directly learns to predict the trajectories from initial forces or relative position of the physical object to the shooting mechanism.
bird or initial forces and learns to predict values predicted by encoder. These
Our first baseline model is based on the idea of autoencoders (Rumelhart al. (1866). Contrary to the proposed model in section 3.2 this model learns to automatically discover features from obser- vations. takes sequence of points T' { 3o), (I1, y1 vi);... (I yn)} as its input and encodes it to a latent space Texe- The encoded trajectory is then used by decoder to reconstruct the trajectory. The second part of this baseline model consists of another MLP that learns associate a relative position of a physical object that generated the trajectory with learned latent space Tene- More for- mally, given trajectory 7 input, encoder feneader, and decoder fdccnders our model reconstructs trajectory T from latent space as follows:
f ddecoder (Jencoder (((10, Ho), (II, 1i),,,, (In, (n)3))
Once the trajectory reconstructed, we compute loss using Mean Squared Error and update the weights of our networks:
n T 2T2
n
The second part of this baseline model a another MLP fassociate that learns to associate given ini- tial relative position of physical object (fro:Mrg) with derived in a previous step encoded trajectory Tenci
Tene fassociate(II,) Vr.))
(3)
In order to update the weights of fassociate we compute the loss using Mean Squared Error between two derived encodings T Tene and Texe-
After that, we predict trajectory using derived Tene as follows: f Sdecoder (Ténc)
3.1.2 SIMPLE BASELINE MODEL.
In order to evaluate the advantages using observations and an encoder-decoder scheme, we use the second baseline model that does not encoder-decoder and directly learns predict trajectory
3

---

Under review as a conference paper at ICLR 2020
from the given in game forces or relative position of physical object. More formally, given relative position of a physical object (Zra yro) and MLP feimple, we compute the trajectory as follows:
π = Ssimpte ((Fro: Uro))
(4)
3.2
PHYSICS AWARE MODEL
InterNet
ResatoNet
MEP
Projetile
Fouetion
foroes
NDP
Figure 2: Left: InferNet internal architecture. MLP predicts single value that represents gravity, while MLP 2 predicts a batch of values that represent initial velocities Vo and angles for each trajectory the input batch. These values are then inserted to the projectile motion equation to calculate the trajectory of the object. The predicted trajectory is then used to calculate the loss and update weights both networks. Right: RelateNet internal architecture. MLP I predicts a batch of values that represent initial velocities Vo, while MLP 2 predicts batch of values that represent angles a for each trajectory in the input batch. The predicted values are then used to calculate the loss and update the weights of both networks.
Similarly to the first baseline model presented section 3.1.1 Physics Aware Network (PhysANee) consists of parts: neural network that discovers physical forces of the environment and action that generated given observations and a neural network that learns the relationship between the in- game actions or forces and predicted physical values. We further refer to these two parts as InferNet and RelateNet.
3.2. INFERNET
The goal of InferNet is to extract physical forces that have generated given trajectory {(z0, Mo), (#1, (I.... }n)} using guidance from known physical equations. The discovered phys- ical forces are then plugged to the projectile motion equation in order to calculate the trajectory.
InferNet consists of two internal small MLPs that are trained together as shown on Figure 2 (Left). The first MLP takes in batch of trajectories and predicts single value of gravity foree for all of them. The second MLP takes in batch of trajectories and for each trajectory in a batch it predicts an initial velocity and angle of shot. These predicted values are then inserted into projectile motion equation in order to calculate the resulting trajectory. The projectile motion equation is defined as follows (Walker (2010))
gr
y tan (0)
(5)
2V3 (0)
In equation 5 is the initial height, g is gravity, 0 is the angle a shot and Vo is initial velocity.
Once it had calculated the trajectory compute the loss between observed trajectory T and pre- dicted trajectory / using Mean Squared Error in similar way as defined in equation 2
4

---

extracted physical forces and properties Under review as a conference paper at ICLR 2020
3.2.2 RELATENET
The goal of RelateNet that is shown Figure (Right) is learn the relationship between in- game variables and physics forces predicted by InferNet. This network tries to predict extracted by InferNet forces directly from the given in in-game values such as relative position of a bird. The in grgaame variables can be any variables with continuous or discrete domain that can be chosen by the playing agent in order to make shot. As an example, in-game variables can the initial forces of the shot or object's relative position to the shooting mechanism. RelateNet consists of two internal MLPs where first MIPP predicts initial velocities and second MI.P predicts initial angles. In order to update the weights of both internal MLPs, we calculate the MSE between values predicted by InferNet and values predicted by RelateNet. More details on the architecture of the PhysANet can be found in the Appendix A
4 EXPERIMENTS
In experiments using Science Birds (Ferreira 2018 and Basketball 3D shooter game [Nagori 2017) as a testing environments.
Figure 3: Slingshot in Science Birds game and Basketball 3D shooter game.
4.1 SCIENCE BIRDS TRAJECTORY PREDICTION
Science Birds is a clone of Angry Birds which has been one of the most popular video games for period of several years. The main goal of the game is to kill all green pigs on the level together with applying as much damage as possible to the surrounding structures. The player is provided with sequence of (sometimes) different birds to shoot from slingshot. In Science Birds, similarly to Angry Birds all game objects are following the laws of Newton's Physics (in 2D). In order to predict the outcomes of the shots, the player needs understand basic physical laws that govern the game and be able to use them for a prediction
bird's trajectory.
Science Birds contains slingshot (Figure which serves shooting mechanism that allows player to shoot bird. Player can select a strength and the angle of the shot by changing a relative position of the bird to the center of slingshot. order accurately predict the trajectory of a bird, player needs to have an understanding of the relationship between relative position of the bird to the slingshot and resulting trajectory. The underlying physical properties of the game such gravity, or initial velocity of related shot or its angle is unknown to the player and has to be estimated from the observations and experiences.
The goal of our model in this experiment is to learn the relationship between relative position of bird and initial velocity and angle of the resulting shot in order to predict trajectories of previously unseen shots. In this experiment we do not provide actual physical properties or forces to our model and it has to learn them directly from observations.
In order to train our model for this experiment, we have collected small data set of 365 trajectories and relative positions of the bird that generated these trajectories from Science Birds. This data set was then split to train, test and validation data sets, resulting in training data set that contains only 200 trajectories. We are using such small training data set in order to prevent a model to simply memorize all possible trajectories and instead learn to predict unseen trajectories.
During the training of model, PhysANet takes in trajectory input and estimates the angle 8 initial velocity Vn and gravity g that generated this trajectory. The predicted values are then inserted projectile motion equation (5) in order recreate the trajectory. As second step, the relative

---

Under review as a conference paper at ICLR 2020
position of the bird to the slingshot (Io, yo) is fed as input to the RelateNet which learns to predict values 8 and Vo predicted by InferNet.
During the testing of model, we only feed the relative position of the bird to RelateNet which predicts values and Vo that are plugged in to projectile motion equation in order to calculate the trajectory.
4.2 BASKETBALL 3D SHOOTER GAME
Basketball 3D shooter is a game where the player has throw ball to the basket in order to earn points. In this game the shooting mechanism is different compared to Science Birds and the initial force that applied to the physical object does not depend on the relative position of that object.
In this experiment we are interested in the ability of our model to transfer the learned relationships and physical properties from one game with shooting mechanism to the other. In partieular, we were interested in ability of our model to use learned knowledge from Science Birds for predictions of the ball's trajectory in Basketball 3D Shooter. Because of the absence the slingshot like mechanism in this game, we train models to predict the trajectories based on the initial force that is applied to the bird or a ball by a shooting mechanism when launched.
As in the Science Birds experiment described in section 4.1 the input to the InferNet is trajectory (x. }n)} and an input to the RelateNet is initial force that is applied to the bird
{ (T1, 31 or basketball ball by a shooting mechanism. We note here that the magnitude of the initial force applied to the ball in Baskethall 3D Shooter game is higher than the magnitude of the initial force applied to the bird in Science Birds.
4.3 TESTING DATASETS
In order to evaluate our model we are using three testing datasets: Test, Generalization and Basket- ball. Test dataset contains trajectories generated by previously unseen (by the model) initial forces or relative positions opposite direction of shots in the training set. In particular, it contains trajectories and the related relative positions of the bird of shots to the left side of a slingshot. The basketball dataset con- tains trajectories and related to them initial forces that were applied the ball in the Basketball 31) shooter game.
bird. Generalization dataset contains trajectories of the shots made to the
These three datasets are not exposed to our models during the training. In the Basketball experiment described in section 4.2] we do not train our models on the basketball dataset but only on the training dataset from Science Birds.
5 RESULTS
5.1
PREDICTION OF TRAJECTORIES IN SCIENCE BIRDS
The results of our first experiment show that PhysANet has the most accurate trajectory prediction of all tested models. PhysANet was able learn associate position of a bird relative to the slingshot with initial velocity and angle of a shot as shown on Figure 4 In particular Figure 4 shows two learned relationships between position of bird before is shot from slingshot and initial parameters of the shot. The graph on the left side of Figure shows how the initial angle between bird and center of a slingshot affects the initial angle of a trajectory. As an example from this graph we can see that bird positioned at angle of 220 degrees would result in trajectory that has initial angle of roughly 45 degrees which as we know from a trigonometry
close to the "truth".. The graph on the right side of Figure shows how initial velocity depends on the distance between initial bird position and center of the slingshot. bird an angle of 225 relatively to the center of slingshot and only changed the distance. From this graph we can see that as one could expect the more we extend the slingshot the stronger our shot is going to be and the higher initial velocity will be.
order to show this relationship we have fixed the
In Figure we present a few examples of the trajectory predicted by PhysANet and actual trajectories that bird took. The presented examples show three trajectories from test dataset which the network

---

Under review as a conference paper at ICLR 2020
300
130
Figure 4: Learned relationships between velocities and angle and relative position of bird. Graph on the left side shows how initial angle between bird and center of a slingshot related to the initial angle of the trajectory. Graph the right side shows how initial velocity is changing when increas- ing the distance between bird and center of the slingshot. In this example angle between bird and slingshot was fixed at 225 degrees.
Figure 5: Predicted and actual trajectories of bird. Red line represent a predicted trajectory by PhysANet, while blue line represent true trajectory taken by the bird. The first three trajectories are from test data set, whereas the last one is from generalization dataset.
had never seen before and one trajectory the shot to the left side of a slingshot. Despite the fact that our training dataset did not contain the shots to the left side of the slingshot, our model shows a good generalization ability and predicts the trajectory quite accurately. This shows that learned physical properties and relationships between position of a bird and slingshot can be used by PhysANet to accurately predict the trajectories.
PhysANet showed significantly better overall accuracy test and generaliza- tion datasets than our two baseline models.
As shown in Table I
5.2 TRANSFER OF KNOWLEDGE TO BASKETBALL 3D
The results in this experiment show that learned physical properties and relationships in Science Birds can be transferred to the Basketball 3D shooter game. Due to the fact that model was trained on data from a 2D environment, we tested the predictions of 3D trajectories by separating prediction process into predictions in x.y and z,y planes. Surprisingly, despite being trained in 2D envi- ronment, PhysANet showed good results predicting the trajectories both planes without any additional training in new 3D environment.
Figure 6: Comparison of model's abilities to transfer learned knowledge to Baskethall 3D. The first two pictures show predictions of one trajectory in x.y and z.y planes respectively. The of the pictures show predictions only in x.y plane. The first three pictures on the left show trajectories predicted by PhysANet, fourth picture shows trajectory predicted by BMI, the last picture shows a trajectory predicted by BM2.

---

Under review as a conference paper at ICLR 2020
Table 1: Average observed Mean Squared Error between actual trajectory and predicted trajectory.
DATASET
PhysANet MSE
BMT MSE
BM2 MSE
Test
0.324
0.554
0.832
Generalization
0.837
1.637
2.162
Basketball 3D
4.051
8.196
7.676
The Figure shows the examples of predictions of trajectories from the Basketball dataset by PhysANet, BMI and BM2. The first three pictures show trajectories predicted by PhysANet, fourth picture shows prediction made by BMI and the last picture shows prediction made by BM2. The first two pictures show predictions for one trajectory in x.y and z.y planes. The rest of the pictures show
shown in Figure show predictions in x.y plane only. As we can see from Figure [6] despite the fact that initial forces in Basketball 3D shooter game have higher magnitudes, the PhysANet was able to correctly handle it and still predict trajectories in a correct shape and have a relatively low accuracy error. This is contrary to the two baseline models that could not handle new environment properly and predicted seemingly random trajectorics despite being relatively accurate in the predictions in the first experiment described in section
The results presented in Table ] show the comparison of the observed Mean Squared Errors of all three models. These results show that PhysANNet surpasses two baseline models in all testing datasets. Thus for example, error shown by PhysANet in Basketball dataset is nearly two times lower than error shown by Baseline Model (BM1) which was described in section [.1.T Sur- prisingly, despite the fact that BM1 showed better results in Test and Generalization datasets than BM2, showed higher prediction error in the Basketball 3D dataset than simple baseline model (BM2). Despite the fact that PhysANet showed relatively good results in predicting trajectories in previously unseen game there still seems to be some loss the accuracy of the predictions. We hypothesise that the loss of the accuracy can be caused by the differences in physics engines of the two games. One of the possible solutions of this problem is to improve our model abilities to adapt to the new environment after observing few shots, however we leave such an improvement for future research.
DISCUSSION
In this work we have showed that combining known physical laws with deep learning can be bene- ficial for learning more human-interpretable and flexible physical properties directly from the small amount of the observations. We showed that a model that is augmented with physical laws achieved unseen trajectories, was able to transfer learned knowledge game with similar physics and made generally more accurate predictions of trajectories than models without such physical knowledge. Our results show that in physical laws and equations can be beneficial and can guide deep learning model to learn physical properties and relationships that lead to more accurate predictions than predictions made by models that learned features automatically without any supervision. Because learned values can be easily interpreted by human these values can be used in potentially new tasks. As an example, learned gravity of the environment can be used for prediction of physical behaviour of other physical ob- jects the environment. Another important quality of our approach is that learned knowledge can be transferred to task with similar physics as was shown by the Basketball experiment. Lastly, our model can be easily adopted to any other physical task by changing or adding more physical equations and learning different physical properties from the observations. This potentially could allow us to fully master the physics of a game and use learned knowledge as part of artificially intelligent agent in order to achieve human-like performance in physical games.
better generalization
different
situations using already discovered
Our results show that it could be beneficial to use physical laws that were discovered by physicists hundreds of years ago together with deep learning techniques in order unleash their full predict- ing power and bring artificially intelligent agents a step closer to achieving human- -ike physical reasoning.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Rene Baillargeon. Physical reasoning in infancy. Advances in infancy research, 9, 01 1995.
Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray Kavukcuoglu, Interaction networks for learning about objects, relations and physics. CoRR, abs/1612.00222. 2016.
Michael B. Chang, Tomer Ullman, Antonio Torralba, and Joshua B. Tenenbaum. A compositional object-based approach learning physical dynamics. CoRR, abs/1612.00341, 2016.
National Research Council. How People Learn: Brain, Mind, Experience, and School: Expanded Edition. The National Academies Press, Washington, DC, 2000.
Lucas N. Ferreira. Science birds. 2018.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553),436-4444 2015. ISSN 0028-0836. doi: 10. (038/nature145333
Nikhil Nagori.
Basketball 3d
shooter.
http:://codepprojjettssoo terniitt-engggnee ne-neewwee wow.ch--ooocc--ooeoooo 2017.
basketball-a cehhotter--ae-iiine
Jochen Renz, Xiaoyu Ge, Matthew Stephenson, and Peng Zhang. Ai meets angry birds. Nature
Machine Intelligence, 1:328, 07 2019. doi: 10. 1..10///2226-011-00772
D. E. Rumelhart, G. Hinton, and R. Williams. Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. chapter Learning Internal Representations by Error Propagation, pp. 318-362. MIT Press, Cambridge, MA, USA, 1986. ISBN 0-262-68053-X.
Alvaro Sanzaez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin A. Ried- miller, Raia Hadsell, and Peter W. Battaglia. Graph networks learnable physies engines for inference and control. CoRR, abs/1806.01242. 2018.
James S. Walker. Physics with Mastering Physics, 4th Edition. Pearson, Western Washington University, 2010.
Nicholas Watters, Andrea Tacchetti, Theophane Weber, Razvan Pascanu, Peter W. Battaglia, and Daniel Zoran. Visual interaction networks. CoRR, htss/7066001333 2017.
Jiajun Wu, Ilker Yildirim, Joseph Lim, Bill Freeman, and Josh Tenenbaum. Galileo: Perceiv- ing physical object properties by integrating a physics engine with deep learning. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (eds.). Advances in Neural Information Processing Systems 28, pp. 127-135. Curran Associates, Inc., 2015.
Jiajun Wu, Joseph J. Lim, Hongyi Zhang, Joshua B. Tenenbaum, and William T. Freeman. Physics 101: Learning physical object properties from unlabeled videos. In BMYC, 2016.
Yu Zhao, Rennong Yang, Guillaume Chevalier, Rajiv Shah, and Rob Romijnders. Applying deep bidirectional LSTT and mixture density network for basketball trajectory prediction. CoRR, abs/1708.05824, 2017.
A APPENDIX
B PHYSANET ARCHITECTURE
As was mentioned in section 3.2 PhysANet has two separate neural networks: InferNet and Re-
lateNet.
InferNet consists two neural networks (MLPI and MLP2) each with a single hidden layer of size 200 and 100 respectively. MIP1 takes in trajectories consisting of most 25 points. Trajectories

---

using ReLU activation function and activation function for the last Under review as a conference paper at ICLR 2020
with less than 25 points padded with zeros, and trajectories with more than 25 points cut. Trajectories and related in gage variables are fed to the network in batches of size
RelateNet also consists of two neural networks (MLP and MLP 2) each with single hidden layer of size 100. MLP and MLP 2 both take in the in-game variables in batches of size 8.
For all hidden layers layer.
In order to train PhysANet. first train InferNet until it converges value of a loss being close to zero and after that we train RelateNet. Simultancous training of both networks is also possible, however in experiments training RelateNet after InferNet showed better results.
10


Reviews:
[{"review_id": "ryxaWOnSKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The problem addressed by this paper is the estimation of trajectories of moving objects thrown / launched by a user, in particular in computer games like angry birds or basketball simulation games. A deep neural network is trained on a small dataset of ~ 300 trajectories and estimates the underlying physical properties of the trajectory (initial position, direction and strength of initial force etc.). A new variant of deep network is introduced, which is based on an encoder-decoder model, the decoder being a fully handcrafted module using known physics (projectile motion).\n\nI have several objections, which can be summarized by the simplicity of the task (parabolic trajectories without any object/object or object/environment collisions / interactions), the interest of the task for the community (how does this generalize to other problems?), and the writing and structuring of the paper. I will detail these objections further in the rest of the review.\n\nLearning physical interactions is a problem which has received considerable attention in the computer vision and ML communities. The problem is certainly interesting, but I think we should be clear on what kind of scientific knowledge we want to gain by studying a certain problem and by proposing solutions. The tasks studied by the community are mostly quite complex physical phenomena including multiple objects of different shapes and properties and which interact with each other. All these phenomena can be simulated with almost arbitrary precision with physics engines, and these engines are mostly also used for generating the data. In other words, the simulation itself is solved and is not the goal of this body of work. The goal is to learn differentiable models, which can be used as inductive bias in larger models targeting more general tasks in AI.\n\nCompared to this goal, the proposed goal is far too easy: learning projectile motion is very easy, as these trajectories can be described by simple functions with a small number of parameters, which also have a clear and interpretable meaning. The simplicity of the task is also further corroborated by the small number of samples used to estimate these parameters (in the order of 300). A further indication is the fact, that the decoder in the model is fully hardcoded. No noise modelling was even necessary, which further corroborates that a very simple problems is addressed.\n\nIn order words, I am not really sure what kind of scientific problem is solved by this work, and how this knowledge can help us to solve other problems, harder problems.\n\nMy second objection is with the written form of the paper. The paper is not well enough structured and written, many things are left unsaid. First of all, the problem has never been formally introduced, we don’t know exactly what needs to be estimated. What are the inputs, outputs? Is computer vision used anywhere? How are the positions of the objects determined if not with computer vision? How are the user forces gathered? What are “in game variables” mentioned multiple times in the document? No notation has been introduced, no symbols have been introduced (or too late in the document). For instance, there is no notation for the latent space of the encoder-decoder model.\n\nThe figures are not very helpful, as the labelling of the blocks and labels is very fuzzy. As an example, For InferNet, inputs and trajectories are “Trajectories”, so what is the difference? Of course we can guess that (inputs are measured trajectories, outputs are reconstructed trajectories), but we should not guess things when reading papers.\n\nThe figure for encoder-decoder model is very confusing, as the different arrows have different functional meanings and we have no idea what they mean. The outputs of the encoder and the MLP both point to the latent space and at a first glance the reader might think that they are concatenated, which raises several questions. Reading the text, we infer that first a model is trained using on one of the arrows (the one coming from the encoder) and ignoring the other one, and then the MLP is learned to reconstruct the latent space using the other arrow (the one coming from the MLP), but this is absolutely impossible to understand looking at the figure, which does not make much sense. We can infer all this from the text around equations (1) to (3), which is itself quite fuzzy and difficult to understand, in spite of the simplicity of the underlying maths.\n\nThe relationship of RelateNet and InferNet is not clear. While the task of InferNet is clear, the role of InferNet in the underlying problem is not clear and it has not been described how it interacts with RelateNet.\n\nIt is unclear how the transfer between science birds and basketball has been performed and what exactly has been done there.\n\nAs mentioned above, the role of “in game variables” is unclear. What are those? I suggest to more clearly define their roles early in the document and use terms from well-defined fields like control (are they “control inputs”) or HCI (are they “user actions”?).\n\nIn the evaluation section, we have baseline models BM1 and BM2, but they have never been introduced. We need to guess which of the models described in the paper correspond to these.\n\nThe related work section is very short and mostly consists of an enumeration of references. The work should be properly described and related to the proposed work. How does the proposed work address topics which have not yet been solved by existing work?\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "S1ecWacacH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an architecture that encodes a known physics motion equation of a trajectory of a moving object. The modeled equation has 3 variables and the network works in a latent space- contrary to taking raw images. It uses an auxiliary network (named InferNet) to train the final one used at inference time (named RelateNet). The former aims to reconstruct the input sequence of positions representing trajectory, and has intermediate 3 latent variables that correspond to the 3 variables of the modeled equation, while as decoder it uses the modeled known equation itself. The latter is a mapping from the relative position of the object to 2 latent variables of the former InferNet, and is trained with MSE loss. At inference, RelateNet takes as input the relative position of the object, predicts 2 variables of the equation and finally uses the motion equation to calculate the trajectory.\n\nIt is not easy for me to understand the use-case of the proposed method. In which real-world scenarios we would have the exact motion equation, and why given that we know such an equation we would want to learn a mapping from the relative position to a trajectory. In other words, it would be much more useful to learn the projectile motion equation itself. How does the proposed method handle input sequences which do not follow equation 5? To use this method do we need to know in advance the exact motion equation and its relevant ‘in-game variables’? In which cases would the former hold and in which cases would the latter be easy to obtain from raw pixels? Could the authors elaborate on it?\n\nIf I understand correctly, the trajectories (the input to InferNet) were generated with known $G$, $V_0$ and $\\theta$ (the 3 latent variables of InferNet). It is not clear to me why the authors don’t use these for the MSE loss used to train InferNet (rather than using the projectile motion equation). \n\nIn my opinion, the introduction and related work sections do not reflect what is proposed in the paper. As an example, paragraph 2 of the introduction refers to use-cases where we would like to learn dynamics that govern certain motions directly from observations, whereas the proposed method uses extracted positions as input, and handcrafts the motion equation. The third paragraph of page 2 mentions agents failing to solve a game with Newtonian physics, whereas the method in this paper does not demonstrate empirically a way that this architecture could be used by an agent.\n\n- Is the ‘projectile motion equation’ missing from Fig.2-right; is it used for inference? Is G from InferNet also input to RelateNet?\n\nIn summary, in my opinion, the technical novelty of this paper is limited as it uses MLP mappings that in some sense aim at learning the inverse of the equation that generated the data. Moreover, after reading the paper the use-case of the proposed method is not clear to me and the writing is unclear (see examples above and below).\n\n— Minor —\n- The term ‘in-game variables’ is used in a few places and is explained later in the text (Pg.5). I think that It would be helpful if it is explained in more detail the first time it is mentioned.\n- I don’t understand the second sentence of the abstract.\n- Pg1: build a relationships -> build relationships.\n- Pg2: I don’t understand what the authors mean by ‘clone of Angry Birds.’\n- Pg3: is $f_{associate}$ trained jointly or afterwards?\n- Pg4: was MSE the loss used for $f_{simple}$?\n- It would help adding sub-captions in Fig. 6.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HkxD3tnBKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method for predicting the trajectories of objects in video (or phone) games by learning the physical parameters underlying the movements. To do so, the authors used some multi-layer perceptrons on the (x,y) trajectory in order to i) estimate the physical values of the equation (equation supposed to be known, here a parabolic trajectory); and then ii) predict the trajectories from new initial conditions.\n\nGeneral comment:\nWhile the links between physics and machine learning is clearly interesting and trendy, I found the paper unclear, not well motivated and I think the work is not enough for a paper in ICLR. Yet, I think the authors did spend some time and this work might be suited for a workshop.\n\nPositive aspects:\n- the authors really tried to focus on games that are used today.\n- the results are showing that they do learn the parameters they wanted, as the new trajectories are indeed working.\n\nRemarks and questions:\n- the writing of the paper is not enough to make it clear, and a lot of sentences are not readable. Starting at the second sentence of the abstract. Try to make small sentences, and add all the pronouns 'a', 'the', .... Avoid the too numerous 'and' and cut the sentences.\n- What is a shooting mechanism? Is it something scientifically defined? otherwise, explain better: the parabolic trajectory is I guess more explanatory of what you are doing.\n- What is the goal of your work? Estimating the parameters of an equation used in a game is not really interesting.. as we have to know the equation, it has to be simple, we have to extract the trajectory from the game... but there might be other related applications that could motivate this work.\n- Related work: I don't really see the novelty of your work. You are saying that the physical properties have to be learned from experience, but you are actually relying on a known equation, and just tuning the 3 parameters. Do games always follow the physical laws? From my knowledge, some games change the physical laws in order to be more pretty, ect. in that case, your method will not work?\n- More generally, can you explain the difference for you between physical laws and physical properties?\n- Explain what is 'MLP'; how many layers, what size, how did you defined it, ect.\n- in 3.1.1. you are spending quite a long time in explaining what an autoencoder is; I think you can go faster on this.\n- Why the RelateNet has 2 distinct MLPs while InferNet uses the same for inferring V0 and Theta?\n- How did you extract the trajectories, ect. from the games?\n- Figure 6: How come in the two last images, we see different starting points?\n- Table 1: can we have an idea of the errors in meters? and compared to the distance of the trajectory? \n\nSmall remarks:\n- theta is not alpha. Please use a common notation.\n- Figure 5, 6.. : the legends are clearly not visible. they might not be useful, but in this case you have to spend time in changing the tick labels so that we can read them.\n- 'build a relationships': no 's'\n... a lot of  grammatical/ sentence problems\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper aims to estimate the parameters of a projectile physical equation from a small number of trajectory observations in two computer games. The authors demonstrate that their method works, and that the learnt model generalises from one game to another. However, the reviewers had concerns about the simplicity of the tasks, the longer term value of the proposed method to the research community, and the writing of the paper. During the discussion period, the authors were able to address some of these questions, however many other points were left unanswered, and the authors did not modify the paper to reflect the reviewers’ feedback. Hence, in the current state this paper appears more suitable for a workshop rather than a conference, and I recommend rejection.
Decision: Reject