Learning Underlying Physical Properties From Observations For Trajectory Prediction

1 INTRODUCTION . Games that follow Newton ’ s laws of physics despite being a relatively easy task for humans , remain to be a challenging task for artificially intelligent agents due to the requirements for an agent to understand underlying physical laws and relationships between available player ’ s actions and their effect in the environment . In order to predict the trajectory of a physical object that was shot using some shooting mechanism , one needs to understand the relationship between initial force that was applied to the object by a mechanism and its initial velocity , have a knowledge of hidden physical forces of the environment such as gravity and be able to use basic physical laws for the prediction . Humans , have the ability to quickly learn such physical laws and properties of objects in a given physical task from pure observations and experience with the environment . In addition , humans tend to use previously learned knowledge in similar tasks . As was found by researchers in human psychology , humans can transfer previously acquired abilities and knowledge to a new task if the domain of the original learning task overlaps with the novel one ( Council , 2000 ) . The problem of learning properties of the physical environment and its objects directly from observations and the problem of using previously acquired knowledge in a similar task are important to solve in AI as this is one of the basic abilities of human intelligence that humans learn during infancy ( Baillargeon , 1995 ) . Solving these two problems can bring AI research one step closer to achieving human-like or superhuman results in physical games . In this work we explore one of the possible approaches to these two problems by proposing a model that is able to learn underlying physical properties of objects and forces of the environment directly from observations and use the extracted physical properties in order to build a relationships between available in-game variables and related physical forces . Furthermore , our model then uses learned physical knowledge in order to accurately predict unseen objects trajectories in games that follow Newtonian physics and contain some shooting mechanism . We also explore the ability of our model to transfer learned knowledge by training a model in a 2D game and testing it in a 3D game that follows similar physics with no further training . Our approach combines modern deep learning techniques ( LeCun et al. , 2015 ) and well-known physics laws that were discovered by physicists hundreds of years ago . We show that our model automatically learns underlying physical forces directly from the small amount of observations , learns the relationships between learned physical forces with available in-game variables and uses them for prediction of unseen object ’ s trajectories . Moreover , we also show that our model allows us to easily transfer learned physical forces and knowledge to the game with similar task . In order to evaluate our model abilities to infer physical properties from observations and to predict unseen trajectories , we use two different games that follow Newtonian Physics . The first game that we use as a testing environment for our model is Science Birds . Science Birds is a clone of Angry Birds - a popular video game where the objective is to destroy all green pigs by shooting birds from a slingshot . The game is proven to be difficult for artificially intelligent playing agents that use deep learning and many agents have failed to solve the game in the past ( Renz et al. , 2019 ) . The second game that we are using as our testing environment is Basketball 3D shooter game . In this game the objective of a player is to shot a ball into a basket . In order to test the abilities of our model to transfer knowledge to a different game we first train our model on a small amount of shot trajectories from Science Birds game and then test trained model for predictions of the ball trajectory in the Basketball 3D shooting game . We compare the results of our proposed model that is augmented with physical laws against a two baseline models . The first baseline model learns to automatically extract features from observations without knowledge of physical laws , whereas the second baseline model learns to directly predict trajectories from the given in-game variables . 2 RELATED WORK . Previous AI work in predicting future dynamics of objects has involved using deep learning approaches such as : graph neural networks for prediction of interactions between objects and their future dynamics ( Battaglia et al . ( 2016 ) ; Watters et al . ( 2017 ) ; Sanchez-Gonzalez et al . ( 2018 ) ) , Bidirectional LSTM and Mixture Density network for Basketball Trajectory Prediction ( Zhao et al . ( 2017 ) ) and Neural Physics Engine ( Chang et al . ( 2016 ) ) . Some of the researchers also tried to combine actual physical laws with deep learning . In one of such works , researchers propose a model that learns physical properties of the objects from observations ( Wu et al . ( 2016 ) ) . Another work proposes to integrate a physics engine together with deep learning to infer physical properties ( Wu et al . ( 2015 ) ) However , most of the work on predicting future objects dynamics is focused on learning physics from scratch or uses some known physical properties in order to train a model . This could be a problem as in most real-world physical games the underlying physical properties are not known to the player unless one has an access to the source code of a physics engine . Because of that , these properties have to be learned directly from experience with the environment without any supervision on the actual values of physical properties . Another important point , is that instead of learning physics from scratch we can use to our benefit already discovered and well-established laws of physics . In this work we propose an approach that combines classical feedforward networks with well-known physical laws in order to guide our model learning process . By doing so , our model learns physical properties directly from observations without any direct supervision on actual values of these properties . Another contribution is that our model learns from a very small training dataset and generalizes well to the entire space . Furthermore , learned values can be easily interpreted by humans which allows us to use them in any other task in the presented test domains and can be easily transferred to other games with similar physics . 3 APPROACH . 3.1 BASELINE MODELS . In order to measure the advantages of combining classical physical laws together with deep learning we are comparing our model against pure deep learning approaches with similar architectures . 3.1.1 ENCODER BASELINE MODEL . Our first baseline model is based on the idea of autoencoders ( Rumelhart et al . ( 1986 ) ) . Contrary to the proposed model in section 3.2 this model learns to automatically discover features from observations . It takes a sequence of points T = { ( x0 , y0 ) , ( x1 , y1 ) , ... , ( xn , yn ) } as its input and encodes it to a latent space Tenc . The encoded trajectory is then used by decoder to reconstruct the trajectory . The second part of this baseline model consists of another MLP that learns to associate a relative position of a physical object that generated the trajectory with learned latent space Tenc . More formally , given trajectory T as input , encoder fencoder , and decoder fdecoder , our model reconstructs a trajectory T̂ from latent space as follows : T̂ = fdecoder ( fencoder ( { ( x0 , y0 ) , ( x1 , y1 ) , ... , ( xn , yn ) } ) ) ( 1 ) Once the trajectory is reconstructed , we compute the loss using Mean Squared Error and update the weights of our networks : 1 n n∑ i=1 ( T − T̂ ) 2 ( 2 ) The second part of this baseline model is a another MLP fassociate that learns to associate given initial relative position of a physical object ( xr0 , yr0 ) with derived in a previous step encoded trajectory Tenc : ˆTenc = fassociate ( ( xr0 , yr0 ) ) ( 3 ) In order to update the weights of fassociate we compute the loss using Mean Squared Error between two derived encodings Tenc and ˆTenc . After that , we predict trajectory using derived ˆTenc as follows : T̂ = fdecoder ( ˆTenc ) 3.1.2 SIMPLE BASELINE MODEL . In order to evaluate the advantages of using observations and an encoder-decoder scheme , we use the second baseline model that does not use encoder-decoder and directly learns to predict trajectory from the given in-game forces or relative position of a physical object . More formally , given relative position of a physical object ( xr0 , yr0 ) and MLP fsimple , we compute the trajectory as follows : T̂ = fsimple ( ( xr0 , yr0 ) ) ( 4 ) 3.2 PHYSICS AWARE MODEL . Similarly to the first baseline model presented in section 3.1.1 , Physics Aware Network ( PhysANet ) consists of two parts : a neural network that discovers physical forces of the environment and action that generated given observations and a neural network that learns the relationship between the ingame actions or forces and predicted physical values . We further refer to these two parts as InferNet and RelateNet . 3.2.1 INFERNET . The goal of InferNet is to extract physical forces that have generated a given trajectory { ( x0 , y0 ) , ( x1 , y1 ) ... ( xn , yn ) } using guidance from known physical equations . The discovered physical forces are then plugged to the projectile motion equation in order to calculate the trajectory . InferNet consists of two internal small MLPs that are trained together as shown on Figure 2 ( Left ) . The first MLP takes in a batch of trajectories and predicts a single value of gravity force for all of them . The second MLP takes in a batch of trajectories and for each trajectory in a batch it predicts an initial velocity and angle of a shot . These predicted values are then inserted into a projectile motion equation in order to calculate the resulting trajectory . The projectile motion equation is defined as follows ( Walker ( 2010 ) ) : y = h+ x tan ( θ ) − gx 2 2V 20 cos ( θ ) 2 ( 5 ) In equation 5 , h is the initial height , g is gravity , θ is the angle of a shot and V0 is initial velocity . Once it had calculated the trajectory we compute the loss between observed trajectory T and predicted trajectory T̂ using Mean Squared Error in a similar way as was defined in equation 2 . 3.2.2 RELATENET . The goal of RelateNet that is shown on Figure 2 ( Right ) is to learn the relationship between ingame variables and physics forces predicted by InferNet . This network tries to predict extracted by InferNet forces directly from the given in-game values such as relative position of a bird . The in-game variables can be any variables with continuous or discrete domain that can be chosen by the playing agent in order to make a shot . As an example , in-game variables can be the initial forces of the shot or object ’ s relative position to the shooting mechanism . RelateNet consists of two internal MLPs where first MLP predicts initial velocities and second MLP predicts initial angles . In order to update the weights of both internal MLPs , we calculate the MSE between values predicted by InferNet and values predicted by RelateNet . More details on the architecture of the PhysANet can be found in the Appendix A .

Summary: The problem addressed by this paper is the estimation of trajectories of moving objects thrown / launched by a user, in particular in computer games like angry birds or basketball simulation games. A deep neural network is trained on a small dataset of ~ 300 trajectories and estimates the underlying physical properties of the trajectory (initial position, direction and strength of initial force etc.). A new variant of deep network is introduced, which is based on an encoder-decoder model, the decoder being a fully handcrafted module using known physics (projectile motion).