A Bayesian-Symbolic Approach to Reasoning and Learning in Intuitive Physics

1 Introduction . Imagine a ball rolling down a ramp . If asked to predict the trajectory of the ball , most of us will find it fairly easy to make a reasonable prediction . Not only that , simply by observing a single trajectory people can make reasonable guesses about the material and weight of the ball and the ramp . It is astonishing that while the exact answers to any of these prediction and reasoning tasks requires an in-depth knowledge of Newtonian mechanics and solving of some intricate equations , yet an average human can perform such tasks without any formal training in physics . Studies suggest that from early age humans come to understand physical interactions with very limited supervision , and can 1Source code as well as training and testing data can be accessed at https : //bsp.xuk.ai/ . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . efficiently reason and plan actions in common sense tasks , even in absence of complete information ( Spelke , 2000 ; Battaglia et al. , 2013 ) . For example , with limited data , 4 or 5 years old children are capable of learning the physical laws behind magnetism ( Bonawitz et al. , 2019 ) . Physical reasoning is considered a core domain of human common-sense knowledge ( Spelke & Kinzler , 2007 ) . Recent studies suggest that the ability to efficiently learn physical properties and interactions with limited supervision is driven by a noisy model of Newtonian dynamics , referred to as the intuitive physics engine ( IPE ; Bates et al. , 2015 ; Gerstenberg et al. , 2015 ; Sanborn et al. , 2013 ; Lake et al. , 2017 ; Battaglia et al. , 2013 ) . This has led to a surge in research aimed at developing agents with an IPE , or a model of the environment dynamics ( Amos et al. , 2018 ; Chang et al. , 2016 ; Grzeszczuk & Animator , 1998 ; Fragkiadaki et al. , 2015 ; Battaglia et al. , 2016 ; Watters et al. , 2017 ; Sanchez-Gonzalez et al. , 2019 ; Ehrhardt et al. , 2017 ; Kipf et al. , 2018 ; Seo et al. , 2019 ; Baradel et al. , 2020 ) . These efforts have created methods that either trade-off data-efficiency , by using deep neural networks ( NNs ) , for high predictive accuracy ( Breen et al. , 2019 ; Battaglia et al. , 2016 ; Sanchez-Gonzalez et al. , 2019 ) or trade-off flexibility to learn from data for data-efficiency by using symbolic methods ( Ullman et al. , 2018 ; Smith et al. , 2019 ; Sanborn et al. , 2013 ; Bramley et al. , 2018 ) . Inspired by the highly data-efficient ability of humans to learn and reason about their physical environment with incomplete information , we present Bayesian-symbolic physics ( BSP ) , a Bayesian-symbolic model with an expectation-maximization ( EM ) algorithm that combines the sample efficiency of symbolic methods with the accuracy and generalization of data-driven approaches , using statistical inference of unobserved object properties and symbolic learning of physical force laws . In BSP , we model the evolution of the environment ’ s dynamics over time as a generative program of entities interacting under Newtonian mechanics . As a probabilistic method , BSP treats the properties of entities , such as mass and charge , as random variables . Since Newtonian force laws are functions of these properties , in BSP we replace data-hungry NNs with symbolic regression ( SR ) to learn explicit force expressions , and then evolve them deterministically using equations of motion . A naive SR implementation here is not enough though due to two issues . One is that if it operates on a vanilla grammar that does not constrain the search space over force-laws , it can potentially have worse data-efficiency than a NN . Therefore , we introduce a grammar of Newtonian physics that leverages dimensional analysis to induce a physical unit system over the search space and impose physics-based constraints on the production rules . This prunes physically meaningless laws , therefore , drastically speeding up SR. Another issue is that the symbolic force expressions usually contain global constants , e.g . the gravitational constant , to learn , and common ways to deal with this challenge turn out to be inefficient especially in an EM setup . We tackle this challenge by using SR in a bilevel optimization framework in which a lower-level gradient based optimization step is used to optimize the constant . In short , our three main contributions are : • We introduce a Bayesian-symbolic model for physical dynamics and an EM based algorithm , which combines approximate inference methods and SR , for maximum likelihood learning . • We introduce a grammar of Newtonian physics that appropriately constrains SR for data-efficient learning , based on priors from dimensional analysis and physics-based constraints . • Empirically , we show that BSP reaches human-like data-efficiency , often requiring just 1 to 5 synthetic scenes to learn the underlying force laws – much more data efficient than the closest neural alternatives . We then illustrate how BSP can discover physical laws from real-world common sense scenes from Wu et al . ( 2016 ) . Finally , we study BSP on tasks previously used to study human physical reasoning in Ullman et al . ( 2018 ) and discuss the similarity and differences with human results . 2 Related work . Many symbolic and data driven models of learning and reasoning about physics can be broken down into smaller components that are either learned or fixed . In figure 1 , we compare some of the closely-related recent work on physics learning . Starting on the right end , we have fully learned , deep NN approaches such as that used by Breen et al . ( 2019 ) . This approach does not use any prior knowledge about physics , and learns to predict dynamics in a purely data-driven way . In the middle are hybrid models that introduce some prior knowledge about physical interactions or dynamics , in their NN-based prediction models . These include interaction networks ( INs ; Battaglia et al. , 2016 ) , ODE graph networks ( OGNs ) , and Hamiltonian ODE graph networks ( HOGNs ; Sanchez-Gonzalez et al. , 2019 ) . Since these middle approaches use deep NNs , they tend to have very good predictive accuracy , yet poor sample complexity , requiring orders of magnitude more data to train than humans ( Ullman et al. , 2018 ; Battaglia et al. , 2016 ; Sanchez-Gonzalez et al. , 2019 ) . On the other end of the spectrum ( left ) are fully symbolic , rule-based physics models and engines ( Smith et al. , 2019 ; Allen et al. , 2019 ; Wu et al. , 2015 ; Ullman et al. , 2018 ) . While these methods are suitable for reasoning tasks , they lack the flexibility of data-driven , learned models as they can not generalize or adapt to changes in the environment that their fixed physics engine can not simulate . For example , inference can fail on physically implausible scenes , and may require additional workarounds such as ‘ low probability events ’ outside of the dynamics ( Smith et al. , 2019 ) . Symbolic regression has been used for general physics learning in prior research , ranging from Schmidt & Lipson ( 2009 ) ’ s work on discovering force laws from experimental data , to the more recent work of Cranmer et al . ( 2020 ) on distilling symbolic forces from INs using genetic algorithms . More recently , Udrescu & Tegmark ( 2020 ) proposed AI Feynman , which recursively simplifies the SR problem using dimensional analysis and symmetries inferred by neural networks , to discover the underlying physics equations of the data . The focus of these types of work has been to discover underlying laws and equations based on direct input-output data . The focus of BSP , on the other hand , is on physics learning based on indirect signals from the environment ; this is a task of interest in both intuitive physics studies with humans , and for human-like AI . Further , most symbolic approaches learn physics while assuming all properties in a system are known , which renders them inapplicable to environments with incomplete information . Some neural approaches focus on addressing such limitations in an end-to-end fashion ( Zheng et al. , 2018 ; Veerapaneni et al. , 2020 ; Janner et al. , 2019 ) . 3 Bayesian-symbolic physics . BSP represents the physical environment using a generative model that evolves under Newtonian dynamics ( section 3.1 ) . In this model , physical laws are treated as learnable symbolic expressions and learned using symbolic regression and a specialized grammar of Newtonian physics that confines the search space and prevent the model from learning physically meaningless laws ( section 3.2 ) . BSP does not require all properties of the entities to be fully observed . It models these properties as latent random variables and infers them using Bayesian learning . To fit BSP on data , with incomplete information , we propose an EM algorithm that iterates between Bayesian inference of the latent properties , and SR which gives maximum likelihood estimation of the force expressions ( section 3.3 ) . 3.1 Generative model of the environment . In BSP ’ s generative model , we represent each entity i ∈ { 1 . . . N } by a vector of intrinsic physical properties zi ( such as mass , charge , and shape ) , and a time dependent state vector sit = ( p i t , v i t ) which describes the evolution of its position pit ∈ Rd and velocity vit ∈ Rd under Newtonian dynamics . Here , d refer to the dimensionality of the environment , and is typically 2 or 3 . Let { τ i } Ni=1 be the set of observed trajectories from an environment with N entities , where τ i = pi1 : T : = ( p i 1 , . . . , p i T ) . Then , together with the prior on z , for an observed trajectory data , D , the generative model of BSP defines a joint probability distribution p ( D , z ; F ) over D and latent properties z , given the force function F .2 The state transition of an entity in a Newtonian system depends on its properties and current state as well as its interaction with other entities . So , in BSP the force on entity i at time t is defined as f it = ∑N j=1 F ( z i , sit , z j , sjt ) , where F ( z i , sit , z j , sjt ) is the interaction force between entities i and j . Then , the trajectory τi of entity i is generated by a transition function T that consumes the current state and the resultant force to compute sit+1 = T ( sit , f i t ) . Similar to Sanchez-Gonzalez et al . ( 2019 ) , we use numerical integration to simulate the Newtonian dynamics inside T. Specifically , we choose the Euler integrator and expand T as ait = f i t/m i , vit+1 = v i t + at∆t , p i t+1 = p i t + v i t+1∆t , ( 1 ) where mi is the mass of the recipient of the force f it and ∆t is the step size of the Euler integrator . Finally , we add Gaussian noise to each trajectory { τ i } Ni=1 , that is , D : = { τ̃ i } Ni=1 where τ̃ i : = ( p̃i1 , . . . , p̃ i T ) , p̃ i t ∼ N ( pit , σ2 ) and σ is the noise level . See appendix A.1 for the details of the complete generative process and illustrative examples .

Summary: This paper is proposing a new Bayesian symbolic framework that can assist in being able to perform types of physical reasoning over items and environments. The motivation in the paper is that humans are able to understand the physical properties of a system given very few data samples therefore we should be able to develop machine learning systems that can determine the underlying physical properties of the environment with similar data efficiency. The idea in the paper is to use a Bayesian inference method in order to be able to determine a list of parameters and how to combined different functions to describe the physics of a particular environment.  Pros - The method does propose an interesting grammar that can be used to more efficiently explore the search space of possible physical functions to describe an environment - The method also shows some promising performance across a number of different environments for understanding the underlying physical properties and only a few data samples. - Learning the physical functions as a combination of terms is also a promising direction for this research. It could be interesting to explore this further and apply the method to systems where the physics may not be well known to discover the physical properties of new physical systems.   Cons - The assumptions for the model are still not perfectly clear and well contextualized with respect to prior work. This is referring to where and how the symbolic representation for the items in the environment comes from. It's not clear if prior methods also have access to this or similar information. - It would also be good to better outline the limitation on the types of physical systems this model can learn to predict over. The equations that are supported seem to indicate the most Newtonian/classical physical systems should be possible but it is not clear what are the limitations of the grammar, along with the types of functions that can be represented. - Well it is mentioned in section 6, it's not clear how the perceptual video data used in the real experiments or The ULLMAN experiment is processed into symbolic values that can be used by BSP. Is this mechanism easy to automate such that it's not a significant limitation of applying the method in the paper?