Learning Lagrangian Fluid Dynamics with Graph Neural Networks

1 INTRODUCTION . For many science and engineering problems , fluids are an essential integral part . How to simulate fluid dynamics accurately has long been studied by researchers and a large class of numerical models have been developed . However , computing high-quality fluid simulation is still computationally expensive despite the advances in computing power . Also , the time of calculation usually increases drastically when the resolution of the simulating scene scales up . A common way to alleviate computing costs is using a data-driven model . Recent progress in the machine learning domain opens up the possibility of employing learning algorithms to learn and model fluid dynamics . In this paper , we propose a graph-based data-driven fluid dynamics model ( Fluid Graph Networks , FGN ) , which consists of simple multi-layer perceptron and graph inductive architectures . Our model predicts and integrates forward the movement of incompressible fluids based on observations . Compared to previous works in this domain ( Ummenhofer et al. , 2020 ; Sanchez-Gonzalez et al. , 2020 ) , our model enjoys traceability of physical properties of the system , like low velocity-divergence and constant particle density , and it can predict reasonable pressure distribution . Experiments demonstrate that our model can remain stable and accurate in long-term simulation . Although our model is entailed and customized for fluid simulation , it can be extended to simulation of other dynamics under the Lagrangian framework , as it takes universal features ( positions , velocities , particle density ) under the Lagrangian framework as input . 2 RELATED WORKS . Our model is built upon the Lagrangian representation of fluid , where continuous fluids are discretized and approximated by a set of particles . The most prominent advantage of the Lagrangian method is that the particle boundary is the material interface , which makes boundary conditions easy to impose , especially when the material interface is large and changing violently . A well-known Lagrangian method is Smooth Particle Hydrodynamics ( SPH ) ( Monaghan , 1988 ) . SPH and its variants are widely used in the numerical physic simulation , especially fluid dynamics under various environments . Particle-based fluid simulation ( Müller et al. , 2003 ) introduces SPH model to simulate fluids and generate realistic visual effects . Moving particle semi-implicit method ( MPS ) ( Koshizuka and Oka , 1996 ) markedly improves the accuracy and stability of incompressible fluid simulation by introducing a pressure projection procedure that emulates Eulerian grid-based methods . Weakly compressible SPH ( WSPH ) ( Becker and Teschner , 2007 ) introduces equation of state to model the pressure during the simulation . Predictive-corrective incompressible SPH ( Solenthaler and Pajarola , 2009 ) and divergence-free SPH ( Bender and Koschier , 2015 ) use iterative method to improve the accuracy of pressure calculation in incompressible flow simulation . Modeling fluid dynamics in a data-driven way has been explored and studied by many researchers . With advances in machine learning algorithms , many data-driven models employing machine learning algorithms have been built . Ladický et al . ( 2015 ) reformulate the Navier-Stokes equation as a regression problem and build a regressor using random forest , which significantly improves the calculation efficiency . Tompson et al . ( 2016 ) , Xiao et al . ( 2020 ) learn the pressure projection under the Eulerian framework with a convolutional neural network , which accelerates the fluid simulation . Wiewel et al . ( 2018 ) bring significant speed-up by learning a reduced-order representation and predicting the pressure field with an LSTM-based model . Morton et al . ( 2018 ) learn the dynamics of airflow around a cylinder based on Koopman theory . de Avila Belbute-Peres et al . ( 2020 ) predict fluid flow by combining grid-based method with graph convolutional neural networks . Learning and reasoning particle dynamics under graph representation has the following benefits and conveniences . First , particle-based methods model physics phenomena as interactions between particles within a local area . This imposes an inductive bias for learning under the Lagrangian framework : dynamics have a strong locality . The locality of unstructured data under Lagrangian representation can be captured by aggregation operation on graphs , such as GCN and other variants ( Kipf and Welling , 2016 ; Hamilton et al. , 2017 ) . Second , unlike Eulerian grid-based methods , Lagrangian particle-based methods do not have explicit and structured grid , which makes standard Convolutional Neural Network ( CNN ) can not be directly applied to particles without feature processing ( Wang et al. , 2018 ; Ummenhofer et al. , 2020 ) . Third , many dynamics are based on pairwise relation between particles , like collision , which can be easily interpreted as edge attributes of a graph . Given these factors , recently there have been a rich class of works that use graph neural networks ( Scarselli et al. , 2009 ) to learn and reason about underlying physics of interacting objects and particles . ( Battaglia et al. , 2016 ; Chang et al. , 2016 ; Sanchez-Gonzalez et al. , 2018 ; Li et al. , 2018 ; Mrowca et al. , 2018 ) 3 MODEL . 3.1 FLUID DYNAMICS . The governing equation for incompressible fluids is the Navier-Stokes equation and the continuity equation as follows ( Batchelor , 2000 ) : Du Dt = −∇p ρ + ν∇2u + g , ( 1 ) ∇ · u = 0 . ( 2 ) To describe the fluid field , there are two kinds of systems , Eulerian and Lagrangian ones . In this work , we adopt a Lagrangian system . A common method to solve the Navier-Stokes equation and discretize fluids under the Lagrangian framework is Smooth Particle Hydrodynamics ( SPH ) method ( Monaghan , 1988 ) , where physical quantities at an arbitrary point in the space are approximated by the states of nearby particles . In SPH , an arbitrary scalar ( or vector ) field A ( r ) at location r can be represented by a convolution : A ( r ) = ∫ A ( r′ ) W ( |r− r′| , h ) dV ( r′ ) , ( 3 ) where W is weighting function or smooth kernel as defined in SPH , h is the smoothing length , which defines the range of particles to be considered and V ( r′ ) is the volume at r. Numerically , the interpolation can be approximated by replacing the integration with a summation . Based on this model , equation equation 1 and equation 2 can be discretized . The discrete equation system is usually solved under a predictor-corrector scheme , prediction based on advection and correction based on physical properties ( such as divergence-free constraint ) . 3.2 MODEL . Fluids are time-dependent dynamical systems , where location of particles , r , is described by equation of form : dr/dt = f ( r ) . When building a data-driven model to learn and solve this system , we assume the system is Markovian , that is , the state of the system at time step n+ 1 depends only on the state of the previous time step n. The update mechanism in our model can be represented as : { xn+1 , vn+1 } = Gθ ( { xn , vn } ) . ( 4 ) Here { xn , vn } denotes the positional information and velocity of fluid field at time step n. Datadriven model Gθ , parameterized by θ , maps the state of time step n to time step n+ 1 . In order to build a robust and accurate data-driven model , the structure of our model is physicinformed , which enables the model to give interpretable output without losing many physical properties of the system . In general , our model mimics the predictor-corrector scheme and includes three parts , advection net , collision net , and pressure net . They can be divided into two types of graph networks ( GN ) according to the network structure ( Battaglia et al. , 2018 ) . Specifically , advection net and pressure net are node-focused graph networks , while collision net is edge-focused networks . As each of these networks has a specific task and different output , they are trained on different data separately . Node-focused Graph Network Advection net is responsible for the prediction of advection effect and pressure net is responsible for pressure projection . Considering a particle i , the node-focused graph network first aggregates node features from neighbor particles { vj |∀j ∈ N ( i ) } and output node embedding fi . The embedding fi will then be passed to a processor gR . gR will predict the desirable physical quantities oi ( i.e . acceleration a in advection net and pressure p in pressure net ) . The whole message passing procedure can be defined as : oi = gR ( gA ( Vi ) ) , Vi = { vj } ∀j∈N ( i ) . ( 5 ) Edge-focused Graph Network To prevent particle penetration and increase model stability , we propose a graph network model that is responsible for predicting the effect of collision . As the relative position and relative velocity will have different signs with different observation perspective ( i.e . relative velocity vij = −vji ) , thus the graph in collision net is directed . In collision net , relative features , eij ( relative positions , relative velocities between particle i and j ) , are passed to processor fR as edge features . The processor will output the edge embedding rij between each pair of nodes . Lastly , edge embedding rij is aggregated via aggregator fA , to gather the influence from all nearby particles and predict an overall effect oi on the center particle i . The whole process is defined as : oi = fA ( fR ( Ei ) ) , Ei = { eij } ∀j∈N ( i ) . ( 6 ) The advantage of using relative position and velocity instead of global ones as input features is that this explicitly imposes a spatial invariance to the network , given that collision between two particles is invariant to the global positions they are at . 4 IMPLEMENTATION . We adopt the numerical model in SPH to evaluate the physical quantities like particle density and differential operators like gradient . To construct graph representation for particles , we establish edges between particles within the control radius . 4.1 UPDATE SCHEME . In general , given the state ( position xn and velocity vn ) of the current time step n , we derive the state of next time step n + 1 by passing the state information through advection net , collision net , and pressure net sequentially . The input features to advection net are positions and velocities of particles , [ xn , vn ] , along with g , which indicates the external body force per mass of fluid , and viscosity parameter ν , which denotes the magnitude of fluid viscosity . The advection net predicts acceleration of particles : aadv = Gadv ( x n , vn , g , ν ) , ( 7 ) and updates the state of fluid particles to an intermediate state [ x∗ , v∗ ] . v∗ = vn + aadv∆t , ( 8 ) x∗ = xn + v∗∆t . ( 9 ) Where aadv = [ aadv1 , ... , a adv N ] , x n = [ xn1 , ... , v n N ] , v n = [ vn1 , ... , v n N ] for particles i ∈ { 1 , .. , N } . We will use the same notation throughout illustration . The collision net takes relative positions and velocities between particles , [ x∗i −x∗j , v∗i −v∗j ] as input , and predicts correction to the velocity , ∆v = Gcol ( x ∗ r , v ∗ r ) , ( 10 ) where [ x∗r , v ∗ r ] denotes the relative position and velocity in intermediate state . The velocity is then updated with predicted correction : v∗∗ = v∗ + ∆v . ( 11 ) The updated intermediate position and velocity are taken as input by the pressure net , along with particle number density ρ. p̂ = Gpres ( x ∗ , v∗∗ , ρ ) . ( 12 ) The state of fluid field is then updated to next time step n+ 1 , vn+1 = v∗∗ − ∇p̂ ρc ∆t , ( 13 ) xn+1 = x∗ + vn+1∆t . ( 14 ) Where p̂ = [ p̂1 , ... , p̂N ] , ρ = [ ρ1 , ... , ρN ] , ρc is the density parameter of fluid . Predicting pressure of fluid field using particle density and velocity is based on the observation that advection will incur a temporary compression on fluid body , which means fluid density has changed . Therefore the goal of pressure net is to impose a pressure projection to mitigate these deviations . During the above calculation , the global positional information is only used to construct graph on fluid particles and will not be passed into aggregator and processor as features . The relative position and particle density are normalized before input .

Summary: The paper deals with the prediction of 3D Lagrangian Fluid Simulations. Therefore the problem is divided into 3 subproblems, oriented on numerical simulations. An advection part, where the acceleration of the particles is calculated, a collision step, where the boundary effects are included, and a pressure prediction part, where the pressure for maintaining the volume is determined. A graph-based network is used for each part, which is either node or edge-based according to the requirements.