L EARNING L AGRANGIAN F LUID D YNAMICS WITH G RAPH N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
We present a data-driven model for ﬂuid simulation under Lagrangian representation.
Our model uses graph to describe ﬂuid ﬁeld, where physical quantities are encoded as node and edge features.
Instead of directly predicting the acceleration or position correction given current state, we decompose the simulation scheme into separate parts - advection, collision and pressure projection.
For these different reasoning tasks, we propose two kinds of graph neural network structures, node-focused networks and edge-focused networks.
Our tests show that the learned model can produce accurate results and remain stable in scenarios with large amount of particles and different geometries.
Unlike many previous works, further tests demonstrate that our model is able to retain many important physical properties of incompressible ﬂuids, such as minor divergence and reasonable pressure distribution.
Additionally, our model can adopt a range of time step sizes different from ones using in the training set, which indicates its robust generalization capability.
I NTRODUCTION
For many science and engineering problems, ﬂuids is an essential integral part.
How to simulate ﬂuid dynamics accurately has long been studied by researchers and a large class of numerical models have been developed.
However, computing high-quality ﬂuid simulation is still computationally expensive despite the advances in computing power.
Also, the time a high-quality numerical model uses usually increase drastically when resolution of scene scales up.
A common way to alleviate computing cost in many situations is using data-driven model.
Recent progress in machine learning domain opens up the possibility for employing learning algorithms to understand and model ﬂuid dynamics.
In this paper, we propose a graph based data-driven ﬂuid dynamics model (Fluid Graph Networks,
FGN), which consists of simple multi-layer perceptron and graph inductive architectures.
Our model predicts and integrates forward the movement of incompressible ﬂuids based on observations.
Compared to previous works in this domain (Ummenhofer et al., 2020; Sanchez-Gonzalez et al., 2020), our model enjoys traceability of physical properties of the system, like low velocity divergence and constant particle density, and it can predict reasonable pressure distribution.
Experiments demonstrate that our model can remain stable and accurate in long-term simulation.
Although our model is entailed and customized for ﬂuid simulation, it can be extended to simulation of other dynamics under Lagrangian framework as it takes universal feature (positions, velocities, particle density) under Lagrangian framework as input.
R ELATED W ORKS
Our model is build upon Lagrangian representation of ﬂuid, where continuous ﬂuid are discretized and approximated by a set of particles.
The most prominent advantage of Lagrangian methods is that, particle boundary is the material interface, which makes boundary condition easy to impose, especially when material interface is large and changing violently.
A well-known Lagrangian method is Smooth Particle Hydrodynamics (SPH) (Monaghan, 1988).
SPH and its variants are widely used in numerical physic simulation, especially ﬂuid dynamics under various environments.
Particle-based ﬂuid simulation (Müller et al., 2003) introduces SPH model to generate realistic ﬂuid visual effects.
Moving particle semi-implicit method (MPS) (Koshizuka and Oka, 1996) markedly improves accuracy and stability of incompressible ﬂuid simulation by introducing a pressure projection procedure that emulates Eulerian grid methods.
Weakly compressible SPH (Becker and Teschner, 2007) introduces equation of state to model the pressure during simulation.
Predictive-corrective incompressible SPH (Solenthaler and Pajarola, 2009) and divergence-free SPH (Bender and Koschier, 2015) uses iterative method to improve the accuracy of pressure calculation in incompressible ﬂow simulation.
Modelling ﬂuid dynamics in a data-driven way have been explored and studied by many researchers.
With advances in machine learning algorithms, many high-ﬁdelity data-driven models employing machine learning algorithms have been built.
Ladický et al.
(2015) reformulate the Navier-Stokes equation as an regression problem and build a regressor using random forest, which signiﬁcantly improves the calculation efﬁciency.
Tompson et al.
(2016), Xiao et al.
(2020) learn the pressure projection under Eulerian framework with convolutional neural network, which accelerates the ﬂuid simulation.
Wiewel et al.
(2018) bring signiﬁcant speed-up by learning a reduced order representation and predicting pressure ﬁeld with a LSTM-based model.
Morton et al.
(2018) learn the dynamics of airﬂow around a cylinder based on Koopman theory.
de Avila Belbute-Peres et al.
(2020) predict ﬂuid ﬂow by combining grid-based method with graph convolutional neural networks.
Learning and reasoning particle dynamics under a graph representation has the following beneﬁts and conveniences.
First, particle-based methods model physic phenomenon as interactions between particles within an area.
This imposes an inductive bias for learning under Lagrangian framework: dynamics have strong locality.
The locality of unstructured data under Lagrangian representation can be captured by aggregation operation on graph such as GCN and other aggregator (Kipf and Welling, 2016; Hamilton et al., 2017).
Second, unlike Eulerian grid-based methods, Lagrangian particle-based methods do not have explicit and structured grid, which makes standard Convolutional Neural Network (CNN) cannot be directly applied to particles without feature processing (Wang et al., 2018; Ummenhofer et al., 2020).
Third, many dynamics are based on pairwise relation between particles, like collision, which can be easily interpreted as edge attributes of graph.
Given these factors, recently there have been a rich class of works that uses graph neural networks (Scarselli et al., 2009) to learn and reason about underlying physics of interacting objects and particles.
(Battaglia et al., 2016;
Chang et al., 2016; Sanchez-Gonzalez et al., 2018; Li et al., 2018; Mrowca et al., 2018)
M ODEL 3. F LUID D YNAMICS
The governing equation for incompressible ﬂuids are the Navier-Stokes equation and continuity equation as follow (Batchelor, 2000):
[Equation]
To describe the ﬂuid ﬁeld, there are two kinds of systems, Eulerian and Lagrangian ones. In this work, we adopt a Lagrangian system.
A common method to solve Navier-Stokes equation and discretize ﬂuids under Lagrangian framework is Smooth Particle Hydrodynamics (SPH) method (Monaghan, 1988), where physical quantities at an arbitrary point in the space is approximated by the states of nearby particles.
In SPH, an arbitrary scalar (or vector) ﬁeld A ( r ) at location r can be represented by a convolution:
[Equation]
[Equation] where W is weighting function or smooth kernel as deﬁned in SPH, h is the smoothing length, which deﬁnes the range of particles to be considered and V ( r (cid:48) ) is the volume at r .
Numerically, the interpolation can be approximated by replacing the integration with summation.
Based on this model, equation equation 1 and equation 2 can be discretized.
The discrete equation system is usually solved under a predictor-corrector scheme, prediction based on advection and correction based on physical properties (such as divergence-free constraint).
is a pairwise effect, which can be as attributes between particles.
Hence focused collision
3. M ODEL
Fluids is a time-dependent dynamical system, where location of particles, r , is described by equation of form: d r /dt =
f ( r ) .
When building a data-driven model to learn and solve this system, we assume the system is Markovian, that is, the state of the system at time step n + 1 depends only on the state of previous time step n .
The update mechanism in our model can be represented as:
[Equation]
Here { x n , v n } denotes the positional information and velocity of ﬂuid ﬁeld at time step n .
Datadriven model G θ , parameterized by θ , maps the state of time step n to time step n + 1 .
In order to build a robust and accurate data-driven model, the structure of our model is physicinformed, which enables the model to give interpretable output without losing many physical properties of the system.
In general, our model mimics the predictor-corrector scheme and includes three parts, advection net, collision net and pressure net.
They can be divided into two types of graph networks (GN) according to the network structure (Battaglia et al., 2018).
Speciﬁcally, advection net and pressure net are node-focused graph networks, while collision net is edge-focused networks.
As each of these networks has speciﬁc task and different output, they are trained on different data separately.
Node-focused Graph Network Advection net is responsible for the prediction of advection effect and pressure net is responsible for pressure projection.
Considering a particle i , the node-focused graph network ﬁrst aggregates nodal features from neighbor particles { v j |∀ j ∈ N ( i ) } and output an graphical embedding f i .
Then the embedding f i will be passed to a processor g R , which takes embedding as input and output the desirable physical quantities o
i (i.e. acceleration a in advection net and pressure p in pressure net).
This message passing procedure can be deﬁned as:
[Equation]
Edge-focused Graph Network To handle the interaction between ﬂuid particles and solid wall or obstacles, we propose a graph network which is responsible for predicting this effect of collision.
As relative position and relative velocity will have different signs with different observation perspective (i.e. relative velocity v ij = − v ji ), thus the graph in collision net is directed.
In collision net, relative features, e ij (relative positions, relative velocities between particle i and j ), are passed to processor f R as edge features.
The processor then predicts the relationship r ij between nodes based on input edge features.
Finally we aggregates the output relationship via aggregator f A , to gather the inﬂuence from all nearby particles and predict an overall effect o i on center particle i .
The whole process is deﬁned as:
[Equation]
The advantage of using relative position and velocity instead of global ones as input feature is that it explicitly imposes an spatial invariance to the network. As collision between two particles is invariant to the global positions they are at.
I MPLEMENTATION
We adopt the numerical model in SPH to evaluate the physical quantities like particle density and differential operators like gradient.
To construct a graph representation for particles, we establish edges between particles within control radius.
4. U PDATE S CHEME
In general, given the state (position x n and velocity v n ) of current time step n , we derive the state of next time step n + 1 by passing the state information through advection net, collision net and pressure net sequentially.
The input features to advection net are positions and velocities of particles, [ x n , v n ] , along with g , which indicates the external body force per mass of ﬂuid, and viscosity parameter ν , which denotes the magnitude of ﬂuid viscosity.
The advection net predicts acceleration of particles:
[Equation]
and updates the state of ﬂuid particles to an intermediate state [ x ∗ , v ∗ ] .
[Equation]
Where a adv =
[ a adv 1 , ..., a adv N ] , x n = [ x n 1 , ..., v nN ] , v n =
[ v n 1 , ..., v nN ] for particles i ∈ { 1 , .., N } .
We will use the same notation throughout illustration.
The collision net takes relative positions and velocities between particles, [ x ∗ i − x ∗ j , v ∗ i − v ∗ j ] as input, and predicts correction to the velocity,
[Equation]
where [ x ∗ r , v ∗ r ] denotes the relative position and velocity in intermediate state.
The velocity is then updated with predicted correction:
[Equation]
The updated intermediate position and velocity are taken as input by the pressure net, along with particle number density ρ .
[Equation]
[Equation]
The state of ﬂuid ﬁeld is then updated to next time step n + 1 ,
[Equation]
Where ˆ p =
[ˆ p 1 , ..., ˆ p N ] , ρ = [ ρ 1 , ..., ρ N ] , ρ c is the density parameter of ﬂuid.
Predicting pressure of ﬂuid ﬁeld using particle density and velocity is based on the observation that advection will incur a temporary compression on ﬂuid body, which means ﬂuid density has changed.
Therefore the goal of pressure net is to impose a pressure projection to mitigate these deviations.
During the above calculation, the global positional information is only used to construct graph on ﬂuid particles and will not be passed into aggregator and processor as features.
Relative position and particle density are normalized before input.
4. N ETWORK A RCHITECTURES
For node-focused graph network, to derive a smooth response of the ﬁeld with respect to spatial location, the aggregation from layer l − 1 to l is deﬁned as:
[Equation]
) where the aggregator sums up the features f ( l − 1) j from neighbor vertices { v j | j ∈ N ( i ) } using smooth kernel as weight function, and here self-connection is added to every vertex.
Linear transformation
W and non-linear transformation σ are then applied to the aggregated features.
In practice we found that two layer of aggregations is enough for the model to produce reasonably accurate output (Adding more aggregation layers does not bring in signiﬁcant improvements).
As for the edge-focused network, the aggregation is simply deﬁned as:
[Equation]
where r ji is edge feature processed by processor, W is a linear transformation matrix.
The aggregation in the edge-focused network is at the last layer, so no non-linearity is included here.
The processors in both networks are implemented as shared MLP, where they are shared among nodes or edges depending on network types (e.g. in node-focused network, processor is MLP shared among each node).
In node-focused network, the processor has three hidden layers, with the input node embedding f of size 128.
In edge-focused network, the processor has four hidden layers, with input edge attributes [ x r , v r ] of size 6.
E XPERIMENTS 5. T RAINING
Dataset We use moving particle semi-implicit method (MPS) (Koshizuka and Oka, 1996) with an improved pressure solver (Lee et al., 2011) to generate high-ﬁdelity simulation data of incompressible ﬂow.
MPS is a numerical method based on SPH which prioritizes accuracy over calculation speed.
It enforces the incompressibility of ﬂuid ﬁeld by solving the pressure Poisson equation.
We created 20 scenes by randomly placing ﬂuid blocks, solid obstacles, initializing ﬂuid particles with random velocity (See A.2 for full detail of dataset generation and training strategy).
Loss Function and Optimization We train three networks, advection net, collision net and pressure net separately.
Each network is trained in a supervised way by optimizing the mean squared error between prediction ˆ y and ground truth y .
[Equation] [Equation]
We normalize particle density before inputting into pressure net, which accelerates and stabilizes training.
In the processor, we add LayerNorm (Ba et al., 2016) after activation to each layer (except for the output layer).
The parameters of the model are optimized with Adam (Kingma and Ba, 2014) optimizer.
We implement the model in PyTorch.
All the training and experiments are mainly carried out on NVIDIA GTX 1080Ti GPU.
5. E VALUATION
Baselines Besides the comparison against ground truth data, we also compare our model to two recent works which use data-driven approaches to simulate ﬂuids under Lagrangian framework.
Ummenhofer et al.
(2020) use continuous convolutional kernel to learn ﬂuid dynamics and they reported that their model has outperformed other works in this domain.
Sanchez-Gonzalez et al.
[Table] [Table]
(2020) propose a graph network-based simulator (GNS) as a general-purpose physic simulator under
Lagrangian representation.
Our model and GNS both transform and pass message of ﬂuid ﬁeld via graph structures [Table] but GNS consists of a far larger and deeper network with multiple sub-blocks and thus contains much more parameters than ours.
For all baseline models, we adopt the same training strategy from original papers but train them on our dataset (See A.4 for full detail)
[Table]
Metrics To conduct quantitative analysis, we evaluate models based on several metrics.
We report the asymmetric version of Chamfer Distance between the simulated results of different models and ground truth sequence.
The asymmetric Chamfer distance for two collections of particles X, Y (from X to Y ) is deﬁned as:
[Equation]
(17) where N is the total particle number of point cloud collection X , and distance function d ( x , y ) is evaluated using L2-norm (cid:107) x − y (cid:107) 2 .
We also investigate two essential physical quantities for incompressible ﬂuid simulation - velocity divergence and particle density deviation of ﬂuid ﬁeld.
R ESULTS
Performance In order to measure the performance of our FGN model as simulator, we performed simulation on different cases.
In the two cases, dam collapse and water fall, we ﬁrst qualitatively measure the results of different models via visualization of ﬂuids in Figure 3. We then study the physical property of simulation results and its Chamfer distance to ground truth data.
Quantitative results over the whole simulation sequence are listed in Table 1 (See A.3 for error changing trend ﬁgures).
The results show that our model FGN gives the best accuracy in retaining physical properties and position prediction.
Generalization To test out model’s capability of generalization.
We apply our model to test cases with conditions beyond training distributions.
In the ﬁrst case, we study how our model will predict pressure distribution of circular ﬂow around a cylinder.
This scene contains an inﬂow on the left side which keeps emitting particles during simulation and an outﬂow on the right side.
Additionally, we challenge our model’s robustness by applying different time step sizes to it.
Ablation Study We test and compare the performance of different types of aggregators used in pressure net, as pressure net has the largest impact on prediction accuracy 1 2 .
The evaluation metrics are same as those used in performance comparison against baselines.
Results are listed in Table 2.
In general, all aggregators give similar performance on overall position prediction (similar
Chamfer distance error), yet our model signiﬁcantly improves the the quality of predictions in terms of maintaining a constant density.
C ONCLUSION
In this paper, we present a data-driven Lagrangian ﬂuid model for incompressible ﬂuid simulation by decomposing simulation scheme as separate reasoning tasks based on Navier-Stokes equation.
It can preserve many essential physical properties of the ﬂuid ﬁeld such as low volume compression, and predict reasonable pressure distribution.
Our model also has generalization capability, where it can remain stable when extrapolating to a wide range of different geometries and adopting different time step sizes.
In general, our work is an advance in learning on unstructured data with graph neural networks, and enriches the paradigm of combining learning-based methods with physical models as well.
A PPENDIX A A. I MPLEMENTATION D ETAILS
Graph construction We build the graph by establishing edges between particles within limit radius.
We perform the neighborhood searching on GPU by using cell sort algorithm.
All the edge attributes are stored in sparse matrices.
Although the limit radius does not have signiﬁcant impact on training loss and larger limit radius will increase the computing cost drastically, we found that small limit radius can inﬂuence the long-term stability of the model.
For advection net and pressure projection net, we select the limit radius to be three times the particle diameter D (D = 0.050m), and 0.9D for collision net.
Numerical model
In general, we calculate all the gradient operators and smooth kernel based on numerical model of SPH.
In SPH model, particle density is deﬁned as:
[Equation]
For scalar quantity φ i at location r
i , we approximate its gradient by:
[Equation]
19) where n ∗ is the constant particle number density derived by calculating the maximum particle number density at the initial frame, d is the dimension of the problem, W ij denotes the smooth kernel function value between particle i and j .
Similarly velocity divergence is deﬁned as:
[Equation]
We adopt the same smooth kernel function from Koshizuka and Oka (1996), which is very simple to evaluate.
[Equation]
A. T RAINING
Dataset Generation We place scene objects in boxes with different shapes and initialize velocity uniformly sampled from U (0 , 0 . 1) .
We simulate each scene to 1000 time steps with step size dt = 0 .
Strategy We train each network for 100,000 iterations of gradient updates and decay learning rate from 0 .
001 to 0 .
For the training of three sub-networks (advection net, collision net, pressure net), the batch size of each network is 16, 4 and 32 respectively.
To allow the mini-batching of different graph, we mini-batch the adjacency matrix of different scenes by creating a large sparse matrix and stacking adjacency matrix on the diagonals.
[Figure]
A. M ETRICS E VALUATION D ETAILS
We show how error under different evaluation metrics change as a function of time in Figure 6.
A. B ASELINES I MPLEMENTATION
Continuous Convolution We use the open-source implementation from Ummenhofer et al.
To give a fair benchmark result we train their network with our dataset.
As in our work we model solid obstacles and wall as virtual particles, so we transform these virtual particles into surface and corresponding normals before inputting them into CConv network.
The original CConv uses a time step size of 0.02s but given such a large time step size, the qualitative results can be distinct from ground truth and other model with smaller time step size.
Hence during training and comparison we adopt a time step size of 0.002s for CConv model.
Graph Network-based Simulator We implemented GNS following the description in SanchezGonzalez et al.
We build GNS with 10 unshared GN blocks, conditioned on 5 previous velocities and input relative positions as edge features.
We chose the connectivity radius to be 2.1D, so that the number of neighbors is around 20.
Sanchez-Gonzalez et al.
(2020) use ﬁnite difference to calculate acceleration and velocity, but in our implementation we explicitly maintain an array to store the velocities of all particles.
Additionally, we do not use learned embedding but simple zero and one to indicate particle material type, as in our testing domain there are only two kinds of particles - solid and ﬂuid.
The loss function and training procedure are implemented as described in
Sanchez-Gonzalez et al.
(2020), including noise injection and similar normalization techniques.
For the implementation of GN block, as Sanchez-Gonzalez et al.
(2020) states "We use GNs without global features or global updates (similar to an interaction network)", so we implement the GN block update mechanism following the description in Battaglia et al.
[Equation]
[Equation] [Equation]
24) where φ is the update function and implemented as MLP here
, ρ is aggregation function which aggregates all the edge attributes to its center vertex, u is the global feature and here we append them to the nodal features as input feature, r k denotes receiver vertices and s k denotes sender vertices.
In the testing stage, as authors did not state how to set initial velocities, so we just warm start the simulation by calculating the ﬁrst 5 frames using MPS method and apply GNS to the rest frames.