Learning Lagrangian Fluid Dynamics with Graph Neural Networks
Authors: Zijie Li
Amir Barati Farimani
Venue:  (2021)
Keywords: particle hydrodynamics
graph neural networks
Lagrangian fluids

Abstract: We present a data-driven model for fluid simulation under Lagrangian representation. Our model uses graphs to describe the fluid field, where physical quantities are encoded as node and edge features. Instead of directly predicting the acceleration or position correction given the current state, we decompose the simulation scheme into separate parts - advection, collision, and pressure projection. For these different reasoning tasks, we propose two kinds of graph neural network structures, node-focused networks, and edge-focused networks. By introducing physics prior knowledge, our model can be efficient in terms of training and inference. Our tests show that the learned model can produce accurate results and remain stable in scenarios with a large amount of particles and different geometries. Unlike many previous works, further tests demonstrate that our model is able to retain many important physical properties of incompressible fluids, such as minor divergence and reasonable pressure distribution. Additionally, our model can adopt a range of time step sizes different from ones using in the training set, which indicates its robust generalization capability.

Under review as a conference paper at ICLR 2021
LEARNING LAGRANGIAN FLUID DYNAMICS WITH GRAPH NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We present a data-driven model for fluid simulation under Lagrangian representa- tion. Our model uses graphs to describe the fluid field, where physical quantities are encoded as node and edge features. Instead directly predicting the accelera- tion position correction given the current state, we decompose the simulation scheme into separate parts advection, collision, and pressure projection. For these different reasoning tasks, we propose two kinds of graph neural network structures, networks, and edge-focused networks. By introducing physics prior knowledge, our model can be efficient terms of training and inference. Our tests show that the learned model can produce accurate results and remain stable in scenarios with a large number of particles and different geometries. Unlike many previous works, further tests demonstrate that our model is able to retain many important physical properties of incompressible fluids, such as minor divergence and reasonable pressure distribution. Additionally, model can adopt a range time step sizes different from ones using in the training set, which indicates its robust generalization capability.
INTRODUCTION
For many science and engineering problems, fluids are an essential integral part. How to simulate fluid dynamics accurately has long been studied by researchers and large class of numerical models have been developed. However, computing high-quality fluid simulation is still computationally expensive despite the advances in computing power. Also, the time of calculation usually increases drastically when the resolution of the simulating scene scales up. A common way to alleviate computing costs is using a data-driven model. Recent progress in the machine learning domain opens up the possibility of employing learning algorithms to learn and model fluid dynamics.
In this paper, we propose graph-based data-driven fluid dynamics model (Fluid Graph Networks, FGN), which consists of simple multi-layer perceptron and graph inductive architectures. Our model predicts and integrates forward the movement of incompressible fluids based on observations. Compared to previous works in this domain (Ummenhoter et al. 2020 Sanchez-Gonzalezz et al. (020), and
our model enjoys traceability of physical properties of the system, like low constant particle density, and it can predict reasonable pressure distribution. Experiments demonstrate that our model can remain stable and accurate in long-term simulation. Although our model entailed and customized for fluid simulation, it can be extended to simulation of other dynamics under the Lagrangian framework, as it takes universal features (positions, velocities, particle density) under the Lagrangian framework as input.
2 RELATED WORKS
Our model is built upon the Lagrangian representation of fluid, where continuous fluids are discretized and approximated by a set particles. The most prominent advantage of the Lagrangian method that the particle boundary is the material interface, which makes boundary conditions easy to impose, especially when the material interface is large and changing violently. A well-known Lagrangian method is Smooth Particle Hydrodynamics (SPHyMMonaghanl [1988). SPH and its variants are widely used in the numerical physic simulation, especially fluid dynamics under various environments. Particle-based fluid simulation (Müller et al. 2003) introduces SPH model to simulate fluids and

---

Under review as a conference paper at ICLR 2021
generate realistic visual effects. Moving particle semi-implicit method (MPS) (Koshizuka and Oka 1996) markedly improves the accuracy and stability of incompressible fluid simulation by introducing pressure projection procedure that emulates Eulerian grid-based methods. Weakly compressible SPH (WSPH)(Becker and Teschner 2007) introduces equation of state to model the pressure during the simulation. Predictive-correctie incompressible SPH (Solenthaler and Pajarola| 2009) and divergence- SPH (Bender and Koschier (2155) use iterative method to improve the accuracy of pressure calculation in incompressible flow simulation.
Modeling fluid dynamics in a data-driven way has been explored and studied by many researchers. With advances in machine learning algorithms. many data-driven models employing machine learning algorithms have been built. [adiikk et al. (2015) reformulate the Navier-Stoker equation as regression problem and build a regressor using random forest, which significantly improves the calculation efficiency. Tompson et al. (206)), Xiao et al. (2020) learn the pressure projection under the Eulerian framework with a convolutional neural network, which accelerates the fluid simulation. Wiewel et al. (2018) bring significant speed-up by learning reduced-order representation and predicting the pressure field with an LSTM--ased model. Morton et al. (2018) learn the dynamics of airflow around cylinder based on Koopman theory. de Avila Belbute- Peres et al. (2020) predict fluid flow by combining grid- based method with graph convolutional neural networks.
Learning and reasoning particle dynamics under graph representation has the following benefits and conveniences. First, particle- cles within local area. This imposes an inductive bias for learning under the Lagrangian framework: dynamics have a strong locality. The locality of unstructured data under Lagrangian representation can be captured by aggregation operation on graphs, such as GCN and other variants (Kipf and Welling 2016 Hamilton et al.] (2017). Second, unlike Eulerian grid-based methods. Lagrangian particle- based methods do not have explicit and structured grid, which makes standard Convolutional Neural Network (CNN) cannot be directly applied to particles without feature processing (Wang et al. 2018 [Ummenhofer et al. 2020). Third, many dynamics are based on pairwise relation between particles, like collision, which can be easily interpreted as edge attributes of a graph. Given these factors, recently there have been a rich class of works that use graph neural networks (Scarselli et al. 2009) to learn and reason about underlying physics of interacting objects and particles. (Battaglia et al.
methods model physics phenomena as interactions between parti-
2016 Chang et al. 2016 Sanzaez-Gonzalez et al. 2018 Li et al. 2018 Mrowca et al. 2018)
MODEL
3.1
FLUID DYNAMICS
The governing equation for incompressible fluids is the Navier-Stokes equation and the continuity equation as follows (Batchelor) 2000):
Du
Vp + 2 u +g
(1)
Dt
u=
(2)
To describe the fluid field, there are two kinds of systems, Eulerian and Lagrangian ones. In this work, adopt Lagrangian system. A common method to solve the Navier-Stokes equation and discretize fluids under the Lagrangian framework is Smooth Particle Hydrodynamics (SPH) method (Monaghan 1988), where physical quantities at arbitrary point in the space are approximated by the states of nearby particles.
In SPH, an arbitrary scalar (or vector) field A (r) location can be represented by convolution:
A (r)) - f (r)',,,)) dV (r'),
where W is weighting function or smooth kernel as defined in SPH, is the smoothing length, which defines the range of partieles to be considered and V (r') is the volume r. Numerically, the interpolation can be approximated by replacing the integration with a summation.
Based on this model, equation equation and equation 2 can be discretized. The discrete equation system is usually solved under a rredictor-correcte scheme, prediction based on advection and correction based on physical properties (such divergence-free constraint).
2

---

Under review as a conference paper at ICLR 2021
(a)
V*
Andv
Av
Gadv
Goal
Gpres
n-th frame
n+11-t frame
f
Ieput
Outpont
(b)
0j
Node-focused Network
N
Dg
Aggregata neighbor partioles
Graphical Embedding
Apply processor to graphical Embedding
(c)
Output
Edge-focused Network
Extract features from neighbor particlles
Stacked features
Apply processor to edge features
Aggregate proceased edge
(a) Schematic of our FGN model. Gadv applies the effect of body force and viscosity the fluids. Gpres predicts the pressure. Geol handles collision between particles. (b) In a node-focused network, input are represented as node features and then passed to shared processor. Advection and pressure impose influence each particle separately, thus they can be interpreted messages lying on each node and predicted via node-focus focused networks. (c) In an edge-focusee network, input represented edge features of directed graph, the edge features are then passed shared processor. Collision is a pairwise effect, which can be easily represented as edge attributes between each pair of particles. Hence we use the edge-focused network to predict collision.
Figure 1:
3.2 MODEL
Fluids are time-dependent dynamical systems, where location of particles, is described by equation of form: dr/dt fr). When building data-driven model learn and solve this system, we assume the system is Markovian, that is, the state of the system at time step + depends only on the state of the previous time step n. The update mechanism our model can represented as:
(x=+1 - Ge ((x",v"})
(4)
Here (x", v") denotes the positional information and velocity of fluid field at time step n. Data- driven model Go. parameterized by 0 maps the state of time step to time step 1.
In order to build robust and accurate data-driven model, the structure of our model is physic- informed, which enables the model give interpretable output without losing many physical proper- ties of the system. In general, our model mimics the predictor-correetor scheme and includes three parts, advection net, collision net, and pressure net. They can be divided into two types of graph networks (GN) according to the network structure (Battaglia et al., 2018). Specifically, advection and pressure net are node-focused graph networks, while collision net edge-focused networks. As each of these networks has a specific task and different output, they are trained on different data separately.
Node-focused Graph Network Advection net is responsible for the prediction of advection effect and pressure net is responsible for pressure projection. Considering a particle i, the boce-focused graph network first aggregates node features from neighbor particles {uylVí E (t) and output node embedding f, The embedding f, will then be passed to a processor gn- 9n will predict the desirable physical quantities (i.e. acceleration in advection net and pressure in pressure net).
3

---

Under review as a conference paper at ICLR 2021
The whole message passing procedure can be defined as:
0i 9R (9A (Vi) V2 = (uS)yeNN
(5)
Edge-focused Graph Network To prevent particle penetration and increase model stability, we propose graph network model that is responsible for predicting the effect of collision. As the relative position and relative velocity will have different signs with different observation perspective (i.e. relative velocity Vij -jú). thus the graph in collision net is directed. collision net, relative features, Eij (relative positions, relative velocities between particle and j), are passed to processor fn as edge features. The processor will output the edge embedding rij between each pair of nodes. Lastly, edge embedding fij is aggregated via aggregator fA, to gather the influence from all nearby particles and predict an overall effect og on the center particle i. The whole process is defined as:
0g = fA (Jn (E.)), E, = {egg (ojev())
The advantage of using relative position and velocity instead of global ones as input features is that this explicitly imposes spatial invariance the network, given that collision between two particles invariant to the global positions they are at.
4 IMPLEMENTATION
We adopt the numerical model in SPH to evaluate the physical quantities like particle density and differential operators like gradient. To construct graph representation for particles, we establish edges between particles within the control radius.
4.1 UPDATE SCHEME
In general, given the state (position x" and velocity v") of the current time step we derive the state of next time step n + 1 by passing the state information through advection net, collision net, and pressure net sequentially. The input features to advection net are positions and velocities of particles, [x", v"]. along with g. which indicates the external body force per mass of fluid, and viscosity parameter v, which denotes the magnitude of fluid viscosity. The advection net predicts acceleration of particles:
a adv
Gady (x", v",,,,) :),
(7)
and updates the state of fluid particles to an intermediate state [x", v*].
v + a
(8)
x x" + At.
(9)
Where adv [agdy ..... N x [x] will use the same notation throughout illustration.
v -..,. v for particles E (1,-,N}, We
The collision net takes relative positions and velocities between particles, [x] xf,, vi výl input, and predicts correction to the velocity,
Av = (x,,, vi),
(10)
where [x*., v2] denotes the relative position and velocity in intermediate state. The velocity is then updated with predicted correction:
- + Av.
(11)
The updated intermediate position and velocity are taken as input by the pressure net, along with
particle number density p.
p = Gpees (x", p). The state of fluid field is then updated to next time step n + 1.
(12)
vnt1 V Vp At.
(13)
Pa
n+1
x" + nl At.
(14)
4

---

Under review as a conference paper at ICLR 2021
Where p = fluid field using particle density and velocity is based on the observation that advection will incur a temporary compression fluid body, which means fluid density has changed. Therefore the goal of pressure net is to impose a pressure projection to mitigate these deviations.
p P [p1..... Pe is the density parameter of fluid. Predicting pressure of
During the above calculation, the global positional information is only used to construct graph fluid particles and will not be passed into aggregator and processor as features. The relative position and particle density normalized before input.
4.2 NETWORK ARCHITECTURES
For nocusfocused graph network, to derive a smooth response of the field with respect to spatial location, the aggregation from layer / to is defined as:
1 Σ (1 W (Ir; r)l.h) a;
f.1-1) Vj EN),,
(15)
Σ W (|r 12),
-0= W a
(16)
where the aggregator sums up the features (((-)) from neighbor vertices {uyli (()) using smooth kernel weight function, and here self-connection is added every vertex. Linear transformation W and non-linear transformation a are then applied to the aggregated features. In practice we found that two layer of aggregations is enough for the model produce reasonably accurate output (Adding more aggregation layers does not bring in significant improvements).
As for the edge-focused network, the aggregation is simply defined
a: =
Σ W. Tji-
(17)
JEN((E
where ryt is edge feature processed by processor, W is linear transformation matrix. The aggrega- tion in the edge-focused network is at the last layer, so no non-linearity is included here.
The processors in both networks implemented as shared MLP, where they shared among nodes or edges depending on network types (e.g. in the node-focused network, the processor is an MLP shared among each node). In a node-focused network, the processor has three hidden layers, with the input node embedding of size 128. In an edge- edge-focused network, the processor has four hidden layers, with input edge attributes [x,, vr] of size
EXPERIMENTS
5.1 TRAINING
Dataset We use the Moving Particle Semi-implicit method (MPS) (Koshizuka and Oka 1996) with an improved pressure solver (Lee al., (20T)) to generate high-fidelity simulation data of incompressible flow. MPS is numerical method based SPH which prioritizes accuracy calculation speed. enforces the incompressibility of the fluid field by solving the pressure Poisson equation. We created 20 scenes by randomly placing fluid blocks, solid obstacles, initializing fluid particles with random velocity (See A.2 for full detail of training dataset settings and training strategy).
Loss Function and Optimization We train three networks, advection net, collision net, and pres- sure net separately. Each network is trained in a supervised way by optimizing the mean squared error between prediction ý and ground truth y.
(18)
We normalize particle density before inputting into the pressure net, which accelerates and stabilizes training. In the processor, add LayerNorm (Ba al. 2016) after activation to each layer (except
5

---

Baselines Besides the comparison against ground truth data, we also compare our model two recent works that use data-dha driven approaches to simulate fluids under the Lagrangian framework. Ummenhofer al. (2020) use the continuous convolutional kernel learn fluid dynamics and they et al. Under review as a conference paper at ICLR 2021
Average Chamfer distance (mm)
Density error
Velocity divergence
Case
Model
Mean
Max
Mean
Max
Mean
Max
FGN (this work)
0.0461
0.0900
0.0195
0.0280
24.2
30.1
Dam
GNS
0.0898
0.3387
0.0210
0.0311
26.1
33.1
Collapse
CConv
0.1811
0.2947
0.0338
0.1700
31.4
44.0
Ground Truth
0.0380
0.0710
0.0190
0.0268
FGN (this work)
0.0541
0.1350
0.0207
0.0431
24.8
29.6
Water
GNS
0.1035
0.3512
0.0223
0.0399
25.6
31.8
Fall
CConv
0.2783
0.4121
0.0668
0.2882
40.4
61.5
Ground Truth
0.0429
0.0966
0.0196
0.0398
Table I: Quantitative accuracy analysis. For resolution, the dam collapse case contains about 10k particles and water fall case contains about 80k particles. We report the mean and maximum value of density error and velocity divergence over the whole simulation sequence. We also report the average Chamfer distance between the results of each model and ground truth data.
for the output layer). The parameters of the model are optimized with Adam (Kinmma and Ba 2014) optimizer. We implement the model in PyTorch. All the training and experiments are mainly carried out on NVIDIA GTX 1080Ti GPU.
5.2 EVALUATION
reported that their model has outperformed other works in this domain. [Sanchez-Gonzzz (2020) propose a graph network-based simulator (GNS) as general-purpont physic simulator under Lagrangian representation. Our model and GNS both transform and pass messages of fluid field via graph structures, but GNS consists far larger and deeper network with multiple sub-blocks and thus contains much more parameters than ours. For all baseline models, adopt the same training strategy from original papers but train them on our dataset (See A.5 for full detail).
Metrics To conduct quantitative analysis, evaluate model performance based several metrics. We report the asymmetric version of Chamfer Distance between the simulated results of different models and ground truth sequence. The asymmetric Chamfer distance for two collections of particles X, Y (from X to Y) is defined as:
L(X,Y)= = N
min d (x,y), Σ xEX
(19)
yEy
where N is the total particle number of point cloud collection X, and distance function d(x, y) is evaluated using L2-norm ||x 1x-yl22 We investigate two essential physical quantities for incompressible fluid simulation velocity divergence and particle density deviation of fluid field. addition, we use normalized mean absolute eиror (MAE) and relative tolerance to evaluate the error of advection net and pressure net on single frame inference respectively. The normalized MAE from prediction ý to ground truth y is defined as:
1
LMAE
(20)
N
The relative tolerance of the numerical solution & to a linear system Ax b can be defined as:
|AX bll
tol
(21)
|bll
6 RESULTS
Performance In order to measure the performance our FGN model as a physic simulator, we performed simulations several different test cases. In the first cases, dam collapse and water

---

Under review as a conference paper at ICLR 2021
t-100
250
500
750
1000
Ground Trais
CConv
FGN
GIS
(b)
Ground Truih
CCony
PON
ONS
Figure 2: Qualitative analysis on: (a) Dam collapse. (b) Water fall. In CConv and GNS, we can observe oscillation the free surface fluids, while the results generated from our model maintains much more compact and smoother shape. After long time steps, CConv's result fails to maintain smooth and compact fluid distributions and GNS' prediction significantly slower than ground truth, while our model's 's prediction has minor difference from the ground truth.
FGN
Ground Truth
t=100
t-750
Figure 3: Pressure distribution contour. Our model can learn to generate reasonable pressure distribution. The distribution agreed with ground truth, which captures the shape shifting high pressure and low pressure region.
7

---

Under review as a conference paper at ICLR 2021
00002
dr dr=0.004s 0.000
Figure 4: Qualitative comparison of model using different time step sizes. We use different time step size simulate with our model. Despite the training set is generated with time step size 0.002s, our model can be generalized to larger time step size with neglectable increase in error. Other models diverge immediately when extrapolate to time step size different from training data.
Figure 5: Generalization complex geometries. Top: fluid block drop on Stanford bunny. Bottom: A fluid emitter is placed at the front of Grand Canyon 3D model and emitted fluids gradually fill up reservoir at downstream. Although the training data only has basic geometries such cube and cylinder, our FGN model can be generalized to complex geometries which are beyond the distribution of training data.
Model
Dynamical system
Model Output
Evaluation metric
Error
Advection Net
2 ú g
prediction of ú prediction pf Relative tolerance
Normalized MAE
12.4% + 5.6%
Pressure Net V = + V u²
8.3% 1.1%
Table 2: Quantitative analysis of advection net and pressure net. We evaluate advection net's performance solver for a forward problem. We generate sequences with different of material parameters as test data for advection net. The gravity g for test data ranges from 1.0 to 100.0 and viscosity parameter v ranges from 0.1 0.0001. For pressure net, we report its error as solver for linear equation system. The test data for pressure net is generated using dam collapse, water fall, and Stanford bunny sequences.
fall, we qualitatively measure the results of different models via visualizatior of fluids in Figure We report the physical property of simulation results and its Chamfer distance to ground truth data. Quantitative results over the whole simulation sequence are listed in Table [ (See Figure A.3 for error trend figures). The results show that our model FGN gives the best accuracy in retaining physical properties and position prediction. In addition, each sub-network in our model as stand-alone solvers for sub-dynamical systems and report their relative error in Table 2 For the advection net, challenge it by applying different set of material parameters (i.e. different gravity and viscosity parameters). We report the normalized mean absolute error (MAE) between prediction and ground truth. For pressure net, evaluate the relative tolerance of its predicted solution P to the discretized pressure poisson equation (i.e. Ap = b). The relative low error demonstrates the capability of our sub-networks in learning and predicting physics.
study the performance of
Video link: httpss:/sites googl www./o//vvvieeeee uid-graphhhhhhooo network-ve dww.hoom
8

---

Under review as a conference paper at ICLR 2021
Model
Parameters
Model inference time (ms)
NNS time (ms)
FGN (this work)
41996
5.7
206.4
GNS
1406272
44.6
153,2
CConv
692902
28.9*
24.7'
MPS (ground truth)
520.4
221.7
Table 3: Runtime analysis of different model. The test was carried out a dam collapse scene containing approximately 40k fluid particles. We report the total trainable parameters for each deep learning model, averaged model inference time and nearest neighbor searching time. For ground truth solver, MPS, we report the time used to calculate advection and pressure projection as model inference time. The pressure solver in MPS is preconditioned conjugate gradient (PCG) solver implemented in Pytorch. Note that in CConv, the network and neighbor searching method are based on Tensorflow and Open3D (denoted with *). Besides CConv, all other model are implemented in Pytorch and use spatial hashing on GPU for neighbor searching. Our model has the smallest size and fastest inference time.
Generalization To out the model's capability of generalization. We apply our model test cases with conditions that are beyond training distributions, In the first case, we study how our model will predict the pressure distribution of circular flow around cylinder. This scene contains an inflow on the left side which keeps emitting particles during the simulation and an outflow on the right side. We challenge our model's robustness by applying different time step sizes qualitative comparison of model output under different time step sizes. In addition, on two scenes which contain much more complex geometries. Visualizations of two test scenarios containing complex geometries are shown in Figure 5 Although model is trained with only a fairly small dataset, remains accurate under several different conditions. This demonstrates capability of generalization and robustness. (More details quantitative analysis in A.3) Figure shows position error trend under different time step sizes. Figure 11 shows position error trend under complex scenes.)
Figure Hhowss simulate
Ablation Study We test the performance of different types of aggregators used in pressure net, as pressure net has the largest impact overall prediction accuracy. We compare our aggregator against graph convolution networks (GCN) from Kipf and Welling (2016), Hamilton et al. (211))ss graph SAGE using mean aggregator, and MLP w/o any graph aggregation operation. In general, all aggregators give a similar performance on overall position prediction (similar Chamfer distance error), yet our model significantly improves the quality of predictions in terms of maintaining constant density (See Table in A.4 for full detail).
In addition, report the model size (parameter number) and runtime benchmark in Table Our model is very efficient in terms of training and inference, as it has far less trainable parameters than others.
7 CONCLUSION
In this paper, we present a data-driven Lagrangian fluid model for incompressible fluid simulation by decomposing simulation scheme separate reasoning tasks based on Navier-Stokes equation. It can preserve many essential physical properties of the fluid field such as low volume compression, and predict reasonable pressure distribution. Our model also has generalization capability, where it can remain stable when extrapolating to a wide range of different geometries and adopting different time step sizes. In general, our work is an advance in learning on unstructured data with graph neural networks, and enriches the paradigm of combining learning-bass based methods with physical models as well.
2 During the preliminary experiment, we find that aggregators in advection net do not have significant impact on overall model performance when viscosity parameter of fluids is small, therefore ablation results on it are not listed here; Aggregator in collision net is just summing up all the predicted edge attributes and conducting linear transformation. Using more advanced structure does not bring further improvement.

---

Under review as a conference paper at ICLR 2021
REFERENCES
B. Ummenhofer, L. Prantl, N. Thuerey, and V. Koltun, "Lagrangian fluid simulation with continuous convolutions," in International Conference on Learning Representations, 2020.
A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying. J. Leskovec, and P. W. Battaglia, "Learning to simulate complex physics with graph networks," 2020.
J. Monaghan, "An introductiio to sph," Computer Physics Communications, vol. 48, no. 1, pp. 89 96, 1988. [Online]. Available:
M. Müller, D. Charypar, and M. Gross, "Particle-based fluid simulation for interactive applications." in Proceedings of the 2003 ACM SJGGGAAPH/Fargrapphiccc Symposium Computer Animation, ser. SCA '03. Goslar, DEU: Eurographics Association, 2003, 154-159.
S. Koshizuka and Y. Oka, "Moving-partilee semi amiiimmlicit method for fragmentation of incompressible fluid," Nuclear Science and Engineering, vol. 123, no. pp. 421 434, 1996. [Online]. Available: http://co..ogg11100.111///5599.9
M. Becker and M. Teschner, "Weakly compressible sph for free surface flows," in Proceedings of the 2007 ACM SIGGRAPH/Eurogrphhis Symposium on Computer Animation, ser. SCA '07. Goslar, DEU: Eurographics Association, 2007, p. 209-217.
B. Solenthaler and R. Pajarola, "Predictive-corrective incompressible sph," in ACM SIGGRAPH 2009 Papers, ser. SIGGRAPH '09. New York, NY, USA: Association for Computing Machinery, 2009. [Online]. Available:
145//1166//777.1111 1531346)
J. Bender and D. Koschier, "Divergence- free smoothed particle hydrodynamics," in Proceedings of the 2015 ACM SIGGRAPH//unngraphis Symposium on Computer Animation. ACM, 2015.
L. Ladický, S. Jeong, B. Solenthaler, M. Pollefeys, and M. Gross, "Data-driven fluid simulations using regression forests," ACM Trans. Gruph., vol. 34, no. 6, Oct. 2015. [Online]. Available: http://66b.og/1100144/211133
Tompson, K. Schlachter, Sprechmann, and K. Perlin, "Accelerating culerian fluid simulation with convolutional networks," CoRR, vol. abs/1607.03597, 2016. [Online]. Available: http://arxiv.org/abb/ 1607.03597
X. Xiao, Y. Zhou, H. Wang, and X. Yang, "A novel cnn-based poisson solver for fluid simulation," IEEE Transactions on Visualization Computer Graphies, vol. 26, no. 03, pp. 1454-1465, mar 2020.
S. Wiewel, M. Becher, and N. Thuerey, "Latent-space physies: Towards learning the temporal evolution of fluid flow," CoRR, vol. abs/1 htss/882..0123.. 2018. [Online]. Available: http://arxiv.org/abs///aa 100 10123
Morton, A. Jameson, M. J. Kochenderfer, and F. Witherden, "Deep dynamical modeling and control of unsteady fluid flows," in Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds. Curran Associates, Inc., 2018, pp. 9258 22689926. [Online]. Available: hhtp: (ipapers.iips.c//papr///// http://s..ccpppppeo//.oeeeeeeaaa 38--dee--ynnamicll-ne moweeiingaaoodigin--o-oagene wwwedellinn-n----ooe wowooooooooo-ussteaaaae fluid-lludddooooo flows.pdi
F. de Avila Belbute-Peres, T. D. Economon, and J. Z. Kolter, "Combining differrentablee pde solvers and graph neural networks for fluid flow prediction," 2020.
T. N. Kipf and M. Welling, "Semi-superrvise classification with graph convolutional networks," CoRR, vol. abs/1609.02250 002900 2016. [Online]. Available: http://arxiv.or//ab////b 02907
W. L. Hamilton, R. Ying, and J. Leskovee, "Inductive representation learning on large graphs," 2017.
S. Wang. S. Suo, W.-C Ma, A. Pokrovsky, and R. Urtasun, "Deep parametric continuous convolu- tional neural networks." in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
10

---

Under review as a conference paper at ICLR 2021
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61-80, 2009.
P. W. Battaglia, R. Pascanu, M. Lai, D. J. Rezende, and K. Kavukcuoglu, "Interaction networks for learning about objects, relations and physics," CoRR, vol. abs/1612.00222,, 2016. [Online]. Available:
M. B. Chang, T. Ullman, A. Torralba, and J. B. Tenenbaum, "A compositional object-based approach learning physical dynamies," CoRR, vol. abs/1612.00341,, 2016. [Online]. Available: http://arrivoorr/ab///1...000
A. Gonzalz- N. Heess, J. T. Springenberg. J. Merel, M. A. Riedmiller, R. Hadsell, and P. W. Battaglia, "Graph networks as learnable physics engines for inference and control," CoRR, vol. abs/1 abs/1806.00 2018. [Online]. Available: http://axiv.oor/aab/1186.....
Y. Li, Wu, R. Tedrake, B. Tenenbaum, and A. Torralba, "Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids," CoRR, vol. abs/ 181.001666, 2018. [Online]. Available: http://arxivorrgaa///880.0. 180.00156
D. Mrowca, Zhuang, E. Wang, N. Haber,
Fei-Fei, Tenenbaum, and D. Yamins, "Flexible neural representation for physics prediction," in Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. BessaBBincchi, and R. Garnett, Eds. Curran Associates, Inc., 2018, pp. 8799-8810. [Online]. Available: flexiileeee neural- representation for- for-physics- prw.ictiodddpppppp physics-prediction pdf
G. K. Batchelor, An Introduction to Fluid Dynamics, ser. Cambridge Mathematical Library. Cam- bridge University Press, 2000.
Battaglia, J. Hamrick, V. Bapst, A. Sanzhez- F. Zambaldi, Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, Ç. Gülçehre, H. F. Song, A. J. Ballard, Gilmer, G. E. Dahl, A. Vaswani, K. R. Allen, C. Nash, V. Langston, C. Dyer. N. Heess, D. Wierstra, P. Kohli, M. Botvinick, O. Vinyals, Y. Li, and R. Pascanu, "Relational inductive biases, deep learning, and graph networks," CoRR, vol. abs/ 0../1806601261, 2018. [Online]. Available: http://arxiv.org/abs// 806.01261
B.-H. Lee, J.-C. Park, M.-H. Kim, and S.-C. Hwang. "Step-by-step improvement of mps method in simulating violent free-surface motions and impact-loads," Computer Methods in Applied Mechanics and Engineering, vol. 200, no. 9, pp. 1113 1125, 2011. [Online]. Available:
J. L. Ba, J. R. Kiros, and G. E. Hinton, "Layer normalization," 2016.
D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization." 2014.

---

Under review as a conference paper at ICLR 2021
APPENDIX A
A.1 IMPLEMENTATION DETAILS
Graph construction We build the graph by establishing edges between particles within limit radius. We perform the neighborhood searching on GPU by using cell sort algorithm. All the edge attributes are stored in sparse matrices. Although the limit radius does not have significant impact on training loss and larger limit radius will increase the computing cost drastically, we found that small limit radius can influence the long-term stability of the model. For advection net and pressure projection net, we select the limit radius to be three times the particle diameter D (D 0.050m), and 0.9D for collision net.
Numerical model In general, we calculate all the gradient operators and smooth kernel based on numerical model of SPH.
In SPH model, particle density is defined as:
n4 Σ W (j)
(22)
For scalar quantity Φ; at location ri, we approximate its gradient by:
d
VØ=
(6 VW,, n
(23)
where n² is the constant particle number density derived by calculating the maximum particle number density at the initial frame, is the dimension of the problem, Way denotes the smooth kernel function value between particle and j. Similarly velocity divergence is defined as:
d
V;
Σ(v, v:) VW.j. n
(24)
We adopt the same smooth kernel function from Koshizuka and Oka (199)). which is very simple to evaluate.
h if ||r r, 15112 < if ||: ry r5l1 2 h
Wi = {
1 r
(25)
0
A.2 TRAINING
Dataset Generation We place one or two of the following basic obstacles (as shown Figure 6) in training scenes. In addition to obstacles, each scene is cubic box (80x80x40) containing fluid block (25x25x10) (as shown in Figure (7). We place the fluid block at random place in the box and initialize its velocity of one direction by uniformly sampling from 2/ (0, 0.1). We generated 20 scenes adopting above settings and simulate each training scene to 1000 time steps with step size dt 0.002.
Strategy In each time step of the ground truth simulator, there are mainly three steps. First, advect fluids with body force and viscosity:
v + a adv At,
(26)
x" + A4
(27)
and then solve the pressure poisson equation,
V = At v.v.,
(28)
lastly.
v Vp At.
(29)
Pa
x n+1
x" + nt At.
(30)
12

---

Under review as a conference paper at ICLR 2021
Here, use [x*, v", g, 1// as the training features and a" (i.e. (v* - -v") / as the training label for advection net. [x", v"] and partiele density p (evaluated based on x") are the training features for pressure net, with pressure p label. To train collision net, simulate another particle system that is updated only based on elastic collision rule and applies no other dynamics. We use relative velocity and position before collision inputs, and velocity difference (i.e. Au) output target.
We train each network for 100,000 iterations of gradient updates and decay learning rate from 0.001 0.0000625. For the training of three sub-networks (advection net, collision net, pressure net), the batch size of each network is 16, 4 and 32 respectively. To allow the mini-batching of different graph, we the adjacency matrix of different scenes by creating a large sparse matrix and stacking adjacency matrix on the diagonals.
Figure 6: Three kinds of obstacles: Square pillar(width:10 10, length: 10, height:40), cylinder(radius:4, height:40) and 1albe(15x1xxll
Figure Fluid block and box
(a) Dam Collapse
(b) Water Fall
(c) Flow around Cylinder
Figure 8: Visualizations of initial condition settings in evaluation scenarios
13

---

Under review as a conference paper at ICLR 2021
Dam Collapse
Water Fall
0.6
0.4
Dereay Draktin
Darsit Dosatoon
6.3
c.4
FGN Ground Trush
-4.2
Ground Truo)
-0.3
UMS
OCony
6.4
-0.4
200
600
600
S00
1000
202
Time stop
Weaeett Diverpence
20
10.
PGN
FGN
Groundtrath
30"
Cro nonnndTuuh
OCome
CDaT
GMS
DRS
620
502
1000
203
Time stmp
Time stop
5.05
CCamy
Detrees
2.08
Beetogt DI01000
0.05
0.04
0.02
0.03
2.01
200
400 600
800
1002
200
400 630
800
1000
stop
Figure 9: Error trend with respect to time. The density deviation of our model remain close to zero despite some oscillation at first. Although GNS also maintains low density deviation, its average density oscillates more severely. This consistent with the qualitative comparison, in which the fluid surface from GNS is oscillating and less compact. CConv fails to maintain constant density and its density increases significantly after collision to the wall boundary. Both our model and GNS can predict low divergence velocity field while CConv fails to capture this property. This somehow explains why CConv struggles to maintain constant density. The average Chamfer distance to the ground truth data accumulates at first and stabilizes after system is in equilibrium. CConv has larger average Chamfer distance while graph-based model's distances are smaller.
A.3 METRICS EVALUATION DETAILS
We show the error trend of dam collapse and water fall scene under different evaluation metrics in Figure 9 The position loss analysis of different time step sizes is shown in Figure I0 Position error of complex scenarios shown in Figure IT
A.4 ABLATION STUDY DETAILS
Performance of different graph message aggregate structures is listed in Table 4
A.5 BASELINES IMPLEMENTATION
Continuous Convolution We use the open-source implementation from Ummenhofer et al. (2020)
To give fair benchmark result we train their network with our dataset. Note our training dataset is much smaller than theirs and does not include complex geometries. As in our work, we model
14

---

Under review as a conference paper at ICLR 2021
0.030
0.025
Average Diclance Lim)
0.020
0.010
dt-0.0015
0.005
dt-0.9025
ot-0.0035
0.000
dt=0.004s
0.00
0.25
0.75
1.00
1.75 2.00
Time (sec]
Figure 10: Average Chamfer distance to the ground truth data FGN model under different time step sizes. FGN can generalized different time scale with minor difference in performance. The training data uses time step size of dt = 0.002s, and FGN converges for all dt < 0.005s.
Canyon Flow
Bunny
0.030
0.05
0.025
0.020
0.03
0.015
0.02
0.010
0.030
0.0
2.
75
13.0
12.
15.0
17.5
20.0
Time (sec)
(wer)
Figure 11: Average Chamfer distance to the ground truth data of complex scenes. For both scenes, at first, position error accumulates quickly as system evolves. At some point, the error starts decrease until reaching equilibrium.
Average Chamfer
Density error
Velocity divergence
Case
Model
distance (mm)
Mean
Max
Mean
Max
Mean
Max
FGN (this work)
0.0461
0.0900
0.0195
0.0280
24.2
30.1
GCN
0.0939
0.3832
0.0196
0.0293
27.7
31.6
Dam
SAGE
0.0935
0.3597
0.0196
0.0289
26.4
31.3
Collapse
MLP
0.0963
0.3624
0.0196
0.0289
28.5
32.8
Ground Truth
0.0380
0.0710
0.0190
0.0268
FGN (this work)
0.0541
0.1350
0.0207
0.0431
24.8
29.6
GCN
0.0794
0.3373
0.0210
0.0433
25.5
31.7
Water
SAGE
0.0769
0.3254
0.0209
0.0432
25.3
31.4
Fall
MLP
0.0830
0.2967
0.0209
0.0433
26.3
32.4
Ground Truth
0.0429
0.0966
0.0196
0.0398
Table 4: Quantitative ablation study. We compare our aggregator against graph convolution networks (GCN) from Kipf and Welling (2016), Hamilton al. (2117)ss graph SAGE using mean aggregator and MLP w/o any graph aggregation operation. All aggregators have two layers.
15

---

Under review as a conference paper at ICLR 2021
solid obstacles and wall virtual particles, so transform these virtual particles into surface and corresponding normals before inputting them into CConv network. The original CConv uses a time step size of 0.02s, but given such a large time step size, the qualitative results be distinct from ground truth and other model with smaller time step size. Hence during training and comparison we adopt time step size of 0.002s for CConv model.
Graph Network-bard based Simulator We implemented GNS following the description in Sanchez Gonzalez et al. 2020). We build GNS with 10 unshared GN blocks, conditioned on previous velocities and input relative positions as edge features. We chose the connectivity radius to be 2.I so that the number of neighbors is around 20. [Sanchez-Conzalez et al. (2020) use finite difference array to caleulate acceleration and velocity, but in implementation we explicitly maintain to store the velocities of all particles. Additionally, we do not use learned embedding but simple zero and one to indicate particle material type, in testing domain there only two kinds of particles solid and fluid. The loss function and training procedure are implemented as described in Sanzhez-Gonzalez al. (2020), including noise injection and similar normalization techniques.
For the implementation GN block, as Sanchez- et al. (2020) states "We use GNs without global features global updates (similar an interaction network)" 0", implement the GN block
update mechanism following the description in Battaglia et al. (2018).
e/ = o Vra Veu: u)
(31)
v = de(é/, Vs, u)
(32)
é (
(33)
where is the update function and implemented as MLP here, P is aggregation function which aggregates all the edge attributes to its center vertex, u is the global feature and here we append them the nodal features as input feature, Fx denotes receiver vertices and sy denotes sender vertices.
In the testing stage, as authors did not state how to set initial velocities, so we just warm start the simulation by calculating the first 5 frames using MPS method and apply GNS to the rest frames.
16


Reviews:
[{"review_id": "pKSIpJdDov", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper deals with the prediction of 3D Lagrangian Fluid Simulations. Therefore the problem is divided into 3 subproblems, oriented on numerical simulations. An advection part, where the acceleration of the particles is calculated, a collision step, where the boundary effects are included, and a pressure prediction part, where the pressure for maintaining the volume is determined. A graph-based network is used for each part, which is either node or edge-based according to the requirements. \n\nI think it's a good idea to divide the problem into single subtasks and to orientate on the numerical method. I also find the evaluation of physical properties like density and velocity divergence very relevant and show in my eyes the superiority of the method compared to the state-of-the-art methods. However, there are still some open questions: \n* It is mentioned that tests with different time steps were made as proof of generalization, but I cannot see any corresponding results. \n* Also, I'm not quite sure what the advection network and the collision network calculate exactly. I think it would be more logical if the advection network also performs the integration step, so you could approximate higher-order integration schemes. With the collision network, I'm not quite sure what the GT should be. In most SPH solvers the boundaries are included directly in the pressure computation and not in a separate collision step. In general, I would like to have an ablation study where the relevance of the individual components is more obvious. \n* Another thing is that force-based SPH methods, as adapted in this paper, are usually less stable than position-based methods (PBF), from which state-of-the-art methods are based. I would therefore be interested in a comparison in an extreme scenario, e.g. a setup with a very high water column. I could imagine that the method would fail.\n* Finally, I would recommend adding some videos to the supplementary material, which could prove the temporal coherence of the method.\n\nThere are a few things I noticed about the text itself:\n* First, there are some linguistic errors in the text. \n* Furthermore I would recommend naming the intermediate results in the equations (e.g. equation 10) differently. This can be a bit confusing. \n* As of the last point I found the section about network architecture a bit short and not very informative. I would make it a bit longer and more detailed.\n\nAll in all, I find the method interesting, but in my eyes, there are still some missing points. Unfortunately, I think that the text needs a revision in general because there are quite a lot of linguistic errors. Therefore I would rather vote for a reject.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "WJjKK8AK3i", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nThe paper presents a method for learning Lagrangian fluid dynamics from MPS data. By injecting domain knowledge, and separately training subcomponents of the solver, it achieves low error rates on e.g. divergence.\n\n### Recommendation\nThere are a lot of different approaches for learning physical dynamics, which differ in the amount and type of inductive biases and domain knowledge injected. This is often a tradeoff between generality versus accuracy, and the question of what is a useful balance that has actual advantages over classical solvers is still open.\n\nThat said, I find it hard to make the case for the balance chosen in this paper; the architecture is limited to data from a specific solver (MPS) so unlike general end-to-end approaches (e.g. GNS [Sanchez et al 2020], CConv [Ummenhoffer et al 2019]) it's not a pathway to learning from general data. On the other hand, unlike methods that use a lot of domain knowledge to be more efficient than the ground truth solver (e.g. [Ladicky et al.]), this paper doesn't demonstrate any concrete advantages over MPS. It also doesn't study the impact of its individual choices in great detail, and does not back up some of its claims (generality & generalization, see detailed comments). In its current form, I'm not sure we will learn very much from this method, and hence I recommend rejecting this paper.\n\n### Detailed comments\n- Model: The architecture is very close to the SPH/MPS algorithm in structure. The main difference to a hard-coded solver seems to be that the interpolation kernel is learned, and there is a processor MLP after each block which can act as a corrector. In between sub-block, the output is projected back to position, velocity, so the NN can't learn a richer intermediate representation. The biggest limitation I see it caused by supervising the 3 subcomponent separately. If I understand this correctly, this means like you a) can only train on a solver which has exactly these subcomponents (i.e. MPS, even SPH works slightly differently), b) you need access to solver internals, and c) you will just learn to replicate the solver internals-- and if you can't learn e.g. a better kernel than the solver, why learn it at all?\n- Claims: \n  1. Abstract: *\"our model can adopt a range of time step sizes different from ones using in the training set\"*. While different step sizes are *mentioned* in the results, there is no numbers, figures or other details to substantiate this claim. \n  2. Conclusion: *\"Our model also has generalization capability, where it can remain stable when extrapolating to a wide range of different geometries\"*. The only result related to this I could find is fig. 4. The dataset description was a bit vague, so it's a bit unclear to me that this really is an extrapolation test. Was the cylinder among the objects in the training set? In any case, I don't think a test on a single cylinder obstacle support the statement of \"a wide range of different geometries\".\n  3. Intro: *\"...it can be extended to simulation of other dynamics under Lagrangian framework\"*. I'm not sure this is the case; all the specific details which make this method different from other graph-based approaches *are* specialized to fluids, specifically MPS. How would you e.g. extend this method for elastics or granular materials? For those, there is no pressure or density (depending on approach), but strain/stress etc., and it's unclear which sub-module you'd need or how to supervise them. I'm not even sure this approach would work for Lagrangian fluid data from a different solver (e.g. MPM).\n  4. Although not a direct claim, the method is motivated by the computational cost and bad scaling behavior of classical solvers. However, is this method actually faster than the GT simulator, or can demonstrate better scaling? There's no results on this, and since the method is directly supervised on individual solver components it'd actually be surprised if that is the case.\n- Results: The main result seems to be that divergence is better preserved than in end-to-end approaches like GNS and CConv. This finding is not very surprising, considering that this method directly supervises on the pressure correction module, while the baselines do not. I find it more surprising that the difference to GNS is rather small in fig. 6, even though it does not have a notion of pressure at all.\n\n### Additional questions\n- Just to make sure I have understood the method correctly, how exactly to you train the 3 modules? I.e. where do the labels for supervision come from, are those taken from the corresponding modules in the ground truth simulator?\n- Where does the density come from? Is this computed with a hard-coded interpolation kernel like in SPH? And if yes, why-- as the other modules are specifically build around learning a kernel?\n- Where do you see the advantages of your method compared to using the ground-truth solver?\n\n### Further comments to the authors\nWhile I recommended rejecting this paper in its current form, I think that this is an exciting area to perform research in, and there are many directions worth exploring. I think it's worth thinking about specifically what advantages a learned or semi-learned method can have over ground-truth solvers, and then finding ways of capitalizing on them. Some papers have explored performance, or generality, and the elephant in the room is how to learn from actual real-world sensor data while still exploiting domain knowledge. Also, there's a multitude of papers pushing a specific method, but very little work on thoroughly studying the concrete impact of all the little tricks, choices in representation, and the amount and concrete form of domain knowledge injected.\n\n---\nUpdate: The added generalization tests and performance numbers strengthen the paper, and make the aim more clear. The paper seems now mostly focused on accelerating MPS (or potentially other SPH variants). While the new result table shows speedups compared to MPS for model inference, in practice the model still is bottlenecked by NN search, and total runtime of inference+NN is not significantly different to MPS. And if speed is the aim, the appropriate baseline comparisons should be to methods which -- as this method-- use domain knowledge and solver internals to speed up the solver (e.g. Ladicky et al.).\nOn the method side, it does seems like the proposed approach is limited by supervising specific MPS subcomponent separately, which hinders learning richer intermediate representations, applying the model to other systems, or taking significantly larger timesteps. There could be potential benefits of the specific architecture chosen, but their effect is a bit unclear; as for the comparisons shown the most significant effect is supervision on subcomponents.\nI will therefore keep my score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "MePdS4nmc-l", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper presents a graph neural network approach specialized for Lagrangian fluid simulation.\nInstead of an end-to-end approach a multi-step solution is proposed which divides the computations into 3 distinct networks.\nThe structure of the intermediate results computed by the networks follows the structure of solvers like PBF.\nA goal of this approach is to more accurately model physical properties of fluids.\nThe experiments show comparisons with recent state-of-the-art methods and compare different information aggregation schemes.\n\n\nScore:\nAlthough I think the idea is interesting I tend towards reject. \nThe reason for that is mainly the evaluation, which uses very limited data (number of scenes, sequence length, variety) and is not very convincing.\nThis can be fixed by increasing the test set size and even test on datasets from other works with more variety.\nAnother reason is the very specific network design with multiple small parts in combination with the small timesteps.\nIn comparison with a single end-to-end trained network this approach is less practicable.\nAdditional experiments can help to strengthen the design choices that were made.\n\n\nStrengths:\n* The paper examines important physical quantities such as the density error and velocity divergence.\n* The method is parameter efficient and works with different time steps without retraining.\n* The description of the implementation is easy to follow and Figure 1 gives a good overview.\n\n\nWeaknesses:\n* The proposed network structure is very specific to the problem and the solver used for generating the ground truth.\n  Each of the 3 networks is trained separately imposing the need to compute the intermediate quantities for each of the networks.\n  The sequential network design allows to factor out quantities like the time step which helps with generalization but the strong supervision and the sequential structure also significantly limits the power of the network. \n  This aspect needs to be investigated in detail as this distinguishes this work from other approaches which train end-to-end.\n  It is possible to backpropagate through the update scheme (equations 7 to 12) during training and compare this to the proposed solution.\n  Another comparison can be to remove the intermediate results and let the network output all desired physical quantities in the last layer.\n* The evaluation (Table 1,2) seems to be based on only 2 sequences, which is not enough.\n  The description of the dataset generation process is very short and important information is missing.\n  Are there different types of obstacles or only cylinders? Does the number of particles differ for each scene? What is the number of fluid blocks per scene?\n  Without knowing the variance of the training data and a small test set it is not clear if the evaluation is meaningful.\n* The sequences are very short. The sequence length is only 2 seconds. \n  This is not enough time for the fluid to reach a steady-state, thus the behavior of the model for this particular but important state is not well studied.\n* The generalization to different time steps is good but the generalization to different scenes seems very limited.\n  All scenes are box-like environments.\n* Learning physics is often motivated by reducing computation costs but there is no information about the computation time.\n  Further, the time steps used in the paper are quite small, which increases the number of iterations needed for long simulations and significantly simplifies the learning task.\n\n\nQuestions:\n\nSection 5.1 mentions that the particle density inputs are normalized for the pressure network.\nThis is counterintuitive since density is an important feature for computing the pressure.\nWhy is this necessary to stabilize the training?\n\n\n\nMinor comments:\n\nThere are some minor problems with the writing. Here are some for the first page:\n\nAbstract: \n- Our model uses _a_ graph ...\n\nIntroduction:\n- fluids is an essential -> fluids _are_\n- a large class of numerical models have -> a large class of numerical models _has_\n- usually increase drastically when resolution -> usually _increases_ drastically when _the_ resolution\n- data-driven model -> data-driven _models_\n- dynamics under Lagrangian -> dynamics under _the_ Lagrangian\n- universal feature (...) under Lagrangian framework as input -> universal _features_ (...) as input \n\nRelated Works:\n- build upon Lagrangian representation of fluid -> build upon a Lagrangian fluid representation\n- when material interface -> when the material interface\n\nWill the code be released? If not it would be important to have a section in the appendix with all the important parameters for reproducing the networks in tabular form (layer sizes, normalizations, activations, etc.).", "extra_scores": {}, "extra_text": {}}, {"review_id": "MBNgUrZ_wf", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a learned model specialized on learning Lagrangian fluid dynamics for incompressible fluids. The model is a hybrid between a simulator with explicit advection, collision and pressure correction stages, and a learned model, trained by supervising each of those stages. The authors demonstrate improved stability/conservation of physical properties for a model, and some flexibility to the time-step being changed at test time.\n\nThe main general disadvantage I see is that the model seems oddly specific to the simulator used to generate the data, and specifically to incompressible fluids. The authors say “Although our model is entailed and customized for fluid simulation, it can be extended to simulation of other dynamics under Lagrangian framework as it takes universal feature“, but I am not sure how this work specifically supports extensions, since the main difference between the model and the baselines is precisely that while the baselines are general, this model specializes itself on one very specific type of simulation. Beyond this, there are some other important questions that I have listed below.\n\nThe advantages/claims are improved stability of divergence and density, and less learned parameters. The stability is not too surprising, since the model is built exactly to match the mechanisms that the simulator has to improve those, and in a sense it is using privileged information about the data generation. I would argue that it would have been easier to achieve the same in the baselines, just by adding additional loss components that explicitly minimize divergence and density error, using the same use of the privilege information. The smaller number of learned parameters is a nice result that potentially means the model can also run much faster than the baselines, so maybe this is the most important contribution: even assuming the model is meant to be specific for a domain, it can bring value by showing that it can be much faster than the baselines, and much faster than the ground truth simulator itself. Otherwise, I am not sure what the main selling point of a highly domain specific architecture is in this case.\n\nAll things considered, I am not sure the current state of the paper is sufficient to be accepted, both in terms of generality of the approach/main selling point, quality of the model description and comprehensiveness of the results. So my current rating is Reject (4) which I would be happy to immediately raise to Weak Reject (5) if the authors can confirm that they do not need access to intermediate simulator targets for training (Q1), and reconsider from there, given satisfactory explanations, additional evidence and discussion with other reviewers.\n\nMain Comments/Questions:\n\n1. The papers says:\n\n> We train three networks, advection net, collision net and pressure net separately.\n\nBut also things like:\n\n> the advection net predicts acceleration of particles (a^adv)\n> The collision net takes … the relative position and velocity in intermediate state (let’s call this x*, v*)\n> predicts correction to the velocity (Let’s call this \\Delta v*, and the output v**)\n> the updated intermediate position and velocity are take n as input by the pressure net (x*, v**0)\n> predicts pressure (let’s call this p)\n\nWhat are the inputs and targets of the advection net, the collision net and pressure net at train time, respectively? Does this means that apart from x^n and x^{n+1} your model needs access to all of the internal values of the simulator after each internal stage: (e.g., a^adv, x*, v*, v**, p), so you can supervise those when training each network separately. This would be a pretty strong requirement that the baselines do not require. How would you run this model on data obtained from a real experiment (assuming you had a way to do particle tracking, of course)? Or are in on the contrary the three models trained in order, always taking as inputs the outputs form the learned previous stage, and building the loss against the final position and velocity targets? The section around Equation (16) seems to imply that it is the former. If that is the case, improvements over baselines seem like a small gain considering the required access to all of that simulator-specific intermediate information.\n\n2. In general the model description is quite hard to follow. For example, does the edge-focused and node-focused graph networks have a single step of message passing (later you talk about two layers of aggregation, but it is not clear how these two relate). Does the node focus graph network not embed the nodes features feature the aggregation? A lot of these decisions seem to deviate a bit from standard choices, and should probably be justified and explained more in detail. \n\nHow is \\Nabla p (gradient of the pressure) calculated, do you actually take the gradients of the neural network with respect to its own input positions? Or do you use equation (19) to get a finite different estimate? Assuming it is the latter, is then the model not very sensitive to the connectivity radius of this network? \n\n3. How is density error and velocity divergence defined? Specifically, in Figure 6, density deviation takes both positive and negative values, but in the table it takes only positive values. \nAssuming this is calculated with equations (18) and (20), are these metrics not highly dependent on the neighborhood radius chosen? Is the neighborhood radius the same for the model aggregations than for the metrics? In that case, why is that?\n\nWould it make sense to also report error as a function of the mse of the position somewhere, which has been the main metric in the baselines work?\n\n4. \n> we can observe oscillation on the free surface of fluids\n> maintains a much more compact and smoother shape\n> CConv fail to maintain smooth and compact fluid distributions\n> GNS’ prediction is significantly slower than ground truth\n\nIt would also be very useful to be able to see some videos of the trajectories for the different models, to better understand the differences that the authors mention, rather than a single trajectory per dataset sampled at 5 points. Also, what does “slow” mean in this context?\n\n5. In Fig 2(a) the first frame for GNS (t=100), seems very different from the rest, what is the reason for this? In similar DamCollapse domains in the GNS paper the accuracy seemed better somehow. Was the noise used to adjust the targets too, similar to the GNS model?\n\n6. Any reason why the radius of connectivity is set differently for the advection/pressure net than for the collision net? Was the model sensitive to this?\n\n7. Results section seems too short, only 16 lines worth of text, compared to two pages worth of tables and figures. I think more discussion is needed, and maybe more investigation of what specifically makes the model better. For example:\n\n* Could the model be trained end to end, instead of training the three networks separately?\n\n* The generalization section does not really elaborate enough about what the differences between training and generation datasets. The problem partly is that the dataset generation description itself is not very comprehensive, and seems hard to reproduce without access to the source code.\n\n* The generalization to other time-steps should be studied in more detail, both how it was implemented, and what the results are for more both shorter and longer timesteps (rather than just the qualitative figure 3, with a single generalization time-step), and how specifically was the attempt to make it work for the baselines implemented.\n\n\nMinor comments/typos (did not affect my decision):\n\n“physic-informed” should be “physics-informed”?\n“we aggregates” → (without s)\n“are passed to processor” → “to the”\nW in equation (13) is not defined in the text. Possibly add W next to: “using smooth\nkernel as weight function”\n\nThe value for max density error, Dam collapse, ground truth, seems different in Table1 and Table2.\n\nI would maybe put ground truth as the first row of the results tables for easier comparison to the first row, which has the strongest performance.\n\nIn Figure 6, could you make it so the color legend is consistent across plots, currently it is very confusing.\n\nAnd in general the grammar needs a bit more work (especially in the 3.2 model section).\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The consensus recommendation is that the paper is not ready for publication at this time.
Decision: Reject