Learning Incompressible Fluid Dynamics from Scratch - Towards Fast, Differentiable Fluid Models that Generalize

1 INTRODUCTION . Simulating the behavior of fluids by solving the incompressible Navier-Stokes equations is of great importance for a wide range of applications and accurate as well as fast fluid simulations are a long-standing research goal . On top of simulating the behavior of fluids , several applications such as sensitivity analysis of fluids or gradient-based control algorithms rely on differentiable fluid simulators that allow to propagate gradients throughout the simulation ( Holl et al . ( 2020 ) ) . Recent advances in deep learning aim for fast and accurate fluid simulations but rely on vast datasets and / or do not generalize to new fluid domains . Kim et al . ( 2019 ) present a framework to learn parameterized fluid simulations and allow to interpolate efficiently in between such simulations . However , their work does not generalize to new domain geometries that lay outside the training data . Kim & Lee ( 2020 ) train a RNN-GAN that produces turbulent flow fields within a pipe domain , but do not show generalization results beyond pipe domains . Xie et al . ( 2018 ) introduce a tempoGAN to perform temporally consistent superresolution of smoke simulations . This allows to produce plausible high-resolution smoke-density fields for arbitrary low-resolution inputs , but our fluid model should output a complete fluid state description consisting of a velocity and a pressure field . Tompson et al . ( 2017 ) present how a Helmholtz projection step can be learned to accelerate Eulerian fluid simulations . This method generalizes to new domain geometries , but a particle tracer is needed to deal with the advection term of the Navier-Stokes equations . Furthermore , as Eulerian fluids do not model viscosity , effects like e.g . the Magnus effect or Kármán vortex streets can not be simulated . Geneva & Zabaras ( 2020 ) propose a physics-informed framework to learn the entire update step for the Burgers equations in 1D and 2D , but no generalization results for new domain geometries are demonstrated . All of the aforementioned methods rely on the availability of vast amounts of data from fluid-solvers such as FEniCS , OpenFOAM or Mantaflow . Most of these methods do not generalize well or outsource a major part of the fluid simulation to traditional methods such as low-resolution fluid solvers or a particle tracer . In this work , we propose a novel unsupervised training framework to learn incompressible fluid dynamics from scratch . It does not require any simulated fluid-data ( neither as ground truth data , nor to train an adversarial network , nor to initialize frames for a physics-constrained loss ) and generalizes to fluid domains unseen during training . It allows CNNs to learn the entire update-step of mapping a fluid domain from time-point t to t + dt without having to rely on low resolution fluid-solvers or a particle-tracer . In fact , we will demonstrate that a physicsconstrained loss function combined with a simple strategy to recycle fluid-data generated by the neural network at training time suffices to teach CNNs fluid dynamics on increasingly realistic statistics of fluid states . This drastically simplifies the training pipeline . Fluid simulations get efficiently unrolled in time by recurrently applying the trained model on a fluid state . Furthermore , the fluid models include viscous friction and handle effects such as the Magnus effect and Kármán vortex streets . On top of that , we show by a gradient-based optimal control example how backpropagation through time can be used to differentiate the fluid simulation . Code and pretrained models are publicly available at https : //github.com/aschethor/ Unsupervised_Deep_Learning_of_Incompressible_Fluid_Dynamics/ . 2 RELATED WORK . In literature , several different approaches can be found that aim to approximate the dynamics of PDEs in general and fluids in particular with efficient , learning-based surrogate models . Lagrangian methods such as smoothed particle hydrodynamcs ( SPH ) Gingold & Monaghan ( 1977 ) handle fluids from the perspective of many individual particles that move with the velocity field . Following this approach , learning-based methods using regression forests by Ladický et al . ( 2015 ) , graph neural networks by Mrowca et al . ( 2018 ) ; Li et al . ( 2019 ) and continuous convolutions by Ummenhofer et al . ( 2020 ) have been developed . In addition , Smooth Particle Networks ( SP-Nets ) by Schenck & Fox ( 2018 ) allow for differentiable fluid simulations within the Lagrangian frame of reference . These Lagrangian methods are particularly suitable when a fluid domain exhibits large , dynamic surfaces ( e.g . waves or droplets ) . However , to simulate the dynamics within a fluid domain accurately , Eulerian methods , that treat the Navier-Stokes equations in a fixed frame of reference , are usually better suited . Continuous Eulerian methods allow for mesh-free solutions by mapping domain coordinates ( e.g . x , y , t ) directly onto field values ( e.g . velocity ~v / pressure p ) ( Sirignano & Spiliopoulos ( 2018 ) ; Grohs et al . ( 2018 ) ; Khoo et al . ( 2019 ) ) . Recent applications focused on flow through porous media ( Zhu & Zabaras ( 2018 ) ; Zhu et al . ( 2019 ) ; Tripathy & Bilionis ( 2018 ) ) , fluid modeling ( Yang et al . ( 2016 ) ; Raissi et al . ( 2018 ) ) , turbulence modeling ( Geneva & Zabaras ( 2019 ) ; Ling et al . ( 2016 ) ) and modeling of molecular dynamics ( Schöberl et al . ( 2019 ) ) . Training is usually based on physics-constrained loss functions that penalize residuals of the underlying PDEs . Similar to our approach , Raissi et al . ( 2019 ) uses vector potentials to obtain continuous divergence-free velocity fields to approximate the incompressible Navier-Stokes equations . Continuous methods return smooth , accurate results and can overcome the curse of dimensionality of discrete techniques in high-dimensional PDEs ( Grohs et al . ( 2018 ) ) . However , these networks are trained on a specific domain and can not generalize to new environments or be used in interactive scenarios . Discrete Eulerian methods , on the other hand , aim to solve the underlying PDEs on a grid and early work dates back to Harlow & Welch ( 1965 ) and Stam ( 1999 ) . Accelerating such traditional works with deep learning techniques is a major field of research and all of the methods mentioned in the introduction fall into this category . Further methods include the approach by Thuerey et al . ( 2019 ) to learn solutions of the Reynolds-averaged Navier-Stokes equations for airfoil flows , but requires large amounts of training data and does not generalize beyond airfoil flows . In the work by Um et al . ( 2020 ) , a correction step is learned that brings solutions of a low-resolution differentiable fluid solver closer to solutions of a high-resolution fluid simulation . However , generalization results for new domain geometries were not presented . The works of Mohan et al . ( 2020 ) and Kim et al . ( 2019 ) show that vector potentials are suitable to enforce the incompressibility constraint in fluids but do not generalize to new fluid domains beyond their training data . 3 METHOD . In this section , we briefly review the incompressible Navier-Stokes equations , which are to be solved by the neural network . Then , we explain how the Helmholtz decomposition can be exploited to ensure incompressibility within the fluid domain . Furthermore , we provide details of our discrete spatio-temporal fluid representation and introduce the fluid model . Afterwards , we formulate a physics-constrained loss function based on residuals of the Navier-Stokes equations and introduce a pressure regularization term for very high Reynolds numbers . Finally , we explain the unsupervised training strategy . 3.1 INCOMPRESSIBLE NAVIER-STOKES EQUATIONS . Most fluids can be modeled with the incompressible Navier-Stokes equations - a set of non-linear equations that describe the interplay of a velocity field ~v and a pressure field p within a fluid domain Ω : ∇ · ~v = 0 incompressibility on Ω ( 1 ) ρ~̇v = ρ ( ∂~v ∂t + ( ~v · ∇ ) ~v ) = −∇p+ µ∆~v + ~f conservation of momentum on Ω ( 2 ) Here , ρ describes the fluid density and µ the viscosity . Equation 1 states that the fluid is incompressible and thus ~v is divergence-free . Equation 2 states that the change in momentum of fluid particles must correspond to the sum of forces that arise from the pressure gradient , viscous friction and external forces . Here , external forces on the fluid ( such as e.g . gravity ) can be neglected , so we set ~f = 0 . These incompressible Navier-Stokes equations shall be solved by a CNN given initial conditions ~v0 and p0 at the beginning of the simulation and Dirichlet boundary conditions which constrain the velocity field at the domain boundary ∂Ω : ~v = ~vd Dirichlet boundary condition on ∂Ω ( 3 ) 3.2 HELMHOLTZ DECOMPOSITION . A common method to ensure incompressibility of a fluid ( see Equation 1 ) is to project the flow field onto the divergence-free part of its Helmholtz decomposition . The Helmholtz theorem states that every vector field ~v can be decomposed into a curl-free part ( ∇q ) and a divergence-free part ( ∇×~a ) : ~v = ∇q +∇× ~a ( 4 ) Note , that ∇ × ( ∇q ) = ~0 and ∇ · ( ∇ × ~a ) = 0 . The Helmholtz projection consists of solving the Poisson problem ∇ · ~v = ∆q for q , followed by substracting ∇q from the original flow field . However , solving the Poisson equation on arbitrary domains comes at high computational costs for classical methods and one has to rely e.g . on conjugate gradient methods to approximate its solution . Here , we propose a different approach and directly try to learn a vector potential ~a with ~v = ∇×~a . This ensures that the network outputs a divergence-free velocity field within the domain Ω and automatically solves Equation 1 . In this work , we consider 2D fluid simulations , so only the zcomponent of ~a , az , is of interest since vz and all derivatives with respect to the z-axis are zero : ∇× ~a = ( ∂yaz − ∂zay ∂zax − ∂xaz ∂xay − ∂yax ) = ( ∂yaz −∂xaz 0 ) = ( vx vy 0 ) = ~v ( 5 ) 3.3 DISCRETE SPATIO-TEMPORAL FLUID REPRESENTATION . Marker-And-Cell ( MAC ) grid To solve the Navier-Stokes equations , we represent the relation between az , vx , vy , p on a 2D staggered marker-and-cell ( MAC ) grid ( see Figure 1a ) . Therefore , we discretise time and space as follows : ~a ( x , y , t ) = 00 ( az ) t i , j ; ~v ( x , y , t ) = ( ( vx ) ti , j ( vy ) t i , j ) ; p ( x , y , t ) = pti , j ( 6 ) Obtaining gradient , divergence , Laplace and curl operations on this grid with finite differences is straight forward and can be efficiently implemented with convolutions ( see appendix A ) . Explicit , Implicit , Implicit-Explicit ( IMEX ) time integration methods The discretization of the time domain is needed to deal with the time-derivative of the velocity field ∂~v∂t in Equation 2 , which becomes : ρ ( ~vt+dt − ~vt dt + ( ~vt ′ · ∇ ) ~vt ′ ) = −∇pt+dt + µ∆~vt ′ + ~f ( 7 ) The goal is to take as large as possible timesteps dt while maintaining stable and accurate solutions . Stability and accuracy largely depend on the definition of vt ′ . In literature , choosing vt ′ = vt is often referred to as explicit integration methods and frequently leads to unstable behavior . Choosing vt ′ = vt+dt is usually associated with implicit integration methods and gives stable solutions at the cost of numerical dissipation . Implicit-Explicit ( IMEX ) methods , which set vt ′ = ( vt + vt+dt ) /2 are a compromise between both methods and considered to be more accurate but less stable than implicit methods .

Summary: This paper presents a "physics-informed" deep learning model of fluid dynamics. The underlying deep learning architecture employed is a somewhat standard u-net, but one of the proposed method's distinguishing features is that it enforces its adherence to physical behavior at its loss terms, by penalizing predictions that are not incompressible or do not conserve momentum. Notably, this approach allows it to be trained unsupervisedly, without requiring the generation of ground-truth simulations.