Learning Incompressible Fluid Dynamics from Scratch - Towards Fast, Differentiable Fluid Models that Generalize

1 INTRODUCTION . Simulating the behavior of fluids by solving the incompressible Navier-Stokes equations is of great importance for a wide range of applications and accurate as well as fast fluid simulations are a long-standing research goal . On top of simulating the behavior of fluids , several applications such as sensitivity analysis of fluids or gradient-based control algorithms rely on differentiable fluid simulators that allow to propagate gradients throughout the simulation ( Holl et al . ( 2020 ) ) . Recent advances in deep learning aim for fast and accurate fluid simulations but rely on vast datasets and / or do not generalize to new fluid domains . Kim et al . ( 2019 ) present a framework to learn parameterized fluid simulations and allow to interpolate efficiently in between such simulations . However , their work does not generalize to new domain geometries that lay outside the training data . Kim & Lee ( 2020 ) train a RNN-GAN that produces turbulent flow fields within a pipe domain , but do not show generalization results beyond pipe domains . Xie et al . ( 2018 ) introduce a tempoGAN to perform temporally consistent superresolution of smoke simulations . This allows to produce plausible high-resolution smoke-density fields for arbitrary low-resolution inputs , but our fluid model should output a complete fluid state description consisting of a velocity and a pressure field . Tompson et al . ( 2017 ) present how a Helmholtz projection step can be learned to accelerate Eulerian fluid simulations . This method generalizes to new domain geometries , but a particle tracer is needed to deal with the advection term of the Navier-Stokes equations . Furthermore , as Eulerian fluids do not model viscosity , effects like e.g . the Magnus effect or Kármán vortex streets can not be simulated . Geneva & Zabaras ( 2020 ) propose a physics-informed framework to learn the entire update step for the Burgers equations in 1D and 2D , but no generalization results for new domain geometries are demonstrated . All of the aforementioned methods rely on the availability of vast amounts of data from fluid-solvers such as FEniCS , OpenFOAM or Mantaflow . Most of these methods do not generalize well or outsource a major part of the fluid simulation to traditional methods such as low-resolution fluid solvers or a particle tracer . In this work , we propose a novel unsupervised training framework to learn incompressible fluid dynamics from scratch . It does not require any simulated fluid-data ( neither as ground truth data , nor to train an adversarial network , nor to initialize frames for a physics-constrained loss ) and generalizes to fluid domains unseen during training . It allows CNNs to learn the entire update-step of mapping a fluid domain from time-point t to t + dt without having to rely on low resolution fluid-solvers or a particle-tracer . In fact , we will demonstrate that a physicsconstrained loss function combined with a simple strategy to recycle fluid-data generated by the neural network at training time suffices to teach CNNs fluid dynamics on increasingly realistic statistics of fluid states . This drastically simplifies the training pipeline . Fluid simulations get efficiently unrolled in time by recurrently applying the trained model on a fluid state . Furthermore , the fluid models include viscous friction and handle effects such as the Magnus effect and Kármán vortex streets . On top of that , we show by a gradient-based optimal control example how backpropagation through time can be used to differentiate the fluid simulation . Code and pretrained models are publicly available at https : //github.com/aschethor/ Unsupervised_Deep_Learning_of_Incompressible_Fluid_Dynamics/ . 2 RELATED WORK . In literature , several different approaches can be found that aim to approximate the dynamics of PDEs in general and fluids in particular with efficient , learning-based surrogate models . Lagrangian methods such as smoothed particle hydrodynamcs ( SPH ) Gingold & Monaghan ( 1977 ) handle fluids from the perspective of many individual particles that move with the velocity field . Following this approach , learning-based methods using regression forests by Ladický et al . ( 2015 ) , graph neural networks by Mrowca et al . ( 2018 ) ; Li et al . ( 2019 ) and continuous convolutions by Ummenhofer et al . ( 2020 ) have been developed . In addition , Smooth Particle Networks ( SP-Nets ) by Schenck & Fox ( 2018 ) allow for differentiable fluid simulations within the Lagrangian frame of reference . These Lagrangian methods are particularly suitable when a fluid domain exhibits large , dynamic surfaces ( e.g . waves or droplets ) . However , to simulate the dynamics within a fluid domain accurately , Eulerian methods , that treat the Navier-Stokes equations in a fixed frame of reference , are usually better suited . Continuous Eulerian methods allow for mesh-free solutions by mapping domain coordinates ( e.g . x , y , t ) directly onto field values ( e.g . velocity ~v / pressure p ) ( Sirignano & Spiliopoulos ( 2018 ) ; Grohs et al . ( 2018 ) ; Khoo et al . ( 2019 ) ) . Recent applications focused on flow through porous media ( Zhu & Zabaras ( 2018 ) ; Zhu et al . ( 2019 ) ; Tripathy & Bilionis ( 2018 ) ) , fluid modeling ( Yang et al . ( 2016 ) ; Raissi et al . ( 2018 ) ) , turbulence modeling ( Geneva & Zabaras ( 2019 ) ; Ling et al . ( 2016 ) ) and modeling of molecular dynamics ( Schöberl et al . ( 2019 ) ) . Training is usually based on physics-constrained loss functions that penalize residuals of the underlying PDEs . Similar to our approach , Raissi et al . ( 2019 ) uses vector potentials to obtain continuous divergence-free velocity fields to approximate the incompressible Navier-Stokes equations . Continuous methods return smooth , accurate results and can overcome the curse of dimensionality of discrete techniques in high-dimensional PDEs ( Grohs et al . ( 2018 ) ) . However , these networks are trained on a specific domain and can not generalize to new environments or be used in interactive scenarios . Discrete Eulerian methods , on the other hand , aim to solve the underlying PDEs on a grid and early work dates back to Harlow & Welch ( 1965 ) and Stam ( 1999 ) . Accelerating such traditional works with deep learning techniques is a major field of research and all of the methods mentioned in the introduction fall into this category . Further methods include the approach by Thuerey et al . ( 2019 ) to learn solutions of the Reynolds-averaged Navier-Stokes equations for airfoil flows , but requires large amounts of training data and does not generalize beyond airfoil flows . In the work by Um et al . ( 2020 ) , a correction step is learned that brings solutions of a low-resolution differentiable fluid solver closer to solutions of a high-resolution fluid simulation . However , generalization results for new domain geometries were not presented . The works of Mohan et al . ( 2020 ) and Kim et al . ( 2019 ) show that vector potentials are suitable to enforce the incompressibility constraint in fluids but do not generalize to new fluid domains beyond their training data . 3 METHOD . In this section , we briefly review the incompressible Navier-Stokes equations , which are to be solved by the neural network . Then , we explain how the Helmholtz decomposition can be exploited to ensure incompressibility within the fluid domain . Furthermore , we provide details of our discrete spatio-temporal fluid representation and introduce the fluid model . Afterwards , we formulate a physics-constrained loss function based on residuals of the Navier-Stokes equations and introduce a pressure regularization term for very high Reynolds numbers . Finally , we explain the unsupervised training strategy . 3.1 INCOMPRESSIBLE NAVIER-STOKES EQUATIONS . Most fluids can be modeled with the incompressible Navier-Stokes equations - a set of non-linear equations that describe the interplay of a velocity field ~v and a pressure field p within a fluid domain Ω : ∇ · ~v = 0 incompressibility on Ω ( 1 ) ρ~̇v = ρ ( ∂~v ∂t + ( ~v · ∇ ) ~v ) = −∇p+ µ∆~v + ~f conservation of momentum on Ω ( 2 ) Here , ρ describes the fluid density and µ the viscosity . Equation 1 states that the fluid is incompressible and thus ~v is divergence-free . Equation 2 states that the change in momentum of fluid particles must correspond to the sum of forces that arise from the pressure gradient , viscous friction and external forces . Here , external forces on the fluid ( such as e.g . gravity ) can be neglected , so we set ~f = 0 . These incompressible Navier-Stokes equations shall be solved by a CNN given initial conditions ~v0 and p0 at the beginning of the simulation and Dirichlet boundary conditions which constrain the velocity field at the domain boundary ∂Ω : ~v = ~vd Dirichlet boundary condition on ∂Ω ( 3 ) 3.2 HELMHOLTZ DECOMPOSITION . A common method to ensure incompressibility of a fluid ( see Equation 1 ) is to project the flow field onto the divergence-free part of its Helmholtz decomposition . The Helmholtz theorem states that every vector field ~v can be decomposed into a curl-free part ( ∇q ) and a divergence-free part ( ∇×~a ) : ~v = ∇q +∇× ~a ( 4 ) Note , that ∇ × ( ∇q ) = ~0 and ∇ · ( ∇ × ~a ) = 0 . The Helmholtz projection consists of solving the Poisson problem ∇ · ~v = ∆q for q , followed by substracting ∇q from the original flow field . However , solving the Poisson equation on arbitrary domains comes at high computational costs for classical methods and one has to rely e.g . on conjugate gradient methods to approximate its solution . Here , we propose a different approach and directly try to learn a vector potential ~a with ~v = ∇×~a . This ensures that the network outputs a divergence-free velocity field within the domain Ω and automatically solves Equation 1 . In this work , we consider 2D fluid simulations , so only the zcomponent of ~a , az , is of interest since vz and all derivatives with respect to the z-axis are zero : ∇× ~a = ( ∂yaz − ∂zay ∂zax − ∂xaz ∂xay − ∂yax ) = ( ∂yaz −∂xaz 0 ) = ( vx vy 0 ) = ~v ( 5 ) 3.3 DISCRETE SPATIO-TEMPORAL FLUID REPRESENTATION . Marker-And-Cell ( MAC ) grid To solve the Navier-Stokes equations , we represent the relation between az , vx , vy , p on a 2D staggered marker-and-cell ( MAC ) grid ( see Figure 1a ) . Therefore , we discretise time and space as follows : ~a ( x , y , t ) = 00 ( az ) t i , j ; ~v ( x , y , t ) = ( ( vx ) ti , j ( vy ) t i , j ) ; p ( x , y , t ) = pti , j ( 6 ) Obtaining gradient , divergence , Laplace and curl operations on this grid with finite differences is straight forward and can be efficiently implemented with convolutions ( see appendix A ) . Explicit , Implicit , Implicit-Explicit ( IMEX ) time integration methods The discretization of the time domain is needed to deal with the time-derivative of the velocity field ∂~v∂t in Equation 2 , which becomes : ρ ( ~vt+dt − ~vt dt + ( ~vt ′ · ∇ ) ~vt ′ ) = −∇pt+dt + µ∆~vt ′ + ~f ( 7 ) The goal is to take as large as possible timesteps dt while maintaining stable and accurate solutions . Stability and accuracy largely depend on the definition of vt ′ . In literature , choosing vt ′ = vt is often referred to as explicit integration methods and frequently leads to unstable behavior . Choosing vt ′ = vt+dt is usually associated with implicit integration methods and gives stable solutions at the cost of numerical dissipation . Implicit-Explicit ( IMEX ) methods , which set vt ′ = ( vt + vt+dt ) /2 are a compromise between both methods and considered to be more accurate but less stable than implicit methods .

Summary: This paper proposes to learn the dynamics of an incompressible fluid via a physics informed loss formulation using an unsupervised training framework. It employs a custom solver that is executed at training time to learn a Navier-Stokes residual with a incompressible (curl of a stream function) formulation. This setup is demonstrated for two dimensional karman vortex streets, and a control example for the magnus effect.