Learning Time-Aware Assistance Functions for Numerical Fluid Solvers

1 INTRODUCTION . Numerical methods are a central component of many disciplines and widely used for solving a variety of linear and nonlinear problems . One of the long-standing targets is fluid flow , which is renowned for its great diversity and complexity in terms of dynamics . Studies in computational fluid dynamics have focused on numerical simulations for such problems and invested huge efforts in solving spatio-temporal partial differential equations ( PDEs ) such as the Navier-Stokes equations , which represent the well-established physical model for fluids . Traditional methods typically improve accuracy with fine discretizations both in space and time . While the methods and computing power for numerical simulation have seen advances in recent years , there is still a pressing need for better efficiency and accuracy . For most practical applications of computer simulations , we are still far away from fully resolving all necessary scales of nature around us ( Verma et al. , 2018 ; Cummins et al. , 2018 ) . To tackle this problem , we propose a data-driven approach that “ assists ” a given numerical method to improve its accuracy . To this end , we introduce a first learning-based approach that puts special emphasis on the time dimension . We demonstrate two variants to achieve this goal in the context of fluids : a supervised version with an optimization algorithm for acquisition of temporally constrained correction data and an unsupervised version with a differentiable PDE solver that allows us to autonomously takes into account temporal information when training . We compare advantages and disadvantages of both approaches , and our experiments show that , using our trained models , the simulation accuracy of the given solver can be significantly improved . In all cases , our trained models yield improved dynamics , and the learned assistance function lets a coarse simulation reproduce the behavior of the reference data more closely . In particular , we demonstrate the improvements of our approach over ad-hoc learning approaches . 2 RELATED WORK . Data-driven approaches were shown to be highly effective for inferring unknown PDEs and coefficients of corresponding terms ( Brunton et al. , 2016 ) . Their capabilities were investigated for different problems , among others the Navier-Stokes equations ( Rudy et al. , 2017 ; Schaeffer , 2017 ) . Deep learning methods were successfully applied for learning stencils of advection diffusion problems ( Bar-Sinai et al. , 2018 ) , to discover PDE formulations ( Long et al. , 2017 ) , and to analyze families of Poisson equations ( Magill et al. , 2018 ) . While identifying governing equations represents an interesting and challenging task , we instead focus on a general method to improve the solutions of chosen solution spaces . Due to the complexity and nonlinear nature of fluid dynamics , they remain a challenging problem , and accordingly data-driven approaches including deep learning have received a lot of attention to reach an efficient solution for such problems ( Kutz , 2017 ) . Particularly , a turbulence modeling has been a representative objective in this context ( Duraisamy et al. , 2015 ; Maulik & San , 2017 ; Beck et al. , 2018 ; Mohan et al. , 2019 ) . Moreover , both steady ( Guo et al. , 2016 ) and unsteady ( Morton et al. , 2018 ) flows as well as multiphase ( Gibou et al. , 2018 ) flows have been investigated with deep learning based approaches . Additionally , the convolutional neural network ( CNN ) based methods were studied for airfoil flow problems ( Thuerey et al. , 2018 ; Zhang et al. , 2018 ) . For fluid simulation , particularly in the field of computer graphics , data-driven approaches have been considered as an efficient alternative to replace computationally expensive steps of numerical processes ( Ladický et al. , 2015 ; Tompson et al. , 2017 ) . Moreover , deep learning , particularly with generative adversarial models ( Goodfellow , 2016 ) , has been used for efficiently synthesizing details within coarsely resolved simulations ( Chu & Thuerey , 2017 ; Xie et al. , 2018 ) . In addition to scalar transport and smoke flows , liquid droplets have been efficiently tackled by learning stochastic models ( Um et al. , 2018 ) . Due to their capabilities for transforming space-time data into reduced latent spaces , fluid simulations were processed and re-simulated with neural network ( NN ) models ( Kim et al. , 2019 ; Prantl et al. , 2019 ) . Motivated by the success of previous studies regarding datadriven approaches in numerical simulations , we aim for enabling the learning of general correction functions . Our aim differs from methods that synthesize details for low-resolution input data . As such , our method is orthogonal to super-resolution methods and shares similarities with methods to identify discontinuities in finite difference solutions ( Ray & Hesthaven , 2018 ) . However , we investigate the problem for challenging two-dimensional flow problems and propose a new approach for learning the correction bby taking into account temporal dynamics . 3 CORRECTING NUMERICAL SIMULATIONS . Numerical methods yield approximations of a smooth function f ( x ) in a discrete setting with an approximation error of the form O ( hk ) where h is the step size of the discretization . Naturally , higher order methods with larger k are preferable but difficult to arrive at in practice . In our setting , the motions of fluids can be represented by a series of vector fields v ( x ) where bold vectors will denote two- or three-dimensional vector-valued functions in the following . The accuracy of the fluid motion likewise depends on the discretization error for v. In the following , we will focus on Cartesian Eulerian representations , meaning axis aligned grids with square cells . Discretizing functions like v on such a grid yields approximations that scale with the grid spacing h. While small h can yield accurate representations , the computational requirements typically increase superlinearly with the number of unknowns , and as such , all practical methods strive for keeping h as large as possible in order to compute solutions quickly . Due to the nonlinear nature of flow equations and the inherent truncation errors of numerical schemes , varying resolutions often lead to very significant differences in solutions . Such differences are not readily interchangeable across the different resolutions , as especially higher frequency content can be difficult to represent on a coarse grid . In order to improve the accuracy of a numerical method , we propose to learn a correction function that compensates for discretization errors on coarse grids and is representable by a NN . More specifically , we employ a CNN ( Krizhevsky et al. , 2012 ) as the correction should be translation invariant for generalization . We target time-dependent problems for which we infer a per time step correction . This is particularly challenging as approximation errors accumulate over time . In this work , we will focus on finite difference approximations ( Strikwerda , 2004 ) . A discretized representation s of a smooth target function f typically consists of a set of discrete physical quantities . Together , we call these discrete quantities s a state with which our correction approach can be formulated as follows : sn+1 = F ( sn ) + cn ( 1 ) where F denotes a numerical solver for the target function f moving a state sn forward in time to a new state sn+1 . Here , cn denotes the correction function . Note that cn potentially updates only a part of the full state , e.g. , the velocity field v. We target solutions of the Navier-Stokes equations , which for incompressible Newtonian fluids takes the form ∂v ∂t + v · ∇v = −1 ρ ∇p+ ν∇ · ∇v + g subject to ∇ · v = 0 , ( 2 ) where ρ , p , ν , and g denote density , pressure , viscosity , and external forces , respectively . The constraint , ∇ · v = 0 , is particularly important as it restricts motions to the space of divergence-free ( i.e. , volume preserving ) velocities . 3.1 CORRECTION FUNCTION . The natural follow-up question is how to compute and represent the potentially highly nonlinear correction function of Eq . 1 . Thanks to their flexibility and ability to represent nonlinearities , we use fully convolutional networks as our model for the correction function . A fine discretization , optionally with advanced numerical schemes , is employed to compute the reference data . Our goal is , then , to find an optimal approximation of the correction function for the given reference data . We will first approach this problem in a supervised manner with human intervention and then explain an improved approach that enables a NN model to learn the target function in an unsupervised manner . As the velocity v is the key quantity for fluids , we target corrections c of this function in the following . However , our approach likewise could be applied to other functions . The correction is applied additively , i.e. , v+c , and should be computed such that this sum matches the reference velocity vR , i.e. , ideally v + c = vR . In practice , we will not aim for an equality but rather match the reference as closely as possible . As the reference is typically best defined in terms of a fine discretization with reduced approximation errors , it might have an altogether different resolution . In this case , care is needed when transferring functions between different discretizations especially when additional constraints such as physical laws of conservation need to be taken into account . We address this problem via an optimization and aim for minimizing the distance between representations on both grids with respect to a suitable metric . For two different dimensionalities ξ , χ ∈ N with ξ < χ , consider two vector spaces H ∈ Rχ and L ∈ Rξ that both conserve volume ( following Eq . 2 ) , i.e. , ∇ · h = 0 for ∀h ∈ H , and ∇ · l = 0 for ∀l ∈ L. Having a finer vector field cH , which contains the necessary information , we aim to find the closest vector field cL ( ∈ L ) to cH ( ∈ H ) . We first describe how to transfer functions between both spaces in general before explaining how to find those that are particularly amenable for learning . Consider an interpolation operator W that introduces new data points within a vector field cL ( ∈ L ) , i.e. , WcL ∈ Rχ . We , then , strive to minimize the distance between WcL and cH such that cL can best represent the information of cH without loosing its volume conserving properties . Thus , we aim for computing cL with argmin cL ||WcL − cH ||2 subject to ∇ · cL = 0 . ( 3 ) This represents a constrained optimization problem with equality constraints , which we can solve via Lagrange multipliers λ as follows : Φ = ||WcL − cH ||2 + ( ∇ · cL ) > λ . ( 4 ) This results in a system of equations as follows : [ W > W −∇ −∇ > 0 ] [ cL λ ] = [ W > cH 0 ] . ( 5 ) Using the Schur complement , we can simplify this system to speed up calculations : ∇ > ( W > W ) −1∇λ = ∇ > ( W > W ) −1W > cH , ( 6 ) cL = ( W > W ) −1 ( W > cH −∇λ ) . ( 7 ) A NN , then , infers the correction ĉL by minimizing the supervised loss , Lsup = ∑ ||ĉL − cL|| , for a pre-computed data set . While this method can efficiently yield divergence-free changes of resolution , i.e. , move a function such as the aforementioned correction from fine to coarse grids , the solutions obtained in this way are not necessarily optimal for deep learning methods as we will evaluate in more detail below . Hence , we first introduce an extension that takes into account the temporal evolution of the correction function .

Summary: The authors aim at improving the accuracy of numerical solvers (e.g. for simulations of partial differential equations) by training a neural network on simulated reference data. The neural network is used to correct the numerical solver. For different tasks they set up an approximation scheme via minimizing a square loss plus a task specific regularization (e.g. volume preservation in the Navier-Stokes equation example). This is then trained in a supervised manner. They also explore an unsupervised version by back-progagating through a differentiable numberical solver.