Learning Time-Aware Assistance Functions for Numerical Fluid Solvers

1 INTRODUCTION . Numerical methods are a central component of many disciplines and widely used for solving a variety of linear and nonlinear problems . One of the long-standing targets is fluid flow , which is renowned for its great diversity and complexity in terms of dynamics . Studies in computational fluid dynamics have focused on numerical simulations for such problems and invested huge efforts in solving spatio-temporal partial differential equations ( PDEs ) such as the Navier-Stokes equations , which represent the well-established physical model for fluids . Traditional methods typically improve accuracy with fine discretizations both in space and time . While the methods and computing power for numerical simulation have seen advances in recent years , there is still a pressing need for better efficiency and accuracy . For most practical applications of computer simulations , we are still far away from fully resolving all necessary scales of nature around us ( Verma et al. , 2018 ; Cummins et al. , 2018 ) . To tackle this problem , we propose a data-driven approach that “ assists ” a given numerical method to improve its accuracy . To this end , we introduce a first learning-based approach that puts special emphasis on the time dimension . We demonstrate two variants to achieve this goal in the context of fluids : a supervised version with an optimization algorithm for acquisition of temporally constrained correction data and an unsupervised version with a differentiable PDE solver that allows us to autonomously takes into account temporal information when training . We compare advantages and disadvantages of both approaches , and our experiments show that , using our trained models , the simulation accuracy of the given solver can be significantly improved . In all cases , our trained models yield improved dynamics , and the learned assistance function lets a coarse simulation reproduce the behavior of the reference data more closely . In particular , we demonstrate the improvements of our approach over ad-hoc learning approaches . 2 RELATED WORK . Data-driven approaches were shown to be highly effective for inferring unknown PDEs and coefficients of corresponding terms ( Brunton et al. , 2016 ) . Their capabilities were investigated for different problems , among others the Navier-Stokes equations ( Rudy et al. , 2017 ; Schaeffer , 2017 ) . Deep learning methods were successfully applied for learning stencils of advection diffusion problems ( Bar-Sinai et al. , 2018 ) , to discover PDE formulations ( Long et al. , 2017 ) , and to analyze families of Poisson equations ( Magill et al. , 2018 ) . While identifying governing equations represents an interesting and challenging task , we instead focus on a general method to improve the solutions of chosen solution spaces . Due to the complexity and nonlinear nature of fluid dynamics , they remain a challenging problem , and accordingly data-driven approaches including deep learning have received a lot of attention to reach an efficient solution for such problems ( Kutz , 2017 ) . Particularly , a turbulence modeling has been a representative objective in this context ( Duraisamy et al. , 2015 ; Maulik & San , 2017 ; Beck et al. , 2018 ; Mohan et al. , 2019 ) . Moreover , both steady ( Guo et al. , 2016 ) and unsteady ( Morton et al. , 2018 ) flows as well as multiphase ( Gibou et al. , 2018 ) flows have been investigated with deep learning based approaches . Additionally , the convolutional neural network ( CNN ) based methods were studied for airfoil flow problems ( Thuerey et al. , 2018 ; Zhang et al. , 2018 ) . For fluid simulation , particularly in the field of computer graphics , data-driven approaches have been considered as an efficient alternative to replace computationally expensive steps of numerical processes ( Ladický et al. , 2015 ; Tompson et al. , 2017 ) . Moreover , deep learning , particularly with generative adversarial models ( Goodfellow , 2016 ) , has been used for efficiently synthesizing details within coarsely resolved simulations ( Chu & Thuerey , 2017 ; Xie et al. , 2018 ) . In addition to scalar transport and smoke flows , liquid droplets have been efficiently tackled by learning stochastic models ( Um et al. , 2018 ) . Due to their capabilities for transforming space-time data into reduced latent spaces , fluid simulations were processed and re-simulated with neural network ( NN ) models ( Kim et al. , 2019 ; Prantl et al. , 2019 ) . Motivated by the success of previous studies regarding datadriven approaches in numerical simulations , we aim for enabling the learning of general correction functions . Our aim differs from methods that synthesize details for low-resolution input data . As such , our method is orthogonal to super-resolution methods and shares similarities with methods to identify discontinuities in finite difference solutions ( Ray & Hesthaven , 2018 ) . However , we investigate the problem for challenging two-dimensional flow problems and propose a new approach for learning the correction bby taking into account temporal dynamics . 3 CORRECTING NUMERICAL SIMULATIONS . Numerical methods yield approximations of a smooth function f ( x ) in a discrete setting with an approximation error of the form O ( hk ) where h is the step size of the discretization . Naturally , higher order methods with larger k are preferable but difficult to arrive at in practice . In our setting , the motions of fluids can be represented by a series of vector fields v ( x ) where bold vectors will denote two- or three-dimensional vector-valued functions in the following . The accuracy of the fluid motion likewise depends on the discretization error for v. In the following , we will focus on Cartesian Eulerian representations , meaning axis aligned grids with square cells . Discretizing functions like v on such a grid yields approximations that scale with the grid spacing h. While small h can yield accurate representations , the computational requirements typically increase superlinearly with the number of unknowns , and as such , all practical methods strive for keeping h as large as possible in order to compute solutions quickly . Due to the nonlinear nature of flow equations and the inherent truncation errors of numerical schemes , varying resolutions often lead to very significant differences in solutions . Such differences are not readily interchangeable across the different resolutions , as especially higher frequency content can be difficult to represent on a coarse grid . In order to improve the accuracy of a numerical method , we propose to learn a correction function that compensates for discretization errors on coarse grids and is representable by a NN . More specifically , we employ a CNN ( Krizhevsky et al. , 2012 ) as the correction should be translation invariant for generalization . We target time-dependent problems for which we infer a per time step correction . This is particularly challenging as approximation errors accumulate over time . In this work , we will focus on finite difference approximations ( Strikwerda , 2004 ) . A discretized representation s of a smooth target function f typically consists of a set of discrete physical quantities . Together , we call these discrete quantities s a state with which our correction approach can be formulated as follows : sn+1 = F ( sn ) + cn ( 1 ) where F denotes a numerical solver for the target function f moving a state sn forward in time to a new state sn+1 . Here , cn denotes the correction function . Note that cn potentially updates only a part of the full state , e.g. , the velocity field v. We target solutions of the Navier-Stokes equations , which for incompressible Newtonian fluids takes the form ∂v ∂t + v · ∇v = −1 ρ ∇p+ ν∇ · ∇v + g subject to ∇ · v = 0 , ( 2 ) where ρ , p , ν , and g denote density , pressure , viscosity , and external forces , respectively . The constraint , ∇ · v = 0 , is particularly important as it restricts motions to the space of divergence-free ( i.e. , volume preserving ) velocities . 3.1 CORRECTION FUNCTION . The natural follow-up question is how to compute and represent the potentially highly nonlinear correction function of Eq . 1 . Thanks to their flexibility and ability to represent nonlinearities , we use fully convolutional networks as our model for the correction function . A fine discretization , optionally with advanced numerical schemes , is employed to compute the reference data . Our goal is , then , to find an optimal approximation of the correction function for the given reference data . We will first approach this problem in a supervised manner with human intervention and then explain an improved approach that enables a NN model to learn the target function in an unsupervised manner . As the velocity v is the key quantity for fluids , we target corrections c of this function in the following . However , our approach likewise could be applied to other functions . The correction is applied additively , i.e. , v+c , and should be computed such that this sum matches the reference velocity vR , i.e. , ideally v + c = vR . In practice , we will not aim for an equality but rather match the reference as closely as possible . As the reference is typically best defined in terms of a fine discretization with reduced approximation errors , it might have an altogether different resolution . In this case , care is needed when transferring functions between different discretizations especially when additional constraints such as physical laws of conservation need to be taken into account . We address this problem via an optimization and aim for minimizing the distance between representations on both grids with respect to a suitable metric . For two different dimensionalities ξ , χ ∈ N with ξ < χ , consider two vector spaces H ∈ Rχ and L ∈ Rξ that both conserve volume ( following Eq . 2 ) , i.e. , ∇ · h = 0 for ∀h ∈ H , and ∇ · l = 0 for ∀l ∈ L. Having a finer vector field cH , which contains the necessary information , we aim to find the closest vector field cL ( ∈ L ) to cH ( ∈ H ) . We first describe how to transfer functions between both spaces in general before explaining how to find those that are particularly amenable for learning . Consider an interpolation operator W that introduces new data points within a vector field cL ( ∈ L ) , i.e. , WcL ∈ Rχ . We , then , strive to minimize the distance between WcL and cH such that cL can best represent the information of cH without loosing its volume conserving properties . Thus , we aim for computing cL with argmin cL ||WcL − cH ||2 subject to ∇ · cL = 0 . ( 3 ) This represents a constrained optimization problem with equality constraints , which we can solve via Lagrange multipliers λ as follows : Φ = ||WcL − cH ||2 + ( ∇ · cL ) > λ . ( 4 ) This results in a system of equations as follows : [ W > W −∇ −∇ > 0 ] [ cL λ ] = [ W > cH 0 ] . ( 5 ) Using the Schur complement , we can simplify this system to speed up calculations : ∇ > ( W > W ) −1∇λ = ∇ > ( W > W ) −1W > cH , ( 6 ) cL = ( W > W ) −1 ( W > cH −∇λ ) . ( 7 ) A NN , then , infers the correction ĉL by minimizing the supervised loss , Lsup = ∑ ||ĉL − cL|| , for a pre-computed data set . While this method can efficiently yield divergence-free changes of resolution , i.e. , move a function such as the aforementioned correction from fine to coarse grids , the solutions obtained in this way are not necessarily optimal for deep learning methods as we will evaluate in more detail below . Hence , we first introduce an extension that takes into account the temporal evolution of the correction function .

Summary: The paper proposes learning NN to correct for inaccuracies in numerical solvers of PDEs, with experimental focus on fluid flow simulation. It lists two approaches: (1) compute correction in high resolution simulation from reference states, convert to low-resolution correction, and train NN to predict low-res correction (optionally with temporal regularization, and (2) directly simulate forward using correction prediction and differentiable PDE solver and optimize to match the given reference states. It shows empirical results on better approximating fluid flow simulation.