Towards Physics-informed Deep Learning for Turbulent Flow Prediction

1 INTRODUCTION . Modeling the dynamics of physical processes that evolve over space and time and vary over a wide range of spatial and temporal scales is a fundamental task in science . Computational fluid dynamics ( CFD ) is at the heart of climate modeling and has direct implications for understanding and predicting climate change . However , the current paradigm in atmospheric CFD is purely physicsdriven : known physical laws encoded in systems of coupled partial differential equations ( PDEs ) are solved over space and time via numerical differentiation and integration schemes . These methods are tremendously computationally-intensive , requiring significant computational resources and expertise . Recently , data-driven methods , including deep learning , have demonstrated great success in the automation , acceleration , and streamlining of highly compute-intensive workflows for science ( Reichstein et al. , 2019 ) . But existing deep learning methods are mainly statistical with little or no underlying physical knowledge incorporated , and are yet to be proven to be successful in capturing and predicting accurately the properties of complex physical systems . Developing deep learning methods that can incorporate physical laws in a systematic manner is a key element in advancing AI for physical sciences ( Steven Brunton , 2019 ) . Towards this goal , we investigate the challenging problem of predicting a turbulent flow , governed by the high-dimensional nonlinear Navier-Stokes equations . Recently , several studies have attempted incorporating knowledge about a physical system into deep learning . For example , Emmanuel de Bezenac ( 2018 ) proposed a warping scheme to predict the sea surface temperature , but only considered the linear advectiondiffusion equation . Xie et al . ( 2018 ) and Jonathan Tompson ( 2017 ) developed deep learning models in the context of fluid flow animation , where physical consistency is less critical . Wu et al . ( 2019 ) and Tom Beucler ( 2019 ) introduced statistical and physical constraints in the loss function to regularize the predictions of the model . However , their studies only focused on spatial modeling without temporal dynamics , besides regularization being ad-hoc and difficult to tune the hyper-parameters . In this work , we propose a hybrid learning paradigm that unifies turbulence modeling and deep representation learning . We develop a novel deep learning model , Turbulent-Flow Net ( TF-Net ) , that en- hances the capability of predicting complex turbulent flows with deep neural networks . TF-Net applies scale separation to model different ranges of scales of the turbulent flow individually . Building upon a promising and popular CFD technique , the RANS-LES coupling approach ( E. Labourasse , 2004 ) , our model replaces a priori spectral filters with trainable convolutional layers . We decompose the turbulent flow into three components , each of which is approximated by a specialized U-net to preserve invariance properties . To the best of our knowledge , this is the first hybrid framework of its kind for predicting turbulent flow . We compare our method with state-of-the-art baselines for forecasting velocity fields up to 60 steps ahead given the history . We observe that TF-Net is capable of generating accurate and physically meaningful predictions that preserve critical quantities of relevance . In summary , our contributions are as follows : 1 . We study the challenging task of turbulent flow prediction as a test bed to investigate incorporating physics knowledge into deep learning in a principled fashion . 2 . We propose a novel hybrid learning framework , TF-Net , that unifies a popular CFD technique , RANS-LES coupling , with custom-designed deep neural networks . 3 . When evaluated on turbulence simulations , TF-Net achieves 11.1 % reduction in prediction RMSE , 30.1 % improvement in the energy spectrum , 21 % turbulence kinetic energy RMSEs and 64.2 % reduction of flow divergence in difference from the target , compared to the best baseline . 2 BACKGROUND IN TURBULENCE MODELING . Most fluid flows in nature are turbulent , but theoretical understanding of solutions to the governing equations , the Navier–Stokes equations , is incomplete . Turbulent fluctuations occur over a wide range of length and time scales with high correlations between these scales . Turbulent flows are characterized by chaotic motions and intermittency , which are difficult to predict . The physical system we investigate is two-dimensional Rayleigh-Bénard convection ( RBC ) , a model for turbulent convection , with a horizontal layer of fluid heated from below so that the lower surface is at a higher temperature than the upper surface . Turbulent convection is a major feature of the dynamics of the oceans , the atmosphere , as well as engineering and industrial processes , which has motivated numerous experimental and theoretical studies for many years . The RBC system serves as an idealized model for turbulent convection that exhibits the full range of dynamics of turbulent convection for sufficiently large temperature gradients . Let w be the vector velocity field of the flow with two components ( u , v ) , velocities along x and y directions , the governing equations for this physical system are : ∇ ·w = 0 Continuity Equation ∂w ∂t + ( w · ∇ ) w = − 1 ρ0 ∇p+ ν∇2w + f Momentum Equation ∂T ∂t + ( w · ∇ ) T = κ∇2T Temperature Equation ( 1 ) where p and T are pressure and temperature respectively , κ is the coefficient of heat conductivity , ρ0 is density at temperature at the beginning , α is the coefficient of thermal expansion , ν is the kinematic viscosity , f the body force that is due to gravity . In this work , we use a particular approach to modeling RBC that uses a Boussinesq approximation , resulting in a divergence-free flow , so∇·w should be zero everywhere ( Chirila , 2018 ) . Figure 1 shows a snapshot in our RBC flow dataset . CFD allows simulating complex turbulent flows , however , the wide range of scales makes it very challenging to accurately resolve all the scales . More precisely , fully resolving a complex turbulent flow numerically , known as direct numerical simulations – DNS , requires a very fine discretization of space-time , which makes the computation prohibitive even with advanced high-performance computing . Hence most CFD methods , like Reynolds-Averaged Navier-Stokes and Large Eddy Simulations ( McDonough , 2007a ; Pierre Sagaut , 2006 ; McDonough , 2007b ) , resort to resolving the large scales whilst modeling the small scales , using various averaging techniques and/or low-pass filtering of the governing equations ( Eqn . 1 ) . However , the unresolved processes and their interactions with the resolved scales are extremely challenging to model . CFD remains computationally expensive despite decades of advancements in turbulence modeling and HPC . Deep learning ( DL ) is poised to accelerate and improve turbulent flow simulations because welltrained DL models can generate realistic instantaneous flow fields with physically accurate spatiotemporal coherence , without solving the complex nonlinear coupled PDEs that govern the system ( Tompson et al. , 2017 ; Maziar Raissi , 2019 ; 2018 ) . However , DL models are hard to train and are often used as `` black boxes '' in physical science as they lack knowledge of the underlying physics and are very hard to interpret . While these DL models may achieve low prediction errors they often lack scientific consistency and do not respect the physics of the systems they model . Therefore , it is critical to infusing known physics and design efficient turbulent flow prediction DL models that are not only accurate but also physically meaningful . 3 TURBULENT-FLOW NET . Inspired by techniques used in CFD to separate scales of this multi-scale system , the global idea behind TF-Net is to decompose the flow into three components of different scales with trainable modules for simulating each component . First , we provide a brief introduction of the CFD techniques which are built on this basic idea . Reynolds-averaged Navier–Stokes ( RANS ) decomposes the turbulent floww into two separable time scales : a time-averaged mean flow w̄ and a fluctuating quantity w′ . The resulting RANS equations contain a closure term , the Reynolds stresses , that require modeling , the classic closure problem of turbulence modeling . While this approach is a good first approximation to solving a turbulent flow , RANS does not account for broadband unsteadiness and intermittency , characteristic of most turbulent flows . Further , closure models for the unresolved scales are often inadequate , making RANS solutions to be less accurate . T here is the moving average window size . w ( x , t ) = w̄ ( x , t ) +w′ ( x , t ) , where w̄ ( x , t ) = 1 T ∫ t t−T G ( s ) w ( x , s ) ds ( 2 ) Large Eddy Simulation ( LES ) is an alternative approach based on low-pass filtering of the Navier-Stokes equations that solves a part of the multi-scale turbulent flow corresponding to the most energetic scales . In LES , the large scales are a spatially filtered variable w̃ , which is usually expressed as a convolution product by the filter kernel G. The kernel G is often taken to be a Gaussian kernel . Ωi is a subdomain of the solution and depends on the filter size ( Sagaut , 2001 ) . w ( x , t ) = w̃ ( x , t ) +w′ ( x , t ) , where w̃ ( x , t ) = ∫ Ωi G ( x|ξ ) w ( ξ , t ) dξ ( 3 ) The key difference between RANS and LES is that RANS is based on time averaging , leading to simpler steady equations , whereas LES is based on a spatial filtering process which is more accurate but also computationally more expensive . Hybrid RANS-LES Coupling combines both RANS and LES approaches in order to be able to take advantage of both methods ( E. Labourasse , 2004 ; Chaoua , 2017 ) . It decomposes the flow variables into three parts : mean flow , resolved fluctuations and unresolved ( subgrid ) fluctuations . RANS-LES coupling applies the spatial filtering operator G1 and the temporal average operator G2 sequentially . We can define w̄ in discrete form with using w∗ as an intermediate term , w∗ ( x , t ) = G1 ( w ) = ∑ ξ G1 ( x|ξ ) w ( ξ , t ) ( 4 ) w̄ ( x , t ) = G2 ( w ∗ ) = 1 T t∑ s=t−T G2 ( s ) w ∗ ( x , s ) ( 5 ) then w̃ can be defined as the difference between w∗ and w̄ : w̃ = w∗ − w̄ , w′ = w −w∗ ( 6 ) Finally we can have the three-level decomposition of the velocity field . w = w̄ + w̃ +w ′ ( 7 ) Figure 3 shows this three-level decomposition in wavenumber space ( E. Labourasse , 2004 ) . k is the wavenumber , the spatial frequency in the Fourier domain . E ( k ) is the energy spectrum describing how much kinetic energy is contained in eddies with wavenumber k. Small k corresponds to large eddies that contain most of the energy . The slope of the spectrum is negative and indicates the transfer of energy from large scales of motion to the small scales . This hybrid approach combines the ease and computational efficiency of RANS with the resolving power of LES to provide a technique that is less expensive and more tractable than pure LES . Turbulent Flow Net We describe TF-Net , a hybrid deep learning framework based on the multilevel spectral decomposition of hybrid RANS-LES Coupling method . We decompose the velocity field into three components of different scales using two scale separation operators , the spatial filter G1 and the temporal filter G2 . In traditional CFD , these filters are usually pre-defined , such as the Gaussian spatial filter . In our model , both filters are trainable neural networks . The spatial filtering process is realized by applying one convolutional layer with a single 5×5 filter to each input image . The temporal filter is implemented as a convolutional layer with a single 1×1 filter applied to every T images . The motivation for this design is to explicitly guide the DL model to learn the non-linear dynamics of both large and small eddies as relevant to the task of spatio-temporal prediction . We design three identical encoders to encode the three scale components separately . We use a shared decoder to learn the interactions among these three components and generate the final prediction . Each encoder and the decoder can be viewed as a U-net without duplicate layers and middle layer in the original architecture ( Olaf Ronneberger , 2015 ) . The encoder consists of four convolutional layers with double the number of feature channels of the previous layer and stride 2 for down-sampling . The decoder consists of one output layer and four deconvolutional layers with summation of the corresponding feature channels from the three encoders and the output of the previous layer as input . Figure 2 shows the overall architecture of our hybrid model TF-Net . To generate multiple timestep forecasts , we perform one-step ahead prediction and roll out autoregressively . Furthermore , since the turbulent flow under investigation has zero divergence ( ∇ ·w should be zero everywhere ) , we include ||∇ ·w||2 as a regularizer to constrain the predictions , leading to Con TF-Net .

Summary: This paper shows a method that combines a convolutional neural network with a multi-scale physical computational fluid dynamics (CFD), in the scope of predicting turbulent flows. The authors proposed a new network, TF-Net, that is based on a multi-scale CFD: a temporal and a spatial filters are learned, prior to 3 separate 'encoder' networks, that are then grouped in a unique 'decoder' part. The method is tested on a synthetic example, showing interesting results and comparing with a large set of baselines.