T OWARDS PHYSICS - INFORMED DEEP LEARNING FOR TURBULENT FLOW PREDICTION
Anonymous authors
Paper under double-blind review
A BSTRACT
While deep learning has shown tremendous success in a wide range of domains, it remains a grand challenge to incorporate physical principles in a systematic manner to the design, training and inference of such models.
In this paper, we aim to predict turbulent ﬂow by learning its highly nonlinear dynamics from spatiotemporal velocity ﬁelds of large-scale ﬂuid ﬂow simulations of relevance to turbulence modeling and climate modeling.
We adopt a hybrid approach by marrying two well-established turbulent ﬂow simulation techniques with deep learning.
Specifically, we introduce trainable spectral ﬁlters in a coupled model of Reynoldsaveraged Navier-Stokes (RANS) and Large Eddy Simulation (LES), followed by a specialized U-net for prediction.
Our approach, which we call Turbulent-Flow
Net ( TF-Net ), is grounded in a principled physics model, yet offers the ﬂexibility of learned representations.
We compare our model, TF-Net , with state-of-theart baselines and observe signiﬁcant reductions in error for predictions 60 frames ahead.
Most importantly, our method predicts physical ﬁelds that obey desirable physical characteristics, such as conservation of mass, whilst faithfully emulating the turbulent kinetic energy ﬁeld and spectrum, which are critical for accurate prediction of turbulent ﬂows.
I NTRODUCTION
Modeling the dynamics of physical processes that evolve over space and time and vary over a wide range of spatial and temporal scales is a fundamental task in science.
Computational ﬂuid dynamics (CFD) is at the heart of climate modeling and has direct implications for understanding and predicting climate change.
However, the current paradigm in atmospheric CFD is purely physicsdriven : known physical laws encoded in systems of coupled partial differential equations (PDEs) are solved over space and time via numerical differentiation and integration schemes.
These methods are tremendously computationally-intensive, requiring signiﬁcant computational resources and expertise.
Recently, data-driven methods, including deep learning, have demonstrated great success in the automation, acceleration, and streamlining of highly compute-intensive workﬂows for science (Reichstein et al., 2019).
But existing deep learning methods are mainly statistical with little or no underlying physical knowledge incorporated, and are yet to be proven to be successful in capturing and predicting accurately the properties of complex physical systems.
Developing deep learning methods that can incorporate physical laws in a systematic manner is a key element in advancing AI for physical sciences (Steven Brunton, 2019).
Towards this goal, we investigate the challenging problem of predicting a turbulent ﬂow, governed by the high-dimensional nonlinear Navier-Stokes equations.
Recently, several studies have attempted incorporating knowledge about a physical system into deep learning.
For example, Emmanuel de Bezenac (2018) proposed a warping scheme to predict the sea surface temperature, but only considered the linear advectiondiffusion equation.
Xie et al.
and Jonathan Tompson (2017) developed deep learning models in the context of ﬂuid ﬂow animation, where physical consistency is less critical.
Wu et al.
and
Tom Beucler (2019) introduced statistical and physical constraints in the loss function to regularize the predictions of the model.
However, their studies only focused on spatial modeling without temporal dynamics, besides regularization being ad-hoc and difﬁcult to tune the hyper-parameters.
In this work, we propose a hybrid learning paradigm that uniﬁes turbulence modeling and deep representation learning.
We develop a novel deep learning model, Turbulent-Flow Net ( TF-Net ), that en- hances the capability of predicting complex turbulent ﬂows with deep neural networks.
TF-Net applies scale separation to model different ranges of scales of the turbulent ﬂow individually.
Building upon a promising and popular CFD technique, the RANS-LES coupling approach (E. Labourasse, 2004), our model replaces a priori spectral ﬁlters with trainable convolutional layers.
We decompose the turbulent ﬂow into three components, each of which is approximated by a specialized U-net to preserve invariance properties.
To the best of our knowledge, this is the ﬁrst hybrid framework of its kind for predicting turbulent ﬂow.
We compare our method with state-of-the-art baselines for forecasting velocity ﬁelds up to 60 steps ahead given the history.
We observe that TF-Net is capable of generating accurate and physically meaningful predictions that preserve critical quantities of relevance.
In summary, our contributions are as follows:
We study the challenging task of turbulent ﬂow prediction as a test bed to investigate incorporating physics knowledge into deep learning in a principled fashion.
We propose a novel hybrid learning framework, TF-Net , that uniﬁes a popular CFD technique, RANS-LES coupling, with custom-designed deep neural networks.
When evaluated on turbulence simulations, TF-Net achieves 11.1% reduction in prediction RMSE, 30.1% improvement in the energy spectrum, 21% turbulence kinetic energy RMSEs and 64.2% reduction of ﬂow divergence in difference from the target, compared to the best baseline.
B ACKGROUND IN T URBULENCE M ODELING
Most ﬂuid ﬂows in nature are turbulent, but theoretical understanding of solutions to the governing equations, the Navier–Stokes equations, is incomplete.
Turbulent ﬂuctuations occur over a wide range of length and time scales with high correlations between these scales.
Turbulent ﬂows are characterized by chaotic motions and intermittency, which are difﬁcult to predict.
(Chirila, 2018).
The spatial resolution is 1792 x 256 pixels.
The physical system we investigate is two-dimensional Rayleigh-Bénard convection (RBC), a model for turbulent convection, with a horizontal layer of ﬂuid heated from below so that the lower surface is at a higher temperature than the upper surface.
Turbulent convection is a major feature of the dynamics of the oceans, the atmosphere, as well as engineering and industrial processes, which has motivated numerous experimental and theoretical studies for many years.
The RBC system serves as an idealized model for turbulent convection that exhibits the full range of dynamics of turbulent convection for sufﬁciently large temperature gradients.
Let w be the vector velocity ﬁeld of the ﬂow with two components ( u, v ) , velocities along x and y directions, the governing equations for this physical system are:
[Equation]
Continuity Equation
[Equation]
Momentum Equation
[Equation]
Temperature
[Equation]
where p and T are pressure and temperature respectively, κ is the coefﬁcient of heat conductivity, ρ 0 is density at temperature at the beginning, α is the coefﬁcient of thermal expansion, ν is the kinematic viscosity, f the body force that is due to gravity.
In this work, we use a particular approach to modeling RBC that uses a Boussinesq approximation, resulting in a divergence-free ﬂow, so ∇· w should be zero everywhere (Chirila, 2018).
Figure 1 shows a snapshot in our RBC ﬂow dataset.
CFD allows simulating complex turbulent ﬂows, however, the wide range of scales makes it very challenging to accurately resolve all the scales.
More precisely, fully resolving a complex turbulent ﬂow numerically, known as direct numerical simulations – DNS, requires a very ﬁne discretization of space-time, which makes the computation prohibitive even with advanced high-performance computing.
Hence most CFD methods, like Reynolds-Averaged Navier-Stokes and Large Eddy
Simulations (McDonough, 2007a; Pierre Sagaut, 2006; McDonough, 2007b), resort to resolving the large scales whilst modeling the small scales, using various averaging techniques and/or low-pass ﬁltering of the governing equations (Eqn. 1).
However, the unresolved processes and their interactions with the resolved scales are extremely challenging to model.
CFD remains computationally expensive despite decades of advancements in turbulence modeling and HPC.
Deep learning (DL) is poised to accelerate and improve turbulent ﬂow simulations because welltrained DL models can generate realistic instantaneous ﬂow ﬁelds with physically accurate spatiotemporal coherence, without solving the complex nonlinear coupled PDEs that govern the system (Tompson et al., 2017; Maziar Raissi, 2019; 2018).
However, DL models are hard to train and are often used as "black boxes" in physical science as they lack knowledge of the underlying physics and are very hard to interpret.
While these DL models may achieve low prediction errors they often lack scientiﬁc consistency and do not respect the physics of the systems they model.
Therefore, it is critical to infusing known physics and design efﬁcient turbulent ﬂow prediction DL models that are not only accurate but also physically meaningful.
T URBULENT -F LOW N ET
Inspired by techniques used in CFD to separate scales of this multi-scale system, the global idea behind TF-Net is to decompose the ﬂow into three components of different scales with trainable modules for simulating each component.
First, we provide a brief introduction of the CFD techniques which are built on this basic idea.
Reynolds-averaged Navier–Stokes (RANS) decomposes the turbulent ﬂow w into two separable time scales: a time-averaged mean ﬂow ¯ w and a ﬂuctuating quantity w (cid:48) .
The resulting RANS equations contain a closure term, the Reynolds stresses, that require modeling, the classic closure problem of turbulence modeling.
While this approach is a good ﬁrst approximation to solving a turbulent ﬂow, RANS does not account for broadband unsteadiness and intermittency, characteristic of most turbulent
Further, closure models for the unresolved scales are often inadequate, making RANS solutions to be less accurate.
T here is the moving average window size.
[Equation]
where ¯ w ( x , t
[Equation]
= 1 T (cid:90)
[Equation]
Large Eddy Simulation (LES) is an alternative approach based on low-pass ﬁltering of the
Navier-Stokes equations that solves a part of the multi-scale turbulent ﬂow corresponding to the most energetic scales.
In LES, the large scales are a spatially ﬁltered variable ˜ w , which is usually expressed as a convolution product by the ﬁlter kernel G .
The kernel G is often taken to be a
Gaussian kernel.
Ω i is a subdomain of the solution and depends on the ﬁlter size (Sagaut, 2001).
[Equation]
The key difference between RANS and LES is that RANS is based on time averaging, leading to simpler steady equations, whereas LES is based on a spatial ﬁltering process which is more accurate but also computationally more expensive.
Hybrid RANS-LES Coupling combines both RANS and LES approaches in order to be able to take advantage of both methods (E. Labourasse, 2004; Chaoua, 2017).
It decomposes the ﬂow variables into three parts: mean ﬂow, resolved ﬂuctuations and unresolved (subgrid) ﬂuctuations.
RANS-LES coupling applies the spatial ﬁltering operator G 1 and the temporal average operator G 2 sequentially.
We can deﬁne ¯ w in discrete form with using w ∗ as an intermediate term,
[Equation]
then ˜ w can be deﬁned as the difference between w ∗ and ¯ w :
[Equation]
Finally we can have the three-level decomposition of the velocity ﬁeld.
[Equation]
Figure 3 shows this three-level decomposition in wavenumber space (E. Labourasse, 2004).
k is the wavenumber, the spatial frequency in the Fourier domain.
E ( k ) is the energy spectrum describing how much kinetic energy is contained in eddies with wavenumber k .
Small k corresponds to large eddies that contain most of the energy.
The slope of the spectrum is negative and indicates the transfer of energy from large scales of motion to the small scales.
This hybrid approach combines the ease and computational efﬁciency of RANS with the resolving power of LES to provide a technique that is less expensive and more tractable than pure LES.
Turbulent Flow Net We describe TF-Net , a hybrid deep learning framework based on the multilevel spectral decomposition of hybrid RANS-LES Coupling method.
We decompose the velocity ﬁeld into three components of different scales using two scale separation operators, the spatial ﬁlter
G 1 and the temporal ﬁlter G 2 .
In traditional CFD, these ﬁlters are usually pre-deﬁned, such as the
Gaussian spatial ﬁlter.
In our model, both ﬁlters are trainable neural networks.
The spatial ﬁltering process is realized by applying one convolutional layer with a single 5 × 5 ﬁlter to each input image.
The temporal ﬁlter is implemented as a convolutional layer with a single 1 × 1 ﬁlter applied to every
T images.
The motivation for this design is to explicitly guide the DL model to learn the non-linear dynamics of both large and small eddies as relevant to the task of spatio-temporal prediction.
We design three identical encoders to encode the three scale components separately.
We use a shared decoder to learn the interactions among these three components and generate the ﬁnal prediction.
Each encoder and the decoder can be viewed as a U-net without duplicate layers and middle layer in the original architecture (Olaf Ronneberger, 2015).
The encoder consists of four convolutional layers with double the number of feature channels of the previous layer and stride 2 for down-sampling.
The decoder consists of one output layer and four deconvolutional layers with summation of the corresponding feature channels from the three encoders and the output of the previous layer as input.
Figure 2 shows the overall architecture of our hybrid model TF-Net .
To generate multiple timestep forecasts, we perform one-step ahead prediction and roll out autoregressively.
Furthermore, since the turbulent ﬂow under investigation has zero divergence ( ∇ · w should be zero everywhere), we include ||∇ · w || 2 as a regularizer to constrain the predictions, leading to Con TF-Net .
R ELATED WORK
Turbulence Modeling Recently, machine learning models, especially DL models have been used to accelerate and improve the simulation of turbulent ﬂows.
For example, Ling et al.
; Fang et al.
(2018) studied tensor invariant neural networks to learn the Reynolds stress tensor while preserving Galilean invariance, but Galilean invariance only applies to ﬂows without external forces.
In our case, RBC ﬂow has gravity as an external force.
Most recently, Kim & Lee (2019) studied unsupervised generative modeling of turbulent ﬂows but the model is not able to make real time future predictions given the historic data.
Raissi et al.
(2017) applied a Galerkin ﬁnite element method with deep neural networks to solve PDEs automatically, what they call “Physics-informed deep learning”.
Though these methods have shown the ability of deep learning in solving PDEs directly and deriving generalizable solutions, the key limitation of these approaches is that they require explicitly inputs of boundary conditions during inference, which are generally not available in realtime.
Arvind Mohan (2019) proposed a purely data-driven DL model for turbulence, compressed convolutional LSTM, but the model lacks physical constraints and interpretability.
Wu et al.
and
Tom Beucler (2019) introduced statistical and physical constraints in the loss function to regularize the predictions of the model.
However, their studies only focused on spatial modeling without temporal dynamics, besides regularization being ad-hoc and difﬁcult to tune the hyper-parameters.
Fluid Animation
In parallel, the computer graphics community has also investigated using deep learning to speed up numerical simulations for generating realistic animations of ﬂuids such as water and smoke.
For example, Tompson et al.
(2017) used an incompressible Euler’s equation with a customized Convolutional Neural Network (CNN) to predict velocity update within a ﬁnite difference method solver.
Chu & Thuerey (2017) propose double CNN networks to synthesize high-resolution ﬂow simulation based on reusable space-time regions.
Xie et al.
and Jonathan Tompson (2017) developed deep learning models in the context of ﬂuid ﬂow animation, where physical consistency is less critical.
Steffen Wiewel (2019) proposed a method for the data-driven inference of temporal evolutions of physical functions with deep learning.
However, ﬂuid animation emphases on the realism of the simulation rather than the physical consistency of the predictions or physics metrics and diagnostics of relevance to scientists.
Video Prediction Our work is also related to future video prediction.
Conditioning on the observed frames, video prediction models are trained to predict future frames, e.g., Mathieu et al.
(2015); Finn et al.
; Xue et al.
; Villegas et al.
; Chelsea Finn (2016).
Many of these models are trained on natural videos with complex noisy data from unknown physical processes.
Therefore, it is difﬁcult to explicitly incorporate physical principles into the model.
The turbulent ﬂow problem studied in this work is substantially different from natural video prediction because it does not attempt to predict object or camera motions.
Instead, our approach aims to emulate numerical simulations given noiseless observations from known governing equations.
Hence, some of these techniques are perhaps under-suited for our application.
E XPERIMENTS 5. D ATASET
The dataset for our experiments comes from two dimensional turbulent ﬂow simulated using the
Lattice Boltzmann Method (Chirila, 2018).
We use only the velocity vector ﬁelds, where the spatial resolution of each image is 1792 x 256.
Each image has two channels, one is the turbulent ﬂow velocity along x direction and the other one is the velocity along y direction.
The physics parameters relevant to this numerical simulation are: Prandtl number = 0 .
71 , Rayleigh number = 2 .
5 × 10 8 and the maximum Mach number = 0.1.
We use 1500 images (snapshots in time) for our experiments.
The task is to predict the spatiotemporal velocity ﬁelds up to 60 steps ahead given 10 initial frames.
We divided each 1792 by 256 image into 7 square sub-regions of size 256 x 256, then downsample them into 64 x 64 pixels sized images.
We use a sliding window approach to generate 9,870 samples of sequences of velocity ﬁelds: 6,000 training samples, 1,700 validation samples and 2,170 test samples.
The DL model is trained using back-propagation through prediction errors accumulated over multiple steps.
We use a validation set for hyper-parameters tuning based on the average error of predictions up to six steps ahead.
The hyper-parameters tuning range can be found in Table 2 in the appendix.
All results are averaged over three runs.
5. B ASELINE
We compare our model with a series of state-of-the-art baselines for turbulent ﬂow prediction.
• ResNet (Kaiming He, 2015): a 34-layer Residual Network by replacing the ﬁnal dense layer with a convolutional layer with two output channels.
• ConvLSTM (Xingjian Shi, 2015): a 3-layer Convolutional LSTM model used for spatiotemporal precipitation nowcasting.
• U-Net
Olaf Ronneberger, 2015): Convolutional neural networks originally developed for image segmentation, also used for video prediction.
• GAN : U-net trained with a discriminator like the Generative Neural Networks.
• SST (Emmanuel de Bezenac, 2018): hybrid deep learning model using warping scheme for linear energy equation to predict sea surface temperature, which is also applicable to the linearized momentum equation that governs the velocity ﬁelds.
• DHPM (Raissi, 2018):
Deep Hidden Physics Model is to directly approximate the solution of partial differential equations with fully connected networks using space and time as inputs.
The model is trained twice on the training set and the test set with boundary conditions.
Here ResNet , ConvLSTM , U-net and GAN are pure data-driven spatiotemporal deep learning models for video predictions.
SST and DHPM are hybrid techniques that aim to incorporate prior physical knowledge into deep learning for ﬂuid simulation.
5. E VALUATION M ETRICS
Even though Root Mean Square Error (RMSE) is a widely accepted metric for quantifying the differences between model predictions and the ground truth, it is still insufﬁcient to apply the predicted turbulent ﬂows with good RMSE to scientiﬁc ﬁelds.
We need to check whether the predictions are physically meaningful and preserve desired physical quantities, such as Turbulence Kinetic Energy,
Divergence and Energy Spectrum.
Therefore, we include a set of additional metrics for evaluation.
Root Mean Square Error
We calculate the RMSE of all predicted values from the ground truth for each pixel, (cid:113)(cid:80) Ni
=1 ( ˆ w i − w i ) 2 /N .
Divergence Since we investigate incompressible turbulent ﬂows in this work, which means the divergence, ∇ · w , at each pixel should be zero, we use the average of absolute divergence over all pixels at each prediction step as an additional evaluation metric.
Turbulence Kinetic Energy
In ﬂuid dynamics, turbulence kinetic energy is the mean kinetic energy per unit mass associated with eddies in turbulent ﬂow.
Physically, the turbulence kinetic energy is characterised by measured root mean square velocity ﬂuctuations, (( u (cid:48) )
+( v (cid:48) ) 2 ) / 2 , where ( u (cid:48) )
2 = 1 T (cid:80) Tt =
( u ( t ) − ¯ u ) 2 and t is the time step.
We calculate the turbulence kinetic energy for each predicted sample of 60 velocity ﬁelds.
Energy Spectrum
The energy spectrum of turbulence, E ( k ) , is related to the mean turbulence kinetic energy as (cid:82) ∞
E ( k ) dk
=
(( u (cid:48) )
+ ( v (cid:48) ) 2 ) / 2 .
k is the wavenumber, the spatial frequency in 2D Fourier domain.
We calculate the Energy Spectrum on the Fourier transformation of the
Turbulence Kinetic Energy ﬁelds.
The large eddies have low wavenumbers and the small eddies correspond to high wavenumbers.
The spectrum tells how much kinetic energy is contained in eddies with wavenumber k .
R ESULTS
Figure 4 shows the growth of RMSE with prediction horizon up to 60 time steps ahead.
TF-Net consistently outperforms all baselines, and constraining it with divergence free regularizer can further improve the performance.
We also found DHPM is able to overﬁt the training set but performs poorly when tested outside of the training domain.
Neither Dropout nor regularization techniques can improve its performance.
Also, the warping scheme of the Emmanuel de Bezenac (2018) relies on the simpliﬁed linear assumption, which was too limiting for our non-linear problem.
Figure 5 shows the averages of absolute divergence over all pixels at each prediction step.
TF-Net has lower divergence than other models even without additional divergence free constraint for varying prediction step.
It is worth mentioning that there is a subtle trade-off between
RMSE and divergence.
Even though explicitly constraining model with the divergence-free regularizer can reduce the divergence of the model predictions, it also has the side effect of smoothing out the small scale eddies, which results in a larger RMSE.
Figure 6 displays the turbulence kinetic energy ﬁelds of all models’ predictions at the leftmost square ﬁeld in the original rectangular ﬁeld.
Figure 7 shows the energy spectrum of our model and two best baseline at the leftmost square sub-ﬁeld.
We also convert square predicted images back to the big rectangular ones and calculate the Energy Spectrum on the entire domain, which can be found in Figure 10 in the appendix.
While the turbulence kinetic energy of TF-Net , U-net and ResNet appear to be similar in Figure 6, however, from the energy spectrum in Figure 7 and Figure 10, we can see that TF-Net predictions are in fact much closer to the target.
Extra divergence free constraint does not affect the energy spectrum of predictions.
Thus, unlike other models, TF-Net is able to generate predictions that are physically consistent with the ground truth.
Figure 8 shows the ground truth and the predicted u velocity ﬁelds from all models from time step 0 to 60 .
We also provide videos of predictions by TF-Net and several best baselines in https://www.youtube.com/watch?v=sLuVGIuEE9A and https://www.youtube.
com/watch?v=VMeYHID5LL8 , respectively.
We see that the predictions by our TF-Net model are the closest to the target based on the shape and the frequency of the motions.
GAN is able to generate ﬂows with ﬁne-grained details but physics are not captured in a correct way, which also shows the beneﬁt of developing hybrid models with embedded physics knowledge.
U-net is the best performing data-driven video prediction baseline.
N. Thuerey (2019) also found the U-net architecture is quite effective in modeling dynamics ﬂows.
We also performed an additional ablation study of TF-net to understand each component of TFnet investigate whether the TF-net has actually learned the ﬂow with different scales.
The video, https://www.youtube.com/watch?v=ysdrMUfdhe0 , includes the predictions of TFnet, and the outputs of each small U-net while the other two encoders are zeroed out.
We can see that the outputs of each small u-net are the ﬂow with different scales.
During Inference, we applied the trained TF-net to the entire input domain instead of square sub-regions.
We observed that the boundaries between square sub-regions in the previous videos have disappeared.
We also did the same experiments on an additional dataset (Rayleigh number = 10 5 ).
TF-Net still consistently outperforms the best two baselines, U-net and ResNet , based on all four evaluation metrics.
The results are shown in in Figure 12 in the appendix.
D ISCUSSION AND F UTURE WORK
We have presented a novel hybrid deep learning model, TF-Net , that uniﬁes representation learning and turbulence simulation techniques.
TF-Net exploits the multi-scale behavior of turbulent ﬂows to design trainable scale-separation operators to model different ranges of scales individually.
We provide exhaustive comparisons of TF-Net and baselines and observe signiﬁcant improvement in both the prediction error and desired physical quantiﬁes, including divergence, turbulence kinetic energy and energy spectrum.
We argue that different evaluation metrics are necessary to evaluate a DL model’s prediction performance for physical systems that include both accuracy and physical consistency.
A key contribution of this work is the skillful combination of state-of-the-art turbulent ﬂow simulation paradigms with deep learning.
Future work includes extending these techniques to very high-resolution predictions, 3D turbulent ﬂows and incorporating additional physical variables to improve the accuracy and faithfulness of physically-informed deep learning models.
A A PPENDIX A. A DDITIONAL R ESULTS
Table 1 displays the number of parameters, the best number of input frames, the best number of accumulated errors for backpropogation and training time for one epoch on 8 v-100 GPUs for each model.
We can conclude that our model has signiﬁcantly smaller number of parameters than most baselines yet achieves the best performance.
About 25 historic images are enough for deep learning models to generate reasonable predictions, and ConvLSTM require large memory and training time, especially when the number of historic input frames is large.
Additionally, Table 2 displays the hyper-parameters tuning range of models.
[Table]
epoch [Table] [Table] Figure 9 shows the ground truth and the predicted v velocity ﬁelds over 60 time steps.
Similar to the u velocity predictions, we observe that the predictions from TF-Net are the closest to the target.
U-net and GAN generate smooth predictions and miss the details of small scale motion.
There is still room for improvement in long-term prediction for all the models.
We converted square predicted images back to the big rectangular ones and calculated the Energy
Spectrum on the entire domain, as shown in Figure 10.
we can see that TF-Net predictions are in fact much closer to the target on small wavenumbers and more stable on large wavenumbers.
We visualized the learned ﬁlters in Figure 11.
We only found two types of spatial and temporal ﬁlters from all trained TF-Net models, with and without the divergence regularizer.
The meaning of these learned ﬁlters are yet to be explored.
We also performed the same experiments on an additional dataset (Rayleigh number = 10 5 ).
TF-Net still consistently outperforms the best two baselines, U-net and ResNet , based on all four evaluation metrics.
The results are shown in Figure 12.
A. I MPLEMENTATION D ETAILS
We adapt SST Emmanuel de Bezenac (2018) to model non-linear turbulent ﬂow.
SST successfully infused a deep learning model into the solution of the linear energy equation to predict sea surface temperature.
If we make the assumption that the advection term ( w ·∇ ) u in the momentum equation is a linear term ( c · ∇ ) u , where c is unknown, then we can use two separate models to predict u and v , and the inputs of both parts are the same stacked u and v from previous time steps.
For DHPM (Raissi, 2018), we approximate both the velocity ﬁeld two 6-layer neural networks with 512 neurons per hidden layer and use and a 4-layer neural networks with 512 neurons per hidden layer to represent pressure p and an non-homogeneous term f that encapsulates the inﬂuence of temperature and viscosity.
During the training, we make sure the outputs of these neural networks satisfy the continuity and momentum equations.
It is worth mentioning that the DHPM model is supposed to be trained twice, ﬁrst on the training set then on the initial and boundary conditions of the test set.
This model can be formulated as below.
[Equation]
Where ˆ w = NN ( x, y, t ) , and f = NN ( x, y, t, ˆ u, ˆ v, ˆ u x , ˆ v x , ˆ u y , ˆ v y , ˆ u xx , ˆ v xx , ˆ u yy , ˆ v yy )
[Figure]