# Machine learning-based vorticity evolution and super-resolution of homogeneous isotropic turbulence using wavelet projection

Tomoki Asaka, Katsunori Yoshimatsu, Kai Schneider

► To cite this version:

Tomoki Asaka, Katsunori Yoshimatsu, Kai Schneider. Machine learning-based vorticity evolution and super-resolution of homogeneous isotropic turbulence using wavelet projection. Physics of Fluids, 2024, 36 (2), 10.1063/5.0185165. [hal-04510300](https://hal.science/hal-04510300v1)

HAL Id: hal-04510300

https://hal.science/hal-04510300v1

Submitted on 20 Mar 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Machine learning-based vorticity evolution and superresolution of homogeneous isotropic turbulence using wavelet projection<sup>a)</sup>

Tomoki Asaka,<sup>1, b)</sup> Katsunori Yoshimatsu<sup>*, 1, c)</sup> and Kai Schneider<sup>1, d)</sup>

(Dated: 15 March 2024)

A wavelet-based machine learning method is proposed for predicting the time evolution of homogeneous isotropic turbulence where vortex tubes are preserved. Three-dimensional convolutional neural networks and long short-term memory are trained with a time series of direct numerical simulation (DNS) data of homogeneous isotropic turbulence at the Taylor microscale Reynolds number 92. The predicted results are assessed by using flow visualization of vorticity and statistics, e.g., probability density functions of vorticity and enstrophy spectra. It is found that the predicted results are in good agreement with DNS results. The small-scale flow topology considering the second and third invariant of the velocity gradient tensor likewise shows an approximate match. Furthermore, we apply the pre-trained neural networks to coarse-grained vorticity data using superresolution. It is shown that the superresolved flow field well agrees with the reference DNS field and thus small-scale information and vortex tubes are well regenerated.
# I. INTRODUCTION

Self-organization in small-scale turbulence is ubiquitous for instance in the form of coherent vortex tubes. The vortices defined as regions of intense vorticity magnitude are characteristic at scales in the dissipation range (e.g., Refs. 1 and 2). They are intermittently distributed in physical space. Moreover, they play key roles for the dynamics of e.g., inertial particle clustering,³ mixing in combustion,⁴ and extreme acceleration of fluid particles.⁵ The representative length scale and time scale are respectively the Kolmogorov length scale $\eta$ and the Kolmogorov time scale $\tau_\eta$. Jiménez et al.¹ found that the typical diameter of the tubes is about $10\eta$. In turbulence modeling, such as large-eddy simulation, the influence of the vortices is statistically modeled without resolving the scales in the dissipation range.

Machine learning in fluid dynamics is an active rapidly evolving and promising field. For reviews we refer to Refs. 6–8. The increasing power and capabilities of machine learning approaches can provide benefit to in particular computational fluid dynamics. Computationally expensive direct numerical simulation (DNS) computations may thus be reduced or even avoided in the near future by training neural networks with available turbulent flow data.⁹,¹⁰ The application of machine learning covers, e.g., extraction of flow features, turbulence modeling, superresolution (SR) of unresolved flows, and time-evolution of flows. Among various applications, we focus in the following on SR and predicting the time evolution of flows.

SR is not limited to improve the resolution of images.¹¹ Dong et al.¹² proposed a deep-learning SR method, by using convolutional neural networks (CNN) which learn the local area of flow structure via convolution filters. SR using two-dimensional (2D) CNN has been applied to turbulence; 2D freely-decaying homogeneous turbulence,¹³ urban turbulence,¹⁴ three-dimensional (3D) forced homogeneous isotropic turbulence (e.g., Refs. 15 and 16), and 3D turbulent channel flows (e.g., Refs. 15 and 16). Liu et al.¹⁵ trained velocity fields and showed that the SR using a time series of the coarse-grained data as an input is able to reconstruct turbulent statistics better than the SR using a single time input data. Generative adversarial networks (GAN) were introduced by Ledig et al.¹⁷ The learning by GAN progresses such that the probability density function (PDF) of the generated data is well superimposed on that of the correct data, and therefore GAN can well reconstruct the statistics of the correct data. SR based on an unsupervised learning model using a cycle-consistent GAN (CycleGAN)¹⁸ has been proposed in Kim et al.¹⁶ and applied in the context of homo-
geneous isotropic turbulence and turbulent channel flow. They trained 2D slices of velocity
fields and showed that SR using the CycleGAN well preserves statistics of velocity and vor-
ticity. Güemes et al. ¹⁹ use a 2D GAN for SR of turbulent velocity fields in channel flows, and
proposed a downsampling factor normalized by the wall-unit quantities in the estimate of SR
of channel turbulent flows. Yousif et al.²⁰ proposed a GAN-based model for reconstructing
3D turbulent velocity fields from velocity data in 2D planes. Asaka et al.²¹ developed an
SR method using wavelets and 3D CNN. They showed that SR of the coarse-grained vor-
ticity field reproduces the vortex tubes much better than that of the coarse-grained velocity
for homogeneous isotropic turbulence. Applying wavelets to the flow fields yields a sparse
multiresolution representation and thus well reduces the number of degrees of freedom of
turbulence at all scales, in particular at the scales in the dissipation range. The wavelet
transform well catches the information of position and scale of the fields and thus reflects
spatial locality and neighboring relationships.

For a review of machine-learning-based SR reconstruction for vortical flows, we refer to Ref. 22. Recently, Refs. 23–25 have developed SR reconstruction from data observed at sparsely distributed positions in physical space using computational and experimental fluid dynamic data. Yousif et al.²⁶ combined a transformer²⁷ with an SR GAN²⁸ for predicting velocity fields of a spatially developing turbulent boundary layer.

As expected the quality of SR becomes worse, when the input data get coarser. We
however anticipate that training of time evolution of the vortex tubes can be a key of
improvement for SR of the vortex tubes. Using DNS of homogeneous isotropic turbulence,
Yoshida et al.²⁹ showed that the time-evolution of the DNS data of small-scale eddies at
scales kη0.2 are perfectly regenerated from the DNS data of larger-scale eddies at scales
kη0.2 after some transient time, if the latter are assimilated at each time step. Here, k is
the modulus of the wavenumber.

Hasegawa et al. ³⁰,³¹ developed a method for predicting flow time evolution, using reduced
order modeling together with 2D CNN, long short-term memory (LSTM) and autoencoder
for 2D flows past a cylinder. LSTM is a neural network which learns stored information over
extended time intervals by recurrent back propagation. ³² In autoencoders, the encoder re-
duces the data size, while the decoder reconstructs the data. ³³ The method was extended to
3D turbulent channel flows. ³⁴ The use of convolutional autoencoders, 3D CNN with LSTM
was proposed by Mohan et al. ³⁵ for different turbulent flows including 3D homogeneous
isotropic turbulence. Shanker et al.³⁶ developed a method for predicting time evolution of 3D homogeneous isotropic turbulent flows, utilizing continuous neural ordinary differential equations, which was proposed by Chen et al.³⁷ instead of LSTM. Guastoni et al.³⁸ developed proper orthogonal decomposition (POD) based machine learning for 3D turbulent channel flow and well predicted the coefficients of POD modes near the wall region. Lucor et al.³⁹ proposed physics-informed neural networks for modeling turbulent convection. These studies train representative quantities at the scales in the energy containing range, such as velocity and temperature fluctuation. The degree of freedoms in small scales are much reduced in the learning. Mohan et al.⁴⁰ proposed a deep-learning model of large and inertial scale dynamics in turbulence, by using wavelet thresholding instead of the autoencoder. Peng et al.⁴¹ introduced a linear attention based model, which is coupled with Fourier neural operators,⁴² for 3D homogeneous isotropic turbulence. The model is trained with vorticity fields at a relatively low Taylor micro-scale Reynolds number of 30, and showed reasonable agreement between their predicted results and the correct data for several large-eddy turnover times. This method is further developed for turbulence modeling in Li et al.⁴³

In this paper, we propose a machine learning method using orthogonal wavelets to predict time evolution of vorticity fields in 3D homogeneous isotropic turbulence, which is one of the most canonical turbulent flows, while preserving vortex tubes. To this end we combine 3D CNN and LSTM techniques in a concise way to design our machine learning approach, and we train coarse-grained vorticity fields. We here use linear wavelet projection, not wavelet thresholding, such that we can track the time-evolution of turbulence in wavelet space without the use of an adaptive wavelet basis for data compression. Orthogonal wavelets are suitable for representing turbulent flow fields, which are multi-scale and intermittent. Coherent vortex tubes can be thus efficiently retained in wavelet space.⁴⁴,⁴⁵ The training and test data are obtained by DNS of homogeneous isotropic turbulence. The wavenumber where its enstrophy spectrum hits the maximum is sufficiently larger than the maximum wavenumber in the large-scale external forcing range. We assess the proposed wavelet-based method of flow prediction by using, e.g., visualization, PDFs of vorticity, and enstrophy spectra. Furthermore, we apply the pre-trained machine learning model to SR in order to predict small-scale vorticity from a coarse-grained vorticity at a given time instant, and assess the results.

The remainder of the manuscript is organized as follows. In Sec. II, we describe DNS
of homogeneous isotropic turbulence. We summarize orthogonal wavelet representation in Sec. III. In Sec. IV, we show a machine learning method for time-evolution of vorticity in homogeneous isotropic turbulence. The method is based on wavelet projection the vorticity. The prediction by the machine learning is then assessed in Sec. V. An application of the pre-trained model to SR is given in Sec. VI. Finally, conclusions are drawn in Sec. VII.

## II. DIRECT NUMERICAL SIMULATION

For our machine learning, we use the data of 3D incompressible homogeneous isotropic turbulence obtained by DNS in a periodic domain $\Omega = [0, 2\pi]^3$. The turbulent dynamics is governed by the Navier-Stokes equations

$$ \frac{\partial \mathbf{u}}{\partial t} + (\mathbf{u} \cdot \nabla)\mathbf{u} = -\frac{1}{\rho}\nabla p + \nu\nabla^2\mathbf{u} + \mathbf{f}, \quad (1) $$

where the velocity field is divergence free, $\nabla \cdot \mathbf{u} = 0$. Here, $\mathbf{u}(\mathbf{x}, t)$ denotes velocity, $p(\mathbf{x}, t)$ pressure, $\mathbf{f}(\mathbf{x}, t)$ the external forcing, $\rho$ the constant density, $\nu$ the constant kinematic viscosity, $t$ time, $\mathbf{x} = (x, y, z)$, and $\nabla = (\partial/\partial x, \partial/\partial y, \partial/\partial z)$. To simplify notation, the arguments $\mathbf{x}$ and $t$ are omitted when useful. The spatial mean velocity of $\mathbf{u}$ is set to zero.

DNS has been carried out, using a Fourier spectral method and a fourth-order Runge-Kutta scheme for time integration. Aliasing errors are removed by a phase shift method and a spherical cutoff filter. Only Fourier modes satisfying $k < \sqrt{2}n_g/3$ are retained, where $k = |\mathbf{k}|$, $\mathbf{k}$ is the wave vector, and $n_g$ denotes the number of grid points in each Cartesian direction. The kinematic viscosity is set to $\nu = 2.3 \times 10^{-3}$. The initial field of the DNS at $t = 0$ is a divergence-free random velocity field whose energy spectrum $\propto k^2 \exp(-k^2/4)$, and the spatial average of energy per unit mass, denoted by $\mathcal{E}$, is set to 0.5: $\mathcal{E} = (1/2)\langle|\mathbf{u}|^2\rangle$, where $\langle\cdot\rangle = (2\pi)^{-3} \int_{\Omega} dx$. For the forcing $\mathbf{f}$, negative viscosity¹ was used only for $k < 2.5$ such that the spatial average of energy per unit mass, denoted by $\mathcal{E}$, remains constant and equals 0.5. The number of grid points $n_g^3$ is 128³, and the time increment is $2.0 \times 10^{-3}$. The data size and the resulting Reynolds number of the DNS are limited due to the memory requirement imposed by our machine learning model (see Sec. IV).

Figure 1 shows the time development of the enstrophy $\mathcal{Z}$, defined as $\mathcal{Z} = (1/2)\langle|\omega|^2\rangle$, where $\omega$ is vorticity. We can see that $\mathcal{Z}$ becomes statistically quasi-stationary for $t>5$. Time-averaged statistics of the DNS are summarized in Table I. The average is obtained by using
<figure><img src="image_2.png" /><figcaption>FIG. 1. Time development of the entrophy $\mathcal{Z}$ of the DNS.</figcaption></figure>

<table><caption>TABLE I. Time-averaged statistics of the 250 snapshots of the DNS data at $t = 10 + 0.08n$ ($n = 0, \dots, 249$). The interval 0.08 is about $0.5\tau_{\eta}$ and $0.17T$. Here, $T$ is a large-eddy turnover time defined by $L/u'$, and $\tau_{\eta}$ is the Kolmogorov time scale defined by $(\nu/\bar{\epsilon})^{1/2}$.</caption><thead><tr><th>ε̄</th><th>L</th><th>η</th><th>T</th><th>τ<sub>η</sub></th><th>R<sub>λ</sub></th></tr></thead><tbody><tr><td>8.57 × 10<sup>-2</sup></td><td>1.21</td><td>1.94 × 10<sup>-2</sup></td><td>0.476</td><td>0.164</td><td>91.9</td></tr></tbody></table>

250 snapshots at every 0.08 in the interval $10.00 \le t \le 29.92$. The time averaged energy dissipation rate per unit density, denoted by $\bar{\epsilon}$, is the time average of $2\nu\mathcal{Z}$. The integral length scale $L$ is defined as $L = \pi/(2u'^2) \int_0^{k_{max}} (1/k)\bar{E}(k)dk$, and $u' = \sqrt{2\mathcal{E}/3}$. Here, $\bar{E}(k)$ is the time-averaged energy spectrum in which the energy spectrum $E(k)$ at each time instant is obtained by $E(k) = (1/2) \sum'_k |\hat{u}(q)|^2$, where $\hat{u}(k)$ is the Fourier transform of $u(x)$, and $\sum'_k$ denotes the summation over the spherical shell, $k-1/2 \le |q| < k+1/2$. The Kolmogorov microscale $\eta$ is defined as $\eta = (\nu^3/\bar{\epsilon})^{1/4}$. Thus, $k_{max}\eta \approx 1.16$, where $k_{max}$ is the maximum wavenumber retained by the DNS. The Taylor microscale Reynolds number $R_\lambda$ is defined by $u'\lambda/\nu$, where $\lambda = (15\nu u'^2/\bar{\epsilon})^{1/2}$.
### III. WAVELET REPRESENTATION

We introduce the wavelet decomposition of a 3D vector field and represent each component, $v(\boldsymbol{x}) \in L^2(\Omega)$, as an orthogonal wavelet series, where $\Omega = [0, 2\pi]^3$. The field is sampled on $2^J$ grid points in each Cartesian direction, where $J$ corresponds to the number of octaves in each space direction in the domain (e.g., $J = 7$ for 128 grid points). The wavelet functions $\psi_{\mu,j,i}(\boldsymbol{x})$ yield an orthogonal basis at scale $2^{-j}$, where $\mu(= 1, \dots, 7)$ denotes directions and $\boldsymbol{i} = (i_x, i_y, i_z)$ position. The scaling function at scale $2^{-j}$ is denoted by $\phi_{j,i}(\boldsymbol{x})$. The fast wavelet transform (FWT) is used to compute the wavelet coefficients from the field and the inverse fast wavelet transform (IFWT) to reconstruct the field from the wavelet coefficients, which in some cases, e.g. for coarse-graining, are filtered. The computational complexity of FWT and IFWT is $O(2^{3J})$. Similar to previous work by Refs. 44 and 45, we use Coiflet 12 wavelets which have filter length 12, four vanishing moments and compact support. We use here PyWavelets$^{46}$ in which the Coiflet 12 wavelets are denoted by 'coif2'.

The field $v(\boldsymbol{x})$ can be then decomposed into an orthogonal wavelet series applying either a periodization or a folding technique,$^{47}$ the latter in the case of subcubes:

$$v_j(\boldsymbol{x}) = v_{j-1}(\boldsymbol{x}) + w_{j-1}(\boldsymbol{x}), \qquad (2)$$

where

$$v_j(\boldsymbol{x}) = \sum_{i_1, i_2, i_3=0}^{2^{j-1}-1} \langle v, \phi_{j,i} \rangle \phi_{j,i}(\boldsymbol{x}), \qquad (3)$$

and

$$w_j(\boldsymbol{x}) = \sum_{\mu=1}^{7} \sum_{i_1, i_2, i_3=0}^{2^{j-1}-1} \langle v, \psi_{\mu,j,i} \rangle \psi_{\mu,j,i}(\boldsymbol{x}), \qquad (4)$$

where $\langle \cdot, \cdot \rangle$ denotes the $L^2$-inner product, defined as $\langle g_1, g_2 \rangle = (2\pi)^{-3} \int_{\Omega} g_1(\boldsymbol{x}) g_2(\boldsymbol{x}) d\boldsymbol{x}$. Note that $v_J = v$, $v_0 = \langle v \rangle$ and $j = 1, \dots, J$. As the wavelets have vanishing moments, including their mean value, we have $\langle w_j \rangle = 0$. The wavelet and scaling coefficients are given respectively by $\langle v, \psi_{\mu,j,i} \rangle$, and $\langle v, \phi_{j,i} \rangle$. At scale $2^{-j}$ we have $2^{3j}$ scaling and $7 \times 2^{3j}$ wavelet coefficients. We recall that the flow fields satisfy periodic boundary conditions.
## IV. MACHINE LEARNING APPROACH

We develop a methodology for predicting time-evolution of vorticity of 3D homogeneous isotropic turbulence, combining 3D CNN, LSTM, and orthogonal wavelet analysis, denoted by WCNN-LSTM here. In the following, we describe the training data and its preprocessing in Sec. IV A, and then present the procedure of WCNN-LSTM in Sec. IV B. In Sec. IV C we describe the test data and the output data.

### A. Training Data

We use 205 snapshots of the DNS data at every time interval $\Delta\tau$ in $10.00 \le t \le 26.32$ in our machine learning, where $\Delta\tau = 40\Delta t$. The time step of the DNS is $\Delta t = 2.0 \times 10^{-3}$, and here $\Delta\tau$ is about half of the Kolmogorov time $\tau_\eta$, i.e., $\Delta\tau \approx 0.5\tau_\eta$. The spatial resolution of the DNS data is $128^3$ grid points, as mentioned in Sec. II. The time correlation of vorticity is defined as $\langle\omega(\mathbf{x}, t)\cdot\omega(\mathbf{x}, t+\Delta\tau)\rangle/[\langle|\omega(\mathbf{x}, t)|^2\rangle\langle|\omega(\mathbf{x}, t+\Delta\tau)|^2\rangle]^{1/2} \approx 0.81$ and the correlation is obtained by the time-average of the snapshots. Sequential six vorticity fields at every time interval $\Delta\tau$ at $t = t_0^{(i)} + n\Delta\tau$ ($n = -4, -3, \dots, 0, 1$) are randomly chosen. Here, our batch size is 4, and $t_0^{(i)}$ ($i = 1, 2, 3, 4$) is an initial time of each set labelled by $^{(i)}$. The five fields for $n \le 0$ are used as input data, while one field for $n = 1$, i.e. at $t = t_0^{(i)} + \Delta\tau$, is used as its correct data. We here use wavelet projection for compression of the DNS data. Each snapshot is decomposed into a 3D orthogonal wavelet series applying the FWT with periodic boundary conditions, decomposing only one level. We then obtain wavelet coefficients in seven directions and the scaling coefficients at the coarser scale $2^{-J+1}$. Here we have $J = 7$. The sampled values of $v(\mathbf{x})$ are well approximated by the scaling coefficients weighted by $2^{3J/2}$ at scale $2^{-J}$.$^{47}$ Our machine learning model trains the weighted coefficients. In the case of Haar wavelets the projection corresponds to the application of a box filter, while for Coiflet 12 we use here the projection corresponds to a more sophisticated low pass filter.

### B. Machine Learning Model

We illustrate the procedure of our machine learning model WCNN-LSTM in Fig. 2. The training scheme is implemented using the TensorFlow open-source library$^{48}$ together with Python 3.6.8 interaction interface. We use ConvLSTM3D layer, LayerNormalization (LN),
<figure><img src="image_3.png" /><figcaption>FIG. 2. Illustration of the structure of the machine learning model, WCNN-LSTM.</figcaption></figure>

and Conv3D layer (keras.io) for 3D CNN and LSTM. Our machine learning model has seven layers; one input layer, five middle layers and one output layer. The first layer is the input layer, while the seventh layer is the output layer. In the middle layers, we use filters which are 3D kernels with their sizes being 3 × 3 × 3 in order to catch local features of the turbulence data. The second layer is ConvLSTM3D layer having 128 filters. The third and fifth layers are LN layers, i.e., for layer normalization. The fourth and sixth layers are Conv3D layers having 48 and 3 filters, respectively. Zero-padding is employed at each layer, to keep the size of the output data being the same as those of the input data. As the activation functions for Conv3D layer, we use the linear function. For LSTM we use tanh as activation function and sigmoid as the recurrent activation function. The adaptive moment estimation optimizer, called Adam optimizer⁴⁹ is used. The number of epochs is set to 200. The error is measured by the use of a loss function, where the mean-absolute of the difference between the output data and the correct data. Figure 3 shows a plot of the loss and the validation loss as a function of the epochs. The optimal parameters in the model are determined by using `modelcheck` such that the validation loss is the smallest in epoch considered here. Our learning follows this procedure 15 times, and then we get 15 possible models. Each model predicts its output data, using remaining training data which
<figure><img src="image_4.png" /><figcaption>FIG. 3. Epoch-dependence of the loss and validation loss in WCNN-LSTM.</figcaption></figure>

are not used in the learning of the 15 possible models. In the following we select the model where the absolute value of the mean of the output data is the lowest. Note that for the DNS data the spatial average, i.e., the mean of vorticity $\omega$ vanishes, $\langle \omega \rangle = 0$. The output of TensorFlow and the code of this machine learning is open access and can be found on GitHub https://github.com/KYoshimatsu/WCNNLSTM.git.

### C. Test Data and Output Data

The flow prediction procedure is detailed in the following. First, we use five vorticity fields obtained by the DNS at time instants, $t = \hat{t} + n\Delta\tau$ ($n = -4, -3, -2, -1, 0$) as input data, and predict the field at $t = \hat{t} + n\Delta\tau$ with $n$ being positive. Our test data are five DNS snapshots in the interval $29.60 \le t \le 29.92$ for each $\Delta\tau$. Here, $\hat{t}$ is a starting time, and we set $\hat{t}$ to 29.92.

Our model predicts the scaling coefficients at scale $2^{-J+1}$ ($J = 7$) and at time $t = \hat{t} + \Delta\tau$. We then apply IFWT to the coefficients to get the output vorticity data at $t = \hat{t} + \Delta\tau$ (= 30.0) in physical space, while the wavelet coefficients at scale $2^{-J+1}$ are set to zero. The output data in physical space satisfy periodic boundary conditions. The input data in the next step are four vorticity fields of DNS at $t = \hat{t} + n\Delta\tau$ ($n = -3, \dots, 0$) and the field predicted by WCNN-LSTM at $t = \hat{t} + \Delta\tau$. Then we predict the vorticity field at $t = \hat{t} + 2\Delta\tau$, using the same procedure as what we have used in getting the output data at $t = \hat{t} + \Delta\tau$. We apply
<table><caption>TABLE II. Time evolution of enstrophy $\mathcal{Z}$ for DNS data, CG1 data, and ML data at time $\hat{t} + n\Delta\tau$ ($n = 1, 4, 6, 8$, and 10).</caption><thead><tr><th></th><th colspan="6">Z t = t<sub>1</sub> t = t<sub>4</sub> t = t<sub>6</sub> t = t<sub>8</sub> t = t<sub>10</sub></th></tr></thead><tbody><tr><th>DNS</th><td>20.0</td><td>20.0</td><td>20.0</td><td>19.9</td><td>19.8</td><td></td></tr><tr><th>CG1</th><td>18.3</td><td>18.4</td><td>18.3</td><td>18.2</td><td>18.1</td><td></td></tr><tr><th>ML</th><td>17.9</td><td>17.9</td><td>17.5</td><td>17.2</td><td>17.1</td><td></td></tr></tbody></table>

this procedure step by step. The input data in the $\ell$-th step with $\ell \in N$ are the data at $t = \hat{t} + n\Delta\tau$ ($n = -5+, \dots, \ell$). The data for positive $n$ are the predicted data obtained by WCNN-LSTM at the previous steps, while the data for $n \le 0$ are the test data. We took another set of DNS data with different $\hat{t}$ and confirmed that we got results which are qualitatively similar to those shown in Sec. V.

## V. MACHINE LEARNING RESULTS OF VORTICITY EVOLUTION

In this section, we verify and estimate our WCNN-LSTM flow prediction, using visualization of vorticity and turbulence statistics. Figure 4 presents the visualization of the modulus of vorticity $|\omega|$ for the DNS data and the predicted data by WCNN-LSTM, at $t = \hat{t} + n\Delta\tau$ ($n = 1, 4, 6$ and 8). Here we denote the predicted data for simplicity by ML. Shown are isosurfaces satisfying $|\omega| = \omega_m + 2.5\omega_\sigma$ with $128^3$ grid points, where $\omega_m$ and $\omega_\sigma$ are respectively the mean and standard deviation of $|\omega|$ of the DNS data at each time instant. We find that pronounced vortex tubes are well preserved by ML, though we can see some discrepancy between the tubes obtained by DNS and by ML. The tubes predicted by ML are less intense compared to those obtained by DNS. To look at the vorticity obtained by the DNS and ML in detail, we visualize one vorticity component, $\omega_x$, in an $x-y$ plane at different time instants in Fig. 5. We can see that the ML vorticity excellently agrees with the DNS vorticity at $n = 1$. However, the discrepancy between the vorticity fields obtained by DNS and by ML seems to become larger with increasing time, as is expected. It is to be noted that owing to flow sensitivity of turbulence, small discrepancy between two statistically identical turbulent flows at a given time instant grows in time.
<figure><img src="image_5.png" /><figcaption>FIG. 4. Visualization of the isosurfaces of $|\omega|$ at $|\omega| = \omega_m + 2.5\omega_\sigma$ at each time instant. Here $\omega_m$ and $\omega_\sigma$ are the mean value and the standard deviation of the corresponding vorticity modulus $|\omega|$ for the DNS data, respectively: (left) DNS data and (right) ML data, which are our predicted data, at $t = \hat{t} + n\tau$ ($n = 1, 4, 6$ and 8).</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>FIG. 5. Visualization of the vorticity components $\omega_x$ on an $x-y$ plane: (left) DNS data and (right) ML data at $t = \hat{t} + n\tau$ ($n = 1, 4, 6$ and 8).</figcaption></figure>

Table II presents the enstrophy $\mathcal{Z}$ at five time instants for the ML data in comparison with $\mathcal{Z}$ for the DNS data and for the coarse-grained data at scale $2^{-J+1}$. The latter is hereafter denoted by CG1. We can see that the enstrophy $\mathcal{Z}$ for the ML data well agrees
<table><caption>TABLE III. Time evolution of energy $\mathcal{E}$ for ML data. The energy of the DNS is kept constant at 0.5, as mentioned in Sec. II. In the CG1 data, $\mathcal{E} \approx 0.5$ and therefore we omit them for brevity from Table III.</caption><thead><tr><th>ε</th><th>t = t<sub>1</sub></th><th>t = t<sub>4</sub></th><th>t = t<sub>6</sub></th><th>t = t<sub>8</sub></th><th>t = t<sub>10</sub></th></tr></thead><tbody><tr><td>ML</td><td>0.48</td><td>0.45</td><td>0.44</td><td>0.45</td><td>0.46</td></tr></tbody></table>

with $\mathcal{Z}$ for the DNS at each time instant within the difference ranging from about 10% to 14%. The difference increases with time $\hat{t} + n\Delta\tau$. The reason is mainly attributed to the wavelet projection at scale $2^{-J+1}$ necessary for getting CG1 data. We recall that before applying the IFWT the wavelet coefficients at scale $2^{-J+1}$ are set to zero in WCNN-LSTM. The enstrophy $\mathcal{Z}$ of the ML data agrees with $\mathcal{Z}$ of the CG1 data much better than $\mathcal{Z}$ of DNS. Table III gives the energy $\mathcal{E}$ for the ML data. We can see that $\mathcal{E}$ of the DNS is well reproduced by the ML at each time. The difference of $\mathcal{E}$ between the ML data and DNS data ranges between 4% to 12%. The velocity fields of the data are obtained from the predicted vorticity $\omega$ by using the Biot-Savart law: $u = -\nabla^{-2}(\nabla \times \omega)$.

We show the PDFs of the $x$-components of vorticity $\omega_x$ and velocity $u_x$ at different time instants in Fig. 6. In Fig. 6(a), we can see that the PDFs of $\omega_x$ for ML well overlap with those of $\omega_x$ for DNS at each time instant. We observe some departure between the PDFs for ML and DNS, especially in the stretched tails of the PDFs. The departure becomes larger as time progresses. The PDFs for ML are narrower than those for DNS, which implies that the vorticity predicted by ML is less intermittent than that of DNS. In Fig. 6(b), we can see that at $n = 1$, the PDF of $u_x$ for ML well overlaps with that for DNS. The PDFs are close to a Gaussian distribution. At $n = 6$ and 8, we see that the PDFs for ML depart from those of DNS. We omit the PDFs of the $y$ and $z$-components of vorticity and velocity, because the observations are the same as in the case of the PDFs of the $x$-components due to statistical isotropy.

In order to get deeper information of the ML data, we examine small-scale flow topology. The topology can be characterized by the second and third invariant of the velocity gradient tensor.50,51 These invariants are respectively defined as $Q = -(1/2)A_{ij}A_{ji}$ and $R = -\det(A_{ij})$, where $A_{ij} = \partial u_i / \partial x_j$ and the Einstein summation convention is used for
<figure><img src="image_7.png" /><figcaption>FIG. 6. PDFs of (a) the *x*-component of vorticity $\omega_x$, and (b) the *x*-component of velocity $u_x$. We here omit the PDF of $\omega_x$ for CG1 data at $n = 1$, because the PDF well agrees with the counterpart of the DNS data.</figcaption></figure>

the repeated subscripts. Here, we use the notation $(x_1, x_2, x_3) = (x, y, z)$. It is to be noted that $A_{ii} = 0$ for DNS, owing to the divergence-free condition. In our ML, $A_{ii} = 0$ likewise vanishes, because we have used the Biot-Savart law for computing the velocity fields for ML. Figure 7 shows isolines of the joint PDFs of $Q$ and $R$. The PDFs for DNS are tear-drop like, as reported in e.g., Refs. 51 and 52. Figure 7 (a) shows that the joint PDF of $Q$ and $R$ well agrees with that of DNS at $n = 1$. We can see in Figs. 7 (b) and 7 (c) for $n = 6, 8$ that the isolines of the highest value $5 \times 10^{-3}$ in the joint PDFs for ML are in fairly good agreement with those for DNS, though we can also see some discrepancy between ML and DNS PDFs for the other smaller isoline values. We omit the PDFs of $Q$ and $R$ for CG1 for brevity, because the PDFs for CG1 excellently agree with those for DNS.

Next, we analyze our ML data considering the enstrophy spectrum $Z(k)$, which is defined as $Z(k) = (1/2) \sum'_{k} |\hat{\omega}(q)|^2$ and we have $Z(k) = k^2 E(k)$, where $E(k)$ denotes the energy
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /><figcaption>FIG. 7. Joint PDFs of $Q$ and $R$ for DNS and ML at (a) $n = 1$, (b) $n = 6$, and (c) $n = 8$. The contour lines for DNS and ML are denoted by the red dashed lines and the blue solid lines, respectively. The contour lines for all cases are set to $10^{-4}$, $10^{-3}$ and $5 \times 10^{-3}$, starting near the origin. The green dotted lines represent $27R^2/4 + Q^3 = 0$.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>FIG. 8. (a) Enstrophy spectra $Z(k)$ and (b) energy spectra $E(k)$ at different time instants ($t = t_0 + n\tau, n = 1, 6, 8$) for the ML data (symbols) and the DNS data (lines).</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>FIG. 9. The divergence spectra $\Xi(k)$ of the vorticity for the ML data. The enstrophy spectrum $Z(k)$ for the DNS data at $n = 1$ is plotted as a reference.</figcaption></figure>
spectrum. The reason for selecting $Z(k)$ is to catch the statistics in the dissipation range well. Here, $\hat{\omega}(k)$ is the Fourier transform of $\omega(x)$. We see in Fig. 8(a) that the peak of $Z(k)$ where the enstrophy spectra hit the maximum is about 0.2, while the external force is imposed in $k\eta0.05$ (see Sec. II and Table I). Hence the peak departs from the forcing range. As $R_\lambda$ increases implying that $\eta$ becomes smaller, a forcing range departs further from the peak (e.g. Ref. 2). Figure 8(a) shows that the enstrophy spectra for ML are in almost perfect agreement with those of DNS. For wavenumbers $k\eta0.5$, we observe some small discrepancy of $Z(k)$ between ML and DNS. We can see that the spectrum for CG1, the coarse-grained data at scale $2^{-J+1}$, well overlaps with $Z(k)$ for ML at $n=1$. This discrepancy can thus be attributed to the wavelet projection where the wavelet coefficients at scale $2^{-J+1}$ are set to zero, as mentioned in Sec. IV B. We therefore conclude that the discrepancy is not crucial. Figure 8(b) plots the energy spectra $E(k)$ for ML and DNS. We see that the spectra of ML excellently agree with those of DNS. The observed differences for the enstrophy spectra $Z(k)$ in $k\eta0.5$ are much reduced for $E(k)$. Therefore, the enstrophy spectra yield better indicators for the verification of machine learning prediction of vorticity than the energy spectra.

In the LSTM flow prediction using an autoencoder and CNN, Nakamura et al.³⁴ show their predicted energy spectra in the high wavenumber range near by the maximum wavenumber. These are enhanced in comparison with the correct spectra for turbulent channel flow, i.e. they overpredict energy. This overprediction can be also observed in Refs. 35 and 36 for the machine-learning time-evolution prediction using an autoencoder of isotropic turbulence.

Finally, we examine the influence of the divergence of the vorticity $\omega$ for the ML data in which $\nabla \cdot \omega \neq 0$ in general, while for DNS we have $\nabla \cdot \omega = 0$. The divergent part is given by $\xi(k) = \{k \cdot \hat{\omega}(k)\}k/k^2$ in $k$ space. Figure 9 shows the spectra of this part $\Xi(k)$ given as $\Xi(k) = (1/2) \sum'_k |\xi(q)|^2$. We can see that the influence of the divergence is not crucial in particular at small scales. The values of $\Xi(k)$ at large scale, $k\eta0.05$ for later times, are somewhat comparable to the value of $Z(k)$, about 25 percent of the magnitude of $Z(k)$. However, this issue can be overcome by using divergence-free biorthogonal wavelets.⁵³,⁵⁴
# VI. APPLICATION OF WCNN-LSTM TO SUPERRESOLUTION

We now apply the pre-trained WCNN-LSTM to regenerate vortices at an instant $t_s(=30.0)$ from a given coarse-grained vorticity field at $t_s$. This can be regarded as a type of SR applied for recovering the fine-scale flow evolution from coarse-scale predicted flow data. A key quantity for the degree of the coarse-graining is the normalized wavenumber $k_c\eta$, where the larger scale data are almost or completely kept for $kk_c$ while the smaller scale data are almost or completely lost for $kk_c$. One can expect that SR becomes more difficult for decreasing $k_c\eta$.

## A. Coarse-Grained Input Data

We first obtain a coarse-grained vorticity field from the DNS data. We apply an FWT, remove the wavelet coefficients at scales $2^{-J+2}$ and $2^{-J+1}$, and then reconstruct the flow field from the scaling coefficients at scale $2^{-J+2}$ using an IFWT at scale $2^{-J}$. We call the obtained coarse-grained vorticity CG2. We confine ourselves to the same DNS data as those used in our machine learning in Sec. IV, taking the same periodic boundary conditions imposed on the DNS, as well as the memory limitation into account. The DNS has been computed at resolution $128^3$ and $k_{max}\eta \approx 1.16$, as described in Sec. II. Thus, we have $J = 7$, and the CG2 field lost the data for $k\eta \ge 0.3$. It is to be noted that the regeneration of vorticity from this type of fields is not learned by our machine learning. We then use the SR method developed by Asaka et al.$^{21}$ using wavelets, subcube division, and 3D CNN. We call the method WCNNSR here. WCNNSR learns the five snapshots of the DNS data of vorticity at $128^3$ grid points from the training data which are described in Sec. II. The data at each time instant are divided into 64 subcubes with $32^3$ grid points in physical space. The subcubes data are then transformed in wavelet space using Coiflet 12 wavelets at one level, by imposing symmetric boundary conditions on the subcubes, the so-called folding technique. In PyWavelets,$^{46}$ the boundary condition requires five extra elements of the array in each direction, and thus the size of each subcube becomes finally $8 \times 21^3$ in wavelet space. Figure 10 shows the losses in the WCNNSR as a function of epoch.

We now apply the WCNNSR to the CG2 data at $t = t_s = 30.0$, and then we use the resulting data as an input data. The CG2 data are much coarse-grained for WCNNSR, and
<figure><img src="image_13.png" /><figcaption>FIG. 10. Epoch-dependence of loss and validation loss in WCNNSR.</figcaption></figure>

therefore WCNNSR does not recover the small-scale flow contributions at least visually well. It is to be noted that the DNS data at $t = t_s$ are not our training data in the WCNNSR and the WCNN-LSTM. Figure 12 shows the isosurfaces of $|ω|$ for DNS, the coarse-grained data CG2, the predicted data using WCNN-LSTM including SR (ML), and SR applied to CG2 only (WCNNSR). The visualization for the DNS data is the same as the DNS at $n = 1$ shown in Fig. 4. The vorticity tubes observed in the DNS are not well retained in CG2, due to the coarse-graining. We can see some fragmentation of the tubes in CG2. In contrast, applying SR to CG2 using WCNNSR, which is denoted by WCNNSR for simplicity, we can observe that some tubes regenerated, however much fewer than in DNS. Moreover, the degree of coarse-graining of the WCNNSR data is almost the same as the CG2 data, as shown in Sec. VIC. Therefore, the WCNNSR data are expected to be likewise suitable as input data for WCNN-LSTM. Asaka et al.²¹ reported that the WCNNSR method works well for less coarse-grained input data at higher $R_λ$. A result of WCNN-LSTM of the CG2 data without WCNNSR is presented in Appendix A.

## B. Procedures of superresolution using WCNN-LSTM

Figure 11 illustrates our SR procedure using the pre-trained WCNN-LSTM, introduced in Sec. IV B. Our input data are four empty data sets whose values are zero for pseudo-time instants $t_s + m\Delta\tau$ ($m = -4, -3, -2, -1$) together with the predicted vorticity data
<figure><img src="image_14.png" /><figcaption>FIG. 11. SR method based on the pre-trained WCNN-LSTM.</figcaption></figure>

obtained by WCNNSR at time $t_s$, where $\Delta\tau \approx 0.5\tau_\eta$. We have chosen the four empty data
sets such that the influence of the input data at the instants given by negative indices $m$
on the SR at $t_s$ can be reduced. WCNN-LSTM predicts the scaling coefficients of vorticity
at scale $2^{-J+1}$ and at time $t_s + \Delta\tau$. Using the FWT with periodic boundary conditions, we
decompose them into the scaling coefficients at scale $2^{-J+2}$ and the wavelet coefficients at
scale $2^{-J+2}$. Then, we copy the scaling coefficients at scale $2^{-J+2}$ of the input WCNNSR
data at $t_s$ to the scaling coefficients at scale $2^{-J+2}$ which have been obtained in the above
decomposition. Therefore, the information corresponding to the wavelet coefficients at scale
$2^{-J+2}$ evolves in our SR results. Afterwards using the IFWT, we obtain the output data in
physical space at $t_s + \Delta\tau$. They are then used as input data in the next step, in addition
to three empty data sets at $t_s + m\Delta\tau$ ($m = -3, -2, -1$) and one WCNNSR data at $t_s$. We
obtain the output data at $t_s + 2\Delta\tau$. The input data in the $\ell$-th step are the data $t_s + m\Delta\tau$
($m = -4 + \ell, -3 + \ell, -2 + \ell, -1 + \ell$), where $\ell \in N$. The data at negative indices $m$ are
empty, the data at $m=0$ are the WCNNSR data, and the data at positive indices $m$ are
predicted by the WCNN-LSTM with the above-mentioned copy of the scaling coefficients at
scale $2^{-J+2}$ of the WCNNSR data. The data at $t_s + \ell\Delta\tau$ are obtained by the same procedure
described above. Eventually we get the output data in physical space using IFWT with the
<figure><img src="image_15.png" /><figcaption>FIG. 12. Visualization of isosurfaces of $|\omega|$ at $|\omega| = \omega_m + 2.5\omega_\sigma$ for the DNS data at $t = t_s$, the CG2 data at $t = t_s$, the SR data obtained by WCNN-LSTM, which is denoted by ML, at $m = 4$, and the WCNNSR data at $t = t_s$. Here, $\omega_m$ and $\omega_\sigma$ are the mean value and the standard deviation of $|\omega|$ for the DNS data, respectively. The grid resolution is 128³.</figcaption></figure>

periodic boundary conditions at each pseudo-time instant.

### C. Results of the superresolution

Now, we verify the SR using the WCNN-LSTM to recover fine-scale information of the flow. Figure 12 shows the isosurfaces of vorticity magnitude $|\omega|$. ML denotes the visualization for the SR of the WCNNSR result, by using the WCNN-LSTM. The ML results seem to well preserve most of the positions of the tubes in DNS, though we can observe some differences between the visualizations of DNS and ML: the isosurfaces of DNS are smoother than those of ML. Figure 13 visualizes $\omega_x$ on an $x-y$ plane. We can see that ML reproduces the vorticity distribution of DNS well, though there are again some differences between ML
<figure><img src="image_16.png" /><figcaption>FIG. 13. Visualization of the vorticity components $\omega_x$ on an $x-y$ plane for the DNS data at $t = t_s$, the CG2 data at $t = t_s$, the ML data at $m = 4$, and the WCNNSR data at $t = t_s$.</figcaption></figure>

and DNS looking at details. Table IV shows the enstrophy $\mathcal{Z}$ for DNS, CG2, WCNNSR, and ML. The enstrophy $\mathcal{Z}$ for ML is comparable to $\mathcal{Z}$ for DNS, while the other values for CG2 and WCNNSR data are much reduced.

We move on to the statistics of small-scale quantities. Figure 14 gives a plot of PDFs of $\omega_x$ for the different fields. We observe that the PDFs for WCNNSR and CG2 are much narrower than the PDF for DNS. We can also see that the PDF of ML becomes wider as the pseudo-time progresses. We find that the PDF of ML at $m = 4$ (orange line) is in good agreement with that of DNS, though the former is somewhat narrower than the latter. In Fig. 15, we compare the joint PDFs of the second and third invariant of the velocity gradient tensor, $Q$ and $R$, for WCNNSR and ML ($m = 4$) with those of DNS. We can see that the PDFs of WCNNSR and ML well agree with the joint PDF of DNS. The departure from the DNS results can be seen only at the smallest isoline value, $10^{-4}$.

To get deeper insight into the small scales, we examine the enstrophy spectra $Z(k)$. In Fig. 16, we can see that for CG2 and WCNNSR, the enstrophy spectra are much reduced
<table><caption>TABLE IV. Enstrophy of the DNS data at $t = t_s$, the CG2 data at $t = t_s$, the ML data at $m = 4$, and the WCNNSR data at $t = t_s$.</caption><thead><tr><th></th><th>DNS</th><th>CG2</th><th>WCNNSR</th><th>ML</th></tr></thead><tbody><tr><td>Z</td><td>20.0</td><td>13.1</td><td>14.4</td><td>19.4</td></tr></tbody></table>

<figure><img src="image_17.png" /><figcaption>FIG. 14. PDFs of $\omega_x$ for DNS, ML ($m = 1, 3, 4$), WCNNSR and CG2.</figcaption></figure>

for $k\eta_{0.2}$. We find that the degree of coarse-graining of the WCNNSR data is almost the same as that of the CG2 data in terms of $Z(k)$. The values of $Z(k)$ for ML for $k\eta_{0.2}$ grow in the pseudo-time, and almost saturate at $m = 3$ and $m = 4$. We can see that $Z(k)$ for ML at $m = 4$ excellently agrees with that for DNS. We recall that we copy the scaling coefficients at scale $2^{-J+2}$ ($J = 7$) of the WCNN data to the output data every pseudo-time instant in the SR method using the WCNN-LSTM prediction. Therefore, the enstrophy spectra of ML are almost the same as $Z(k)$ for WCNNSR for $k\eta_{0.2}$. Figure 17 shows the divergence spectra of the vorticity for ML. This confirms that the influence of divergence is not crucial.

## VII. CONCLUSIONS

We have developed a wavelet-based machine learning method (WCNN-LSTM) for predicting the time evolution of vorticity for homogeneous isotropic turbulence where vortex tubes are preserved. To this end, we combined two neural network architectures, 3D CNN
<figure><img src="image_18.png" /><figcaption>FIG. 15. Joint PDFs of $Q$ and $R$ for (a) DNS and WCNNSR, and for (b) DNS and ML ($m = 4$). The contour lines for DNS, WCNNSR, and ML are denoted by the red dashed lines, black solid lines, and the blue solid lines, respectively. The contour lines for all cases are set to $10^{-4}$, $10^{-3}$ and $5 \times 10^{-3}$, starting near the origin. The green dotted lines represent $27R^2/4 + Q^3 = 0$.</figcaption></figure>

and LSTM. The latter permits learning the time evolution, while the former trains the network to reproduce not only the turbulent flow but also the pronounced and well-localized vortex tubes. We have used compactly supported orthogonal wavelets. By construction wavelets well catch the information of position and scale of the fields. The wavelets efficiently represent multi-scale and intermittent fields, here turbulent vorticity fields exhibiting vortex tubes.44,45 The fast wavelet transform and its inverse allow switching rapidly between physical space and wavelet coefficient space. Wavelet projection is then used to reduce the size of the input data. Therefore, the projection reduces the memory required for learning. We applied the developed WCNN-LSTM to DNS data of homogeneous isotropic
<figure><img src="image_19.png" /><figcaption>FIG. 16. Enstrophy spectra $Z(k)$ vs $k\eta$ for DNS, ML ($m = 1, 3, 4$), WCNNSR and CG2.</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>FIG. 17. The divergence spectra $\Xi(k)$ vs $k\eta$ for ML ($m = 1, 3, 4$). The enstrophy spectrum $Z(k)$ of the DNS data at $t_s$ is plotted as a reference.</figcaption></figure>

turbulence in a $(2\pi)^3$ periodic box at the Taylor microscale Reynolds number 92 computed at resolution $128^3$ and $k_{max}\eta \approx 1.16$. The dimensionless wavenumber $k\eta$ where the enstrophy spectra hit the maximum is about 0.2, while the external force is imposed in $k\eta$ 0.05. The results predicted by WCNN-LSTM have been assessed in comparison with the DNS data. The predicted fields satisfy periodic boundary conditions by using the IFWT. Visualization of isosurfaces of vorticity magnitude showed that vortex tubes are well retained in the WCNN-LSTM flow prediction. The PDFs of vorticity and velocity, which are predicted by WCNN-LSTM, well agree with those of the DNS data at time instants larger than $3\tau_\eta$, where $\tau_\eta$ is the Kolmogorov time-scale. The flow topology characterized by second and third
invariants of the velocity gradients is likewise well retained by WCNN-LSTM. We employed
the enstrophy spectra to get deeper information into the dissipation range. We observed
that the predicted enstrophy spectra well agree with those for the DNS. The influence of the
divergence of the predicted vorticity was shown to be negligible. As expected we found that
the enstrophy spectra are more suitable than the energy spectra for verification of machine
learning prediction of vorticity.

Then, we applied our pre-trained WCNN-LSTM model to the SR of a coarse-grained
vorticity of the DNS data at a time instant $t_s$. The information at $k\eta0.2$ is much reduced
in the coarse-grained data, and the vortex tubes of the DNS data at $t = t_s$ are lost in
the coarse-grained vorticity. We showed that the vortex tubes are well regenerated from
the coarse-grained vorticity by the SR, though the previously developed wavelet-based SR
model at a time instant²¹ failed in the SR of the coarse-grained vorticity. It is to be noted
that the WCNN-LSTM model was not trained to learn the regeneration. The vortex tubes
are visualized by using isosurfaces of the modulus of the vorticity. We can see that the
isosurfaces for the predicted vorticity well agree with those of the DNS data, though the
latter are smoother than the former. Further improvement of the quality of SR using machine
learning of the time evolution of vorticity remains an interesting issue for future studies.

Predicting the dynamics in fully developed turbulent flow with DNS is a costly endeavor,
because the number of degrees of freedom increases approximately with $R_\lambda^{9/2}$. Machine
learning of turbulent flows thus requires much more memory and computational cost, as the
Reynolds number increases. The memory and cost could be reduced by multi-level wavelet
decomposition with or without nonlinear wavelet filtering. In machine learning based on
multi-level wavelet decomposition, the wavelet coefficients at each scale are learned, which
means scale-by-scale machine learning. In Farge et al.,⁵⁵ coherent vortex simulation was
proposed to compute the time evolution of the coherent vorticity, while neglecting the influ-
ence of the incoherent flow to model turbulence dissipation. The coherent vorticity consists
of few intense wavelet coefficients of vorticity, and the intense coefficients are extracted by
using wavelet nonlinear filtering.⁴⁴,⁴⁵ Tracking the time evolution of the coherent vorticity
in wavelet space while reducing the required memory needs adaptive computation based on
wavelets. The application of the adaptive computation to the machine learning remains an
open issue. Moreover, the concept of time parallelization (e.g., Ref. 56) could be useful
for efficient learning longer time evolution of turbulence. We look forward to parallelization
of machine learning that will resolve the present limitation of the GPU memory and the
computational time. It could be also interesting to examine whether machine learning can
reconstruct the time evolution of 3D turbulent flows from 2D slices of the flow. The pre-
diction of time evolution of turbulence with different dynamics, e.g., inertial particles can
be interesting on the basis of Oujia et al.<sup>57,58</sup> for synthesizing preferential concentration of
particles in isotropic turbulence.

ACKNOWLEDGMENTS

This work used computational resources of the supercomputer “Flow” provided by Infor-
mation Technology Center, Nagoya University, partially through the HPCI System Research
Project (Project ID: hp230143). KS acknowledges partial support by the French Federa-
tion for Magnetic Fusion Studies (FR-FCM) and the Eurofusion consortium, funded by the
Euratom research and training programme 2021-2022 under grant agreement No 633053.
The views and opinions expressed herein do not necessarily reflect those of the European
Commission. We also acknowledge T. Maurel-Oujia for fruitful discussion on this paper.

Appendix A: WCNN-LSTM without WCNNSR

We shortly describe the SR of the CG2 data by using the WCNN-LSTM without WC-
NNSR. This SR is here denoted by SRCG2. We select the enstrophy spectra Z(k) here. The
spectra provide good verification of the results in the dissipation range, as discussed in Sec.
VIC. Figure 18 shows that Z(k) for SRCG2 at m = 4 takes somewhat larger values than
the Z(k) of DNS at m = 4 for kη ≈ 0.4.

REFERENCES

a

<footnote>$^b$ask05102569@gmail.com</footnote>

<footnote>$^c$yoshimatsu@nagoya-u.jp</footnote>

<footnote>$^d$kai.schneider@univ-amu.fr</footnote>
<figure><img src="image_21.png" /><figcaption>FIG. 18. Enstrophy spectrum $Z(k)$ vs $k\eta$ for SRCG2 at $m = 4$ together with the enstrophy spectra for DNS, ML at $m = 4$, and CG2.</figcaption></figure>

<footnote n="1">¹J. Jiménez, A. A. Wray, P. G. Saffman, and R. S. Rogallo, “The structure of intense vorticity in isotropic turbulence,” J. Fluid Mech. **255**, 65 (1993).</footnote>

<footnote n="2">²Y. Kaneda and T. Ishihara, “High-resolution direct numerical simulation of turbulence,” J. Turbul. **7**, N20 (2006).</footnote>

<footnote n="3">³L. Brandt and F. Coletti, “Particle-laden turbulence: Progress and perspectives,” Annu. Rev. Fluid Mech. **54**, 159 (2022).</footnote>

<footnote n="4">⁴M. Tanahashi, M. Fujimura, and T. Miyauchi, “Coherent fine-scale eddies in turbulent premixed flames,” Proc. Combust. Inst. **28**, 529 (2000).</footnote>

<footnote n="5">⁵A. La Porta, G. A. Voth, A. M. Crawford, J. Alexander, and E. Bodenschatz, “Fluid particle accelerations in fully developed turbulence,” Nature **409**, 1017 (2001).</footnote>

<footnote n="6">⁶K. Duraisamy, G. Iaccarino, and H. Xiao, “Turbulence modeling in the age of data,” Annu. Rev. Fluid Mech. **51**, 357 (2019).</footnote>

<footnote n="7">⁷S. L. Brunton, B. R. Noack, and P. Koumoutsakos, “Machine learning for fluid mechanics,” Annu. Rev. Fluid Mech. **52**, 477 (2020).</footnote>

<footnote n="8">⁸S. Pandey, J. Schumacher, and K. R. Sreenivasan, “A perspective on machine learning in turbulent flows,” J. Turbul. **21**, 9–10:567 (2020).</footnote>

<footnote n="9">⁹D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner, and S. Hoyer, “Machine learning-accelerated computational fluid dynamics,” Proc. Nat. Acad. Sci. **118**, e2101784118 (2021).</footnote>
<footnote n="10">¹⁰R. Vinuesa and S. L. Brunton, “Enhancing computational fluid dynamics with machine learning,” Nat. Comput. Sci. **2**, 358 (2022).</footnote>

<footnote n="11">¹¹S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. **20**, 3, 21 (2003).</footnote>

<footnote n="12">¹²C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in *Computer Vision – ECCV 2014*, edited by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars (Springer International Publishing, Cham, 2014) pp. 184–199.</footnote>

<footnote n="13">¹³K. Fukami, K. Fukagata, and K. Taira, “Super-resolution reconstruction of turbulent flows with machine learning,” J. Fluid Mech. **870**, 106 (2019).</footnote>

<footnote n="14">¹⁴R. Onishi, D. Sugiyama, and K. Matsuda, “Super-resolution simulation for real-time prediction of urban micrometeorology,” SOLA **15**, 178 (2019).</footnote>

<footnote n="15">¹⁵B. Liu, J. Tang, H. Huang, and X.-Y. Lu, “Deep learning methods for super-resolution reconstruction of turbulent flows,” Phys. Fluids **32**, 025105 (2020).</footnote>

<footnote n="16">¹⁶H. Kim, J. Kim, S. Won, and C. Lee, “Unsupervised deep learning for super-resolution reconstruction of turbulence,” J. Fluid Mech. **910**, A29 (2021).</footnote>

<footnote n="17">¹⁷C. Ledig, L. Theis, F. Huszr, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” in *2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)* (2017) pp. 105–114.</footnote>

<footnote n="18">¹⁸J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in *2017 IEEE International Conference on Computer Vision (ICCV)* (2017) pp. 2242–2251.</footnote>

<footnote n="19">¹⁹A. Güemes, S. Discetti, A. Ianiro, B. Sirmacek, H. Azizpour, and R. Vinuesa, “From coarse wall measurements to turbulent velocity fields through deep learning,” Phys. Fluids **33**, 075121 (2021).</footnote>

<footnote n="20">²⁰M. Z. Yousif, L. Yu, S. Hoyas, R. Vinuesa, and H.-C. Lim, “A deep-learning approach for reconstructing 3D turbulent flows from 2D observation data,” Sci. Rep. **13**, 2529 (2023).</footnote>

<footnote n="21">²¹T. Asaka, K. Yoshimatsu, and K. Schneider, “A wavelet-based three-dimensional convolutional neural network for superresolution of turbulent vorticity In *15th World Congress on Computational Mechanics (WCCM-XV)*,” https://hal.science/hal-03739056/document (2022).</footnote>
<footnote n="22"><sup>22</sup>K. Fukami, K. Fukagata, and K. Taira, "Super-resolution analysis via machine learning: a survey for fluid flows," Theor. Comput. Fluid Dyn. **37**, 421 (2023).</footnote>

<footnote n="23"><sup>23</sup>K. Fukami, R. Maulik, N. Ramachandra, K. Fukagata, and K. Taira, "Global field reconstruction from sparse sensors with voronoi tessellation-assisted deep learning," Nat. Mach. Intell. **3**, 945 (2021).</footnote>

<footnote n="24"><sup>24</sup>A. Güemes, C. Sanmiguel Vila, and S. Discetti, "Super-resolution generative adversarial networks of randomly-seeded fields," Nat. Mach. Intell. **4**, 1165 (2022).</footnote>

<footnote n="25"><sup>25</sup>J. E. Santos, Z. R. Fox, A. Mohan, D. OMalley, H. Viswanathan, and N. Lubbers, "Development of the senseiver for efficient field reconstruction from sparse observations," Nat. Mach. Intell. **5**, 1317 (2023).</footnote>

<footnote n="26"><sup>26</sup>M. Z. Yousif, M. Zhang, L. Yu, R. Vinuesa, and H.-C. Lim, "A transformer-based synthetic-inflow generator for spatially developing turbulent boundary layers," J. Fluid Mech. **957**, A6 (2023).</footnote>

<footnote n="27"><sup>27</sup>A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," in *Advances in Neural Information Processing Systems 30 (NIPS 2017)*, edited by I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. h.-P. Garnett (Curran Associates, Inc., 2017).</footnote>

<footnote n="28"><sup>28</sup>M. Z. Yousif, L. Yu, and H.-C. Lim, "High-fidelity reconstruction of turbulent flow from spatially limited data using enhanced super-resolution generative adversarial network," Phys. Fluids **33**, 125119 (2021).</footnote>

<footnote n="29"><sup>29</sup>K. Yoshida, J. Yamaguchi, and Y. Kaneda, "Regeneration of small eddies by data assimilation in turbulence," Phys. Rev. Lett. **94**, 014501 (2005).</footnote>

<footnote n="30"><sup>30</sup>K. Hasegawa, K. Fukami, T. Murata, and K. Fukagata, "Machine-learning-based reduced-order modeling for unsteady flows around bluff bodies of various shapes," Theor. Comput. Fluid Dyn. **34**, 367 (2020).</footnote>

<footnote n="31"><sup>31</sup>K. Hasegawa, K. Fukami, T. Murata, and K. Fukagata, "CNN-LSTM based reduced order modeling of two dimensional unsteady flows around a circular cylinder at different Reynolds numbers," Fluid Dyn. Res. **52**, 065501 (2020).</footnote>

<footnote n="32"><sup>32</sup>S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput. **9**, 1735 (1997).</footnote>

<footnote n="33"><sup>33</sup>G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science **313**, 504 (2006).</footnote>
<footnote n="34"><sup>34</sup>T. Nakamura, K. Fukami, K. Hasegawa, Y. Nabae, and K. Fukagata, “Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow,” Phys. Fluids **33**, 025116 (2021).</footnote>

<footnote n="35"><sup>35</sup>A. T. Mohan, D. Tretiak, M. Chertkov, and D. Livescu, “Spatio-temporal deep learning models of 3D turbulence with physics informed diagnostics,” J. Turbul. **21**, 9–10:484 (2020).</footnote>

<footnote n="36"><sup>36</sup>V. Shanker, G. D. Portwood, A. T. Mohan, P. P. Mitra, S. Krishnamurthy, C. Rackauckas, L. A. Wilson, D. P. Schmidt, and V. Viswanathan, “Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow,” Phys. Fluids **34**, 115110 (2022).</footnote>

<footnote n="37"><sup>37</sup>R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, “Neural ordinary differential equations,” arXiv:1806.07366v5 (2019).</footnote>

<footnote n="38"><sup>38</sup>L. Guastoni, A. Güemes, A. Ianiro, S. Discetti, P. Schlatter, H. Azizpour, and R. Vinuesa, “Convolutional-network models to predict wall-bounded turbulence from wall quantities,” J. Fluid Mech. **928**, A27 (2021).</footnote>

<footnote n="39"><sup>39</sup>D. Lucor, A. Agrawal, and A. Sergent, “Simple computational strategies for more effective physics-informed neural networks modeling of turbulent natural convection,” J. Comput. Phys. **456**, 111022 (2022).</footnote>

<footnote n="40"><sup>40</sup>A. T. Mohan, D. Livescu, and M. Chertkov, “Wavelet-powered neural networks for turbulence,” in *ICLR 2020 Workshop on Tackling Climate Change with Machine Learning* (2020).</footnote>

<footnote n="41"><sup>41</sup>W. Peng, Z. Yuan, Z. Li, and J. Wang, “Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence,” Phys. Fluids **35**, 015106 (2023).</footnote>

<footnote n="42"><sup>42</sup>Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Fourier neural operator for parametric partial differential equations,” arXiv:2010.08895v3 (2021).</footnote>

<footnote n="43"><sup>43</sup>Z. Li, W. Peng, Z. Yuan, and J. Wang, “Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator,” Phys. Fluids **35**, 075145 (2023).</footnote>

<footnote n="44"><sup>44</sup>M. Farge, G. Pellegrino, and K. Schneider, “Coherent vortex extraction in 3D turbulent flows using orthogonal wavelets,” Phys. Rev. Lett. **87**, 054501 (2001).</footnote>

<footnote n="45"><sup>45</sup>N. Okamoto, K. Yoshimatsu, K. Schneider, M. K., Farge, and Y. Kaneda, “Coherent vortices in high resolution direct numerical simulation of homogeneous isotropic turbulence:</footnote>
A wavelet viewpoint,” Phys. Fluids **19**, 115109 (2007).

<sup>46</sup>G. R. Lee, R. Gommers, F. Waselewski, K. Wohlfahrt, and A. O’Leary, “Pywavelets: A python package for wavelet analysis,” J. Open Source Softw. **4**, 1237 (2019).

<sup>47</sup>S. Mallat, *A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way* (Elsevier, 2008).

<sup>48</sup>M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man, R. Monga, S. Moore, D. Mur-ray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Van-houcke, V. Vasudevan, F. Vigas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” http://download.tensorflow.org/paper/whitepaper2015.pdf (2015).

<sup>49</sup>D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412.6980v9 (2017).

<sup>50</sup>M. S. Chong, A. E. Perry, and B. J. Cantwell, “A general classification of three-dimensional flow fields,” Phys. Fluids A **2**, 765 (1990).

<sup>51</sup>J. Soria, R. Sondergaard, B. J. Cantwell, M. S. Chong, and A. E. Perry, “A study of the fine-scale motions of incompressible time-developing mixing layers,” Phys. Fluids **6**, 871 (1994).

<sup>52</sup>L. Chevillard, C. Meneveau, L. Biferale, and F. Toschi, “Modeling the pressure Hessian and viscous Laplacian in turbulence: Comparisons with direct numerical simulation and implications on velocity gradient dynamics,” Phys. Fluids **20**, 101504 (2008).

<sup>53</sup>E. Deriaz and V. Perrier, “Divergence-free and curl-free wavelets in two dimensions and three dimensions: application to turbulent flows,” J. Turbul. **7**, N3 (2006).

<sup>54</sup>E. Deriaz, M. Domingues, V. Perrier, K. Schneider, and M. Farge, “Divergence-free wavelets for coherent vortex extraction in 3d homogeneous isotropic turbulence,” in *ESAIM: Proceedings*, Vol. 16 (EDP Sciences, 2007) p. 146.

<sup>55</sup>M. Farge, K. Schneider, and N. Kevlahan, “Non-Gaussianity and coherent vortex simulation for two-dimensional turbulence using an adaptive orthonormal wavelet basis,” Phys. Fluids **11**, 2187 (1999).

<sup>56</sup>M. J. Gander, “50 years of time parallel time integration,” in *Multiple Shooting and Time Domain Decomposition Methods: Mus-TDD*, Heidelberg, May 6-8, 2013 (Springer, 2015)
p. 69.

<footnote n="57">⁵⁷T. Oujia, S. S. Jain, K. Matsuda, K. Schneider, J. West, and K. Maeda, “Neural networks for synthesizing preferential concentration of particles in isotropic turbulence,” in *Proceedings of the Summer Program 2022, Center for Turbulence Research, Stanford University* (2022) p. 153.</footnote>

<footnote n="58">⁵⁸T. Maurel-Oujia, S. S. Jain, K. Matsuda, K. Schneider, J. R. West, and K. Maeda, “Neural network models for preferential concentration of particles in two-dimensional turbulence,” arXiv:2312.14829v1 (2023).</footnote>