A New Perspective on Fluid Simulation: An Image-to-Image Translation Task via Neural Networks
Authors: Roman Lehmann
Markus Hoffmann
Simon Leufen
Wolfgang Karl
Venue: ICLR 2022 Submitted (2022)
Keywords: cGAN
CFD
Image-to-Image
Fluid Simulation

Abstract: Standard numerical methods for creating simulation models in the field of fluid dynamics are designed to be close to perfection, which results in high computational effort and high computation times in many cases. Unfortunately, there is no mathematical way to decrease this correctness in cases where only approximate predictions are needed. For such cases, we developed an approach based on Neural Networks that is much less time-consuming but nearly as accurate as the numerical model for a human observer. We show that we can keep our results stable and nearly indistinguishable from their numerical counterparts over tenth to hundreds of time steps.

Under review as a conference paper at ICLR 2022
A NEW PERSPECTIVE ON FLUID SIMULATION: AN IMAGE-TO-IMAGE TRANSLATION TASK VIA NEURAL NETWORKS
Anonymous authors Paper under double-bling review
ABSTRACT
Standard numerical methods for creating simulation models in the field of fluid dynamics are designed to be close to perfection, which results in high computa- tional effort and high computation times in many cases. Unfortunately, there is no mathematical way to decrease this correctness in cases where only approxi- mate predictions are needed. For such cases, we developed an approach based on Neural Networks that is much less time-consuming but nearly as accurate as the numerical model for a human observer. We show that we can keep our results stable and nearly indistinguishable from their numerical counterparts over tenth to hundreds of time steps.
INTRODUCTION
1.1
TARGET ISSUE
Simulating fluids, streams, and flows is task in many fields of science. In most cases, this done by numerical approaches like the method of finite elements (FEM) (Quarteroni & Valli] (208) Lattice Boltzmann methods (LBM) (Mohamad) (011). These methods are providing great advantages like configurable accuracy and fine-grained adaptability regarding the specific given task. However, to get close-to-reality simulations, some issues have to be targeted as well.
One of the most important issues emerges from the approximation property of these methods. This property limits the configuration of accuracy in both directions: if the accuracy becomes too low, the approximation will become to coarse in other words, will no longer reflect reality respectively fulfill the given task. In contrast, if it becomes too high, effects like the curse of dimension will set in, causing exploding problem sizes. In the result, these methods will have an only minimal adjustable computation time range, as the accuracy strongly connects to the problem size and therefore also to computation time, and may exceed every given maximum time span.
To avoid this issue, different approach might be more satisfying. As our focus is on problems with no need of high accuracy, we decided to follow a more approximate path by involving the human observer. This observer typically isn't interested in the numerical representation of the simulation, but in the easy-to-interpee to-image translation for each time step of the simulation. The results of such an approach will be much more inaccurate as typical numerical results - the very limited resolution of the color spaces alone results in high errors compared FEM LBM results - but for wide class of problems the results will be good enough under the condition that the computations are very fast. Therefore, the main task for this approach is translate an image representation time step n into the image representation of step n + 1 as fast and as accurate as possible. For translation, we decided to use neural network based on good results for mmage-to-image translation (Isola al. 2017).
image- based one. That leads directly to the idea to focus on an image-
In total, this approach - starting with an image representation of the starting values and translate it recursively into the next time steps the simulation leads the following question:
Is it possible... regarding computation time and accuracy. Can get results fast enough to accept the additional approximation? Is the accuracy high enough to get useful results? Will this accuracy remain over many simulation time steps?

---

Under review as a conference paper at ICLR 2022
to get accurate simulations ... regarding the real world and the human observer. How much noise in the picture is too much? When and why does our approach begin to fail?
very fast... regarding the computation time and deployed hardware. After training, is it possible run complex simulation on my laptop at home? Will it be fast enough to meet real-time conditions?
with this approach ... regarding the recursive usage of neural networks as well as an unedited image input. Can we use unedited data, or do we need preprocessing steps like FFT? Is the recursive approach expedient? Are image processing steps like morphological filtering needed between each step? Is the cGAN approach with a UNET architecture reasonable? Do we need additional LSTM units?
... while using as little as possible parameters? regarding the input-output-ratio. How much data is needed to produce the same output as LBM or FEM? How generalizable is our approach? Do we need to train every explicit model, geometry, and structure or possible to transfer results?
Some emerging questions aren't 't trivial all, and this paper can't answer all of them. With our work, we want to show the benefits of the mentioned approach and highlight opportunities and limits.
1.2 STATE OF THE ART
As our approach covers multiple topics, this state-of-the-art section will compare our work to nu- merical approaches, image translation methods, architectures of neural networks, comparable ap- proaches for fluid dynamics, and previous work our approach is based on. For a better overview, we will headline the corresponding sections and summarize the latest developments.
Numerical Approaches: Standard numerical methods for solving PDEs, like FEM (Quarteroni & Valli 20088 or LBM (Mohamad [20II), are widely known. Ideas like approximate preconditioning
(Anzt et al. 2018) or multi-precision solvers of systems of equations (Gratton et al. 2019 Aliaga ct al. 2020) are emerging over the last years. These approaches can have great impact on a specific part of the numerical PDE solver, but this doesn't 't necessarily lead to faster run-times for the whole PDE solver. There are also some more global approaches
speed up a complete PDE solving method thanks to mathematical optimization, like Gracie et al. (2006); Du & Wang (2015); Etzmuss jet al, (2003), but these approaches are highly adapted to a specific problem. Based on these findings, pure mathematical approach doesn't 't seem to be the right way.
Our basic idea is based on image translation with a neural network. In the last years many approaches in this direction appear, manly (but not only) to manipulate images or movies in real-time (Liu et al. 2017; Radford et al., 2015) Zhao et al.) 2020). Most of these approaches are not really matching our task, which results in unsuitable network architectures or unrealizable constraints. But there is one matching approach with Isola et al. (2017). Our cGAN approach is inspired by the ideas and the excellent results given there and in additional work on cGAN structures like Karras et al. (017); Zhang et al. 2017).
Image Translation:
Neural Network Architectures: In addition to the cGAN approach, we need to find the right architecture for our neural networks. Based on Isola et al. (017) we used a PatchGAN architecture (Li & Wand, 2016) for the discriminator part. For the generator part, we used the proposed U-Net architecture (Isola et al.) (017). Regarding the iterative structure of our translation task, we added the idea of long short-term memory modules (LSTM) (Hochreiter & Schmidhuber 1997) to improve the U-NET structure.
Neural Fluid Dynamics and PDE Solvers: Combining numerical methods with machine learning algorithms is not entirely new, even in the field of fluid dynamics. Two very up-to-date examples are Li et al. (2020) and Pfaff et al. (022I. While the first one focuses on the numerical operators and numerical errors, the second one tries to work with the numerical discretization. Both as other approaches before are following approaches starting within the numerical solving pipeline. Our approach to work solely on the image representation clearly separates from previous approaches and is unique characteristic of this paper. Unfortunately. our work is not yet at the point to be fully comparable with these approaches, but we are aiming in the near future.
2

---

Under review as a conference paper at ICLR 2022
Additional Influences: Finally, there is one more idea we have to mention part of the base of our work. In Lehmann et al. (2020) the authors showed, why and how a binary map is a good option to define sharp separation of areas within image regarding the image translation task. We adopted this option for our approach, as we need a sharp separation between the streaming area and the environment area in the image representation.
1.3 CONTENT ORGANIZATION
In detail, we will provide data for the following main findings in this paper:
Using -apppxa with U-NET structure and a cGAN training is useful way to get approximated simulation results
Results from recursive application of neuronal networks useful over tenth hundreds of iterations, including an only moderate increasing additional approximation error
The speed-up can be upto 9 (on GPU-based hardware) compared to a FEM-based simula- tion
Different color spaces, boundary mappings, and input data sets are possible and may lead to data-driven approaches
We structured this paper as the following: We start with some basic knowledge about FEM and image comparison, followed by explanations of our neuronal network architecture, and our data generation in section 2. Section 3 will cover our results for our test setting (see (A.3) and will give look over the edge of this specific configured environment. We close this work with a conclusion and summary in section 4 as well as view our future directions. The statement of r/eproducibility can found at the very end.
2 THEORETICAL BACKGROUND
2.1 NUMERICAL BASIS AND MESSURMENT
Looking into approach, everything starts with given PDE (partial differential equation), the mathematical model behind descriptions of physical phenomenons.
our case, this is the Navier- Stokes equations for incompressible flows (Oymak & Selcuk 1996). The starting conditions, bound- ary conditions, and physical parameters are chosen in a way to get a Kármán vortex street within a
canal with an obstacle (fig. T (Schäfer al. 1996). For discretization, the method of lines (Oymak)
Kármán vortex street
Oueflow
Inflow
Eng-alip
Lout
Figure 1: Sketch of the experiment with die inflow boundary condition I'm (inflow speed), the out- flow boundary condition Tout (outflow speed), and with and a no-slip boundary condition I no-sslip-
Selcuk 1996) is used, which leads to discrete time steps each equipped with the same discrete space grid. The values derived from the chosen solvers are finally mapped to chosen color space (in our case it is the [0, 255) grayscale).
This very rough spotlight on the basic method shows where the main issue can be located: There are a lot approximation errors in the reformulation, discretization, numerical solvers, and in final mapping to the color space (Quarteroni & Valli, 2008). Additionally, the single errors may (or may not) accumulate, which is the reason it can be very difficult to change a single solver or transformation method out of the solving pipeline. Especially in the case of neural networks, where
O-step methods for the time direction (Berzins & Furzeland Nemec Zingg 2002) for the non-linear equations in space.
1992 and variant of Newton's 's method
3

---

Under review as a conference paper at ICLR 2022
no useful formal enror limit guaranties can be given, a global replacement approach seems be the more promising than changing a single pipeline step.
As this means to replace the whole numerical method with neural network and only keep the start- ing values to create the first input data, we have to solve the issue to define a meaningful quality measurement for results. For run time simple time difference, respectively speed-up mea- surement, is good enough. For accuracy, comparing the numerically created image of a time step with one created by neural network pixel by pixel may lead to high errors, even if the images are indistinguishable by a human observer. However, the same measurement may only result in small errors even if one can find obvious false streaming data (with small changes in the colors). As we found in previous testing, this problem will occur in our case also for average pixel errors and correlation-ban is based on the mean square error (MSE) and originally developed to evaluate the quality of lossy compression algorithms. It is called peak signal-to-nonl noise ratio (PSNR) (Korhonen & You 2012):
measurements. Therefore, the most promising quality measurement for us
255
EPSNR 10 log10
ddB..
(1)
ÊMSE
In theory, higher PSNR value means less detectable differences in the images, and values above 30 dB should result only undetectable differences for human observer (Mehra, 2016). prac- tice, we noticed that in many cases, values below this significant value might be acceptable as well. The reason that the first observable errors are irrelevant ones regarding our setting, like discolored vertical horizontal pixel lines that obviously cannot be interpreted streaming data. Therefore, we marked the theoretical limit in our charts and stopped our iteration some steps later.
2.2 BASIC APPROACII AND ARCHITECTURE OF NEURONAL NETWORKS
As mentioned earlier, we see the task of fluid simulation within the calculation of the next time step combined with the representation of the result in human-readable fashion in slightly different way. What we see a representation (image) of a current state of the fluid flow, and the goal is a representation (image) of the next state of the fluid flow. Therefore, our eyes, this is nothing else than an pongge-to-image translation task.
Well-known approach for this is pix2pix (Isola et al. 2017]. This is a general-purpor solution for image-to-image translation problems based on conditional Generative Adversarial Networks (cGANs). In contrast to GAN, where these networks learn loss that tries classify the output image is real or not and simultaneously train a generative model to minimize this loss, cGANs learn a conditional generative model (Goodfellow et al.) 2014).
input + y. Oppositely conditional GANs addi- tionally feed an input image x to the generator, G xr, y. Isola et al. (2017) and [Wang Gupta (216)) suggest that in certain cases the usage can be usefully, but we decided not to include the random vector for our generator, as we want a deterministic network. Fig. iillsstaaes this principle.
As explained in Isola et al. (017), the traditional GAN method uses random vector
to the generator network to generate output y.
Parameter
Generator
Discriminator
Net
l- take?
Step
Figure 2: Illustration of the cGAN principle. The generator gets as input the image of the flow at the time step and some additional parameters like the binary map and the inflow speed.
The discriminator network is modeled with the function D : y + that evaluates the likelihood of y being a real image. To note is that the discriminator network has access to the real image æ and tries to guess, if is the real or generated output.
LOGAN (G, D) = Ellog D(=,2) v)) Ellog D(r, G(+)) log ((v, v) log D(r, G(x))
2
2
4

---

Under review as a conference paper at ICLR 2022
where (y) is the imput image (target image). The objective function is divided by two to slow down the training of the discriminator relative to the generator, as suggested in Isola al. (2017).
The objective function for the generator network is composed of two parts the value of the dis- criminator as well as a I.1 distance loss between the target and the predicted image. According to Isola et al. (2017) the L11 loss promotes less blurring and captures the low frequencies details of the images. The final objective function for the generator is thus:
G = arg min Ag max LAGAN + (G),
LGa(G) =
BBll) B G
(3)
D
For all models, used À 100 as in Isola et al. (2017).
2.2.1 1
GENERATOR NETWORK
For the generator network, we oriented as well on Isola et al. (2017). The basic structure is a U-Net (Ronneberger et al. 2013). It is a standard encoder-decoder model (Hinton & Salakhutdinov; 2006) with skip-connections between parts of the encoder and decoder. The basic approach uses blocks of layers from decoder first down-samples the input till the bottleneck layer is reached, followed by an up-sampling to the original size of the input image. In appendix in fig 12 one can see the U-Net with eight layers. The dropout layers work with a dropout rate of 50%. We also experimented with LSTM (Hochreiter & Schmidhuber] 1997) blocks between encoder and decoder to see if features extracted from more than the last step are useful.
(loffe & Szegedy (015). The encoder-
As input, the net receives tensor consisting of the concatenated images of the two velocity di- rections and the binary map of the simulation, which contains the sharp separation between the streaming area and the environment area of the image. In addition, the inflow velocity is passed. After the prediction image is decoded, the binary map is used again to perform pixel-by-pixel multiplication leading to better preservation of the streaming area (Lehmann al. 2020
2.2.2 DISCRIMINATOR NETWORK
For the discriminator, we follow the method of Isola et al. (2017) and we used their PatchGAN discriminator network. To note is that the whole image is given as an input. In our case, we decided to go with a patch size of 286 x 286 pixels, in contrast to the suggested patch size of 90 x 90 pixels Isola et al. (2017). In appendix in fig. [ one see the structure of the discriminator.
EXPERIMENTS
3.1 EXPERIMENT SETUP
To test approach, generated the training data by performing numerous simulations of in- compressible fluid flow around a rectangular object (see fig. []. In our case, we have in total three adjustable parameters- inflow speed 8in, fluid density p and fluid kinematic viscosity v. The values of the parameter were chosen to investigate the region between laminar and turbulent flow the so-called Kármán vortex street.
The simulations were performed numerically by Elmer FEM (elm 2021 a numerical solver library. To visualize the results, we used Paraview (Ahrens al., 2005) and exported grayscale images for the velocity in x- and y-direction, as well as grayscale images for the pressure field. In appendix in fig. 1 one can see images of the time steps t 25, 100, 250 with the inflow speed of sin 1.5625 Further details in appendix A.2
For training, we used 33 random picked simulations. Three of them are used for validation. Fight additional simulations are used for testing the architecture after training. With this, we have clas- sical 80-20 split for testing and training.
We used the Stochastic Gradient Decent (Kiefer & Wolfowitz 1952) with the Adam optimizer (Kingma & Ba 2017) with a learning rate of 0.0002 and standard momentum parameters 31 = 0.9 and 3g 0.999. All models were trained over 45 epochs with batch size of 3. As programming environment we used PyTorch (Paszke et al.) 2019) version 1.8.1. For the testing and training environment, refer appendix A.3 To evaluate our experiments, there
possibilities:
5

---

Under review as a conference paper at ICLR 2022
Figure 3: Performance chart of unetNoL- STM32 with respect PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iter- ation step where the mean value falls below the value of 30 dB is marked with the dotted line.
Figure 4: Predictions of uneINol.STM32 with starting frame zero and 120 at 5, 30 and 70 in comparison to the ground truth.
Single-image performance and recursive-applicatio performance. In the first case, only the next time step is calculated and evaluated with the real image. In the case of recursiv--applicatinn, the calculated result of our architecture is again used directly input of the next calculation cycle. This can be used to estimate how many future time steps can be predicted before the approach falls below the required PSNR (see chap. 2.1
3.2 RESULTS
We examined variety of different networks. The structure is briefly explained in each case, fol- lowing the principle already known from chapter [2.2 The results are then presented in each case regarding the Ligll-image-performmnde and the cuuussee-ppliccaooo-pffomaaee In the appendix A.4 you can find the images and charts in a bigger version.
3.2.1 GENERATOR WITH EIGHT LAYERS WITHOUT LSTM
This net is the starting point of all following nets. The structure follows fig. 12 with 32 feature maps.The The PSNR value is in the pangle-image-vvaan the recursive application. Since the dataset contains frames where the vortex street is fully devel- interesting whether our approach
on average 48.59 dB. More interesting is
oped, but also frames where the vortex street developing, can represent both streaming states. Therefore, we decided to evaluate the recursive approach with two different starting frames: and 120. The PSNR score rises in both cases above the desired value of 30 dB in the begimning (sce. fig. 3). The second finding is that the performance of our net is better on developing vortex streets. With start frame of O, the PSNR value holds from over 30 dB to about 49 future time steps. In contrast, with start frame of 120 this be held only for 47 time steps. In fig. + one can see the result of our net compared with the ground truth for starting frame zero and 120. A possible reason for better performance for developing vortex streets that on the right-hand side of the channel, almost no pixel transformation has to be done per recursion step. From this starting point, we tried to get better accuracy by using the extracted features of more than one iteration step thanks to an LSTM.
3.2.2 GENERATOR WITH EIGHT LAYERS WITH LSTM
In this network, a LSTM block is placed between encoder and decoder. We expect this to provide a better representation of temporal behavior. This results in a single-image PSNR 48.5 dB and slightly worse than without the LSTM. In contrast, starting the recursive-application PSNR starts at a higher value than before (see fig. 5). In the beginning, the standard deviation is smaller too. But after dropping under the line of 30 dB the standard deviation starting get bigger than the standard deviation without a LSTM. With LSTM, the value can be kept above the required value of 30 dB for 60 time steps and is thus about 11 time steps ( 22%) better than before. Starting at frame 120 we see an entirely different picture. The peak value for the first frame is lower
frame 0 the
In the meaning of generating more iteration steps above the limit of 30 d..

---

Under review as a conference paper at ICLR 2022
Figure 5: Performance chart of unetLSTM32 with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with the dotted line.
Figure 6: Predictions of unetLSTM32 with
starting frame zero and 120 at 5, 30 and 70 in comparison to the ground truth.
Statrgu TomeD
Strting Kore
17131
Figure 7: Performance chart of unetl- STM32param with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard devi- ation around the mean value (solid line). The iteration step where the mean value falls be- low the value of 30 dB is marked with the dotted line.
Figure 8: Predictions of unetLSTM32param with starting frame zero and 120 at t 5, 30 and 70 in comparison to the ground truth.
and the standard deviation is significantly higher than without the LSTM-block. Furthermore, the PSNR-score drops early under the 30 dB. As before, in fig. 6 one can see the result of this net compared to the ground truth. To keep the good performance for the starting sequence of the stream and optimize the performance for the later stages, we tried to improve the LSTM module next.
3.2.3 GENERATOR WITH FIGHT LAYERS WITH LSTM WITH ADDITIONAL INPUT TO LSTM
Since the previous LSTM has very restricted access to information, we decided give it the inflow speed of the fluid as an additional parameter. The single-imas image performance results in PSNR score of 45.8 dB. The recursive-aperr appliicatioiiiiiininnnoooo starting at frame 0 shows much worse result than the nets before (see fig. 77. Only about 17 future time steps, the PSNR score can be hold over 30 dB. But the standard deviation significantly lower than before.
Starting at frame 120 the picture is over all the same. The score can be hold over 30 dB about 19 time steps. And the standard deviation is less, too. In summary, these results are not satisfactory and sufficient. As before, in fig. 8 one can the result of this net compared to the ground truth. To note are the developing artifacts in front of the object be flowed around. As simply using LSTM for the whole iteration process my not be the right way of improving our accuracy, the next step was to build up the network structure in total.
3.2.4 GENERATOR WITH EIGHT LAYERS WITH LSTM WITH MORE FEATURE MAPS
A final open question is what happens when one invest more computing power. This was realized in the form of more feature maps. The basic structure of the architecture remained the same. However, the number of feature maps per layer was doubled. Because of the good performance in the starting phase, the LSTM was retained between the encoder and decoder in its original, not parameterized form. The charts in fig. 9 shows the result of this decision. In single image evaluation, the average

---

Under review as a conference paper at ICLR 2022
Saa of
Stafiie Yoor
Figure 9: Performance chart of unetLSTM64 with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with the dotted line.
Figure 10:
Predictions of unetLSTM64
with starting frame zero and 120 at t 5, 30 and 70 in comparison to the ground truth.
PSNR- score is 49.8 dB, which is the best score achieved in experiments. Similar results shown in the recursive application. At starting frame 0 the PSNR-score can be held over 30 db for 72 time steps and starting frame 120 for 46 steps. As before, in fig. [00 one can see the result of this net compared to the ground truth. No artifacts are developing over the time.
3.3 EVALUATION
From our results regarding the accuracy, we can conclude, that it is possible iteratively generate a sequence of streaming data images with an accuracy comparable to the image representations of the numerical solver from the perspective given cGAN approach and a U-NET structure with 64 feature maps in the first layer. In the developing phase of the Kármán vortex street, an LSTM is clearly helpful. In the later stages a parameterized LSTM might help lower the standard deviations, but in total, no LSTM seams to be the better option. Ideally, one would start the simulation using neural networks with an LSTM between encoder and decoder. After certain number of time steps, the positive influence of the LSTM decreases so that it could be helpful to then switch off the LSTM and benefit from the better performance without LSTM in already developed vortex streets.
human observer. We get the best performances with
Before we take a look into the run times, we want to give closer look over the edge of the previously given experiments to show some limits, respectively side effects, of this approach.
3.4 LOOK OVER THE EDGE
While developing a variety of different structures, variety of problems has also arisen. The biggest problem the creation of artifacts and misleading structures which are not directly visible in the PSNR. For instance, black pixels indicates a velocity of zero. Growing areas of black pixels means growing areas without a flow at all. The problem is that these artifacts are small, the PSNR stays nearly untouched. With help of the LSTM between encoder and decoder, the arising of black artifact has been greatly reduced.
We also tested our approach with values mapped to RGB color space. In this case, don't 't need the binary mask anymore because black pixels can be set as boundary or objective. But there other problems, like more artifacts, which negate this advantage. Moreover, the effort of the net structure is multiplied by around 3 because of the additional color channels.
We have also found that our database is relatively small. The Kármán vortex street is very promi- nent and intentionally always occurs in our data set, which is why an overfitting effect is probably certainly present. To counteract this, we experimented with objects both the x direction and the y direction. Thus, it is possible to double the database again by simple mirroring. However, this did not result in improved properties of the mesh. On the contrary, it led to worse results, since the information about the flow direction is lost, Thus, this led to unpredictable events by inverting the flow during a simulation.
the center of the channel in
As the numerical simulation also returns pressure field for each step, tried to use (and generate) it with our approach. We found that in some cases the usage of the pressure field as an additional input improves the accuracy in all stages, in other cases the accuracy decreases. At this moment,
8

---

Under review as a conference paper at ICLR 2022
Table 1: TCP (ToPU) is the time spend calculating 335 time steps CPU (GPU). ScPu (Sapu) is the speedup on CPU (GPU) with our method compared to the numerical method on CPU.
Method
Teru s
TorU in s
Scru
SGrU
unetNoLSSMM32
123.61
14.98
1.13
9.30
unetLSTM32
129.81
15.32
1.07
9.09
unetLSSTM32parmm
131.51
15.39
1.06
9.05
uneiLSTMM4
216.70
15.51
0.64
8.98
Elmer FEM + Paraview
42.32 + 96, 92
cannot clearly separate these cases from each other, and data can be given here. However, the accuracy of the generated pressure field doesn" 't seem to differ from the velocity field ones. Likewise, we experimented with another data with round one and further ensured that vortex street was created. Similar results were obtained with this new data set than with the other data set. This suggests that the architecture is generalizable, but it remains to be verified, however.
In this one, replaced the rectangular object
3.5 CALCULATION TIME COMPARED TO THE NUMERICAL. METHOD
In tab. one can see the comparison of the run time of our methods compared to the numerical calculation with Elmer FEM. All tests are done the in section A.3 mentioned system. The CPU versions were executed on all available processor cores. The Elmer FEM solver is optimized in this respect, which is also reflected in the runtimes. In this respect, optimized for runtime. Thus, the predicted images are written to disk directly after execution and not buffered or exported in parallel the next execution. The situation is similar with the multiplication of the mask after prediction. This is also still done in serial execution. Our method benefits of the use of GPU. As mentioned earlier, we tackle the problem as an image-to- trage-to-image approach. Thus, as a result, we have images of our flow as PNG a human observer can immediately interact with. In contrast, the Elmer FEM solver produces non-intuitive data format that must be processed by Paraview (or similar software). In the combined execution with Elmer FEM followed by Paraview, we achieve speedup of about 9 with our method using the GPU, which is excellent considering that there was focus optimizing the execution time.
approach has yet been
4 CONCLUSION
4.1 CONCLUSION
With this paper, we show an entirely new approach to numerical flow simulation. We consider the aforementioned problem an image-to-image translation task. Thus, we developed an approach to solve the problem using neural networks, since they have shown strong performance in this particular area in the past. Due to the conversion to images, direct comparison with the numerical solution is not possible, so we decided to focus on the human observer. Therefore, we set the PSNR as the evaluation metric and showed that there is lot of potential in this idea. We can answer the question "Is it possible to interpret the numerical flow simulation as an image-to-image translation?" with yes. We also showed that even not fully optimized neural networks can predict up to 72 future time steps without being a significant limitation for the human observer. Without any run time optimization, the new approach also shows significant speed-up of 9 compared to the numerical method when using the appropriate hardware resources.
4.2 FUTURE WORK
Based on the shown results, we see a lot of work for the future. A lot of the questions shown in the introduction are still unanswered. Our network architecture has a lot of optimization potential regarding the prediction accuracy. The iteration process isn't 't optimal regarding the runtime. Our test setting is very limited the moment in every dimension: geometry, structure, basic model, parameter settings, and so forth. Fortunately, our results are promising for these questions and issues, and we see great potential in presented approach.

---

Under review as a conference paper at ICLR 2022
5 STATEMENT OF REPRODUCIBILITY
For reproducibility, we tested our approach on different systems. As mentions in appendix A.3 we run it an HPC and our local server. Training, Testing and Evaluation was done for reproducibility proposes on both system. For time saving and comparability issues, we choose the mentioned setup. Furthermore, we tested it on a second server (Intel(R) Xeon(R) CPU E5-2650 with Nvidia Tesla K80) and a variety of workstations. Even slightly older hardware, the results of the approach was reproduced (e.g. Nvidia GTX 960).
REFERENCES
Elmer FEM open source multiphysical simulation software. http://www.elmeett org/ blog/ 2021. [Online; accessed July/2021).
J. Ahrens, Berk Geveci, and Charles Law. Paraview: An end-user tool for large data visualization. Visualization Handbook, 01 2005.
Jose I. Aliaga, Hartwig Anzt, Thomas Grützmacher, Enrique S. Quintana-Orti, and Andres E. Tomas. Compressed Basis GMRES on High Performance GPUs. In arXiv:2009. [cs.MS], 2020.
Hartwig Anzt, Jack Dongarra, Goran Flegar, Nicholas J. Higham, and Enrique S. Quintana-Orti. Adaptive Precision in Block-Jacobi Preconditioning for Iterative Sparse Linear System Solvers. In Concurrency and Computation: Practice and Experience, Volume 31, Issue 6, 2018.
M. Berzins and R.M. Furzeland. An adaptive theta method for the solution of stiff and nonstiff differential equations. In Applied Numerical Mathematics, Volume 9, Issue 1, 1992.
Ning Du and Hong Wang. A Fast Finite Element Method for Space-Fractions Dispersion Equations on Bounded Domains in IR.. In SIAM Journal on Scientific Computing, Volume 37, Issue 3, 2015.
O. Etzmuss, M. Keckeisen, and W. Strasser. fast finite element solution for cloth modelling. In 11th Pacific Conference on Computer Graphics and Applications, 2003. Proceedings.. 2003.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014.
Robert Gracie, Giulio Ventura, and Ted Belytschko. new fast finite element method for disloca- tions based on interior discontinuities. In Numerical Methods in Engineering, Volume 69, Issue 2, 2006.
Serge Gratton, Ehouarn Simon, David Titley-Pelquinn, and Philippe Toint. Exploiting variable precision in GMRES, In arXiv: 10550 [math.NA]. 2019.
G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313 507, 2006. ISSN 00755 doi: 10.1126/science, 1276477. URL
https: hww.ee/ww..ccecccccccooooreeeeee
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR, abs/1502.031677, 2015. URL http:://axxiv.orrtt/
abs/1502.032 03167
Phillip Isola, Jun-Yan Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. In arXiv: 1710. 10196 [cs.NE], 2017. 2017.
10

---

Under review as a conference paper at ICLR 2022
J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics, 23(3):462-4666 1952. ISSN 00034851. URL http://wwee /www.
jstor. org/ ptrg//stable/2226655
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Jari Korhonen and Junyong You. Peak signal-to-noiss ratio revisited: Is simple beautiful. In 2012
Fourth International Workshop on Quality of Multimedia Experience, 2012, pp. 37-38, 2012.
Roman Lehmann, Stanislav Arnaudov, Markus Hoffmann, and Wolfgang Karl. Binary maps for image separation in iterative neuronal network applications. In Forum Bildverarbeitung 2020. Ed.: T. Längle; M. Heizmann, pp. 363, 2020.
C. Li and M. Wand. Precomputed real-time texture synthesis with markovian generative adversarial networks. In ECCV, 2016.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli. Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier Neural Operator for Parametric Partial Differential Equations. In arXiv:2010.0889) [cs.LG), 2020.
Y. Liu, Z. Qin, Z. Luo, and H. Wang. Auto-painter Cartoon image generation from sketch by using conditional generative adversarial networks. In arXiv: 01 908 [cs. [cs.CV], 2017.
D.R. Mehra. Estimation of the image quality under different distortions. In International Journal Of Engineering And Computer Science 8, 2016.
A. A. Mohamad. Lattice Boltzmann Method Fundamentals and Engineering Applications with Computer Codes. Springer, Heidelberg/Berlin, Germany, 1st edition, 2011.
M. Nemec and D. W. Zingg. Krylon-Krylov Algorithm for Aerodynamic Design Using the Navier- Stokes Equations. In AIAA Journal, Volume 40, Number 6, 2002.
NVIDIA, Péter Vingelmann, and Frank H.P. Fitzek. Cuda, release: 11.1, 2020. URL https:
h/developer..vve wwwiiiiiii..i...i wtt.:iiwwa...occ..eaeeeeeomo www.uuuaaaaoooiioo
Olcay Oymak and Nevin Selcuk. Method-of-Lines Solution of Dependee Two-Dimensional
Navier-Stokes Equations. In Numerical Methods in Fluids, Volume 23, Issue 5, 1996.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach VeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library, 2019. URL htl.http:2 //rrx/arvvvvv/////////// www.riii..../aaaaae. wwtg//as//91../111111 cite arxiv:1912.00 1oii:1991.01708Sommmen: 12 pages, 3 figures, NeurlPS 2019.
Tobias Pfaff, Meire Fortunato, Alvaro Gonzale,, and Peter W. Battaglia. Learning Mesh- Based Simulation with Graph Networks. In arXiv:2010.03409 [cs.LG], 2021.
Alfio Quarteroni and Alberto Valli. Numerical Approximation of Partial Differential Equations. Springer, Heidelberg/Berlin, Germany, 1st edition, 2008
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In arXiv: [cs.LG), 2015.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. CoRR, abs/1555.00459, 2015. URL httt:://axxiv..rrgg/p//o// abs/
1505. 04597
M. Schäfer, S. Turek, F. Durst, E. Krause, and R. Rannacher. Benchmark Computations of Lam- inar Flow Around a Cylinder. In Hirschel E.H. (eds) Flow Simulation with High- Perforrerforancce Computers II. Notes Numerical Fluid Mechanics (NNFM), vol 48. Vieweg+ Teubner Tieweg+Teubner 1996.
Verlag,
Xiaolong Wang and Abhinav Gupta. Generative image modeling using style and structure adversar- ial networks. CoRR, abs/1 10130563333 2016. URL htl.hhtt:///aar. www.oorggg/aa....... 603 05631

---

Under review as a conference paper at ICLR 2022
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris Metaxas. Stack-gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In arX1v:6612.03242 [cs. CVJ, 2017.
Bo Zhao, Weidong Yin, Lili Meng, and Leonid Sigal. Layout2image: Image generation from layout. In International Journal of Computer Vision, Volume 128, pp. I - 18, 2020.
12

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
A.1 ARCHITECTURE DISCRIMINATOR AND GENERATOR
Structure of our discriminator and generator of our cGAN approach is shown in fig.
and 12
PICEN
LR-CN
P-CTA
C Convetioeove
Figure 11: Structure of our discriminator.
Figure 12: Structure of our U-Net with layers.
A.2 DATASET
For an insight into the data set, please refer to fig. direction as well as the pressure field are presented at the time points = 0, 25, 100, 250.
13
Here, the respective images in and y-
1120
Figure 13: Example flow at f = 0, 25, 100, 250 with Sin 1.5625
13

---

Under review as a conference paper at ICLR 2022
A.3 TEST ENVIRONMENT
Training and testing are performed on two different environments. For training, we used the oppor- tunity to calculate on pigh-perfoenance computer. One node consists of two Intel Xeon Gold 6230 CPUs and four NVIDIA Tesla V100. For testing and evaluating, we used our local server with two AMD EPYC 7F32 CPU and one Nvidia RTX A6000.
A.4 FURTHER LOOK INTO THE EVALUATION
A.4.1
GENERATOR WITH EIGITT LAYERS WITHOUT LSTM
unetNoL.57M32
AeetNoLSTMM3
Starting frame 0
Starting frame 120
40-
40-
PSVR (88)
PSWR [88)
30
20
20
10
10-
min
min
20
49
60
80
20
40
47
60
80
Frames since start
Frames since start
Figure 3: Performance chart of unetNoL with respect PSNR. The dashed lines high- lighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with the dotted line.
unetNoLS77M32 Starting at frame
uneINoLS7MM32: Starting frame 120
truth
truth
prediction
prediction
ground Iruth
ground truth
prediction
prediction
ground truth
ground
prediction
prediction
Figure 4: Predictions of unetNoL.S7M32 with starting frame zero and 120 5, 30 and 70 in comparison the ground truth.
14

---

Under review as a conference paper at ICLR 2022
A.4.2
GENERATOR WITH EIGHT LAYERS WITH LSTM
unetLSTMM2:
unetLSTM32
Starting frame 0
Starting at frame 120
50
50
40
40
PSAR (a9)
PSVR (d8)
30
30
20
20
mBK
mean
mmen
min
min
0
20
60
80
20
4042
60
80
start
Figure 5: Performance chart of unerl with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with the dotted line.
STeLLSSMM2: Starting at frame
unedl unedl STM32: Starting at frame 120
graund fruth
ground
truth
==
t=
prediction
prediction
ground
ground
1= 30
== 30
prediction
prediction
ground truhh
ground truth
= 70
prediction
prediction
Figure 6: Predictions of unetLS7M32 with starting frame zero and 120 at 5, and 70 in com- parison to the ground truth.
15

---

Under review as a conference paper at ICLR 2022
A.4.3 GENERATOR WITH EIGHT LAYERS WITH LSTM WITH ADDITIONAL INPUT TO LSTM
unetl WTep-STM2ppaaan
UTet555MM22paam
Starting at frame 0
Starting at frame 120
50
90
40
PSNR (db)
PSNR (d8)
30
30
20
20
ITEX
ITEK
mman
10
mman
min
min
0
1720 40
60
80
1920
40
60
80
Figure 7: Performance chart of unetl unetl.STM32Paree with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around the mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with dotted line.
STM33ppram Starting at frame
unet. SmeLLSTM32ppann Starting at frame 120
ground
graund
truth
truth
==
t=
prediction
prediction
ground
ground truth
1= 30
== 30
prodiction
prediction
ground tuth
ground tuth
= 70
prediction
prediction
Figure 8: Predictions of aneiLSTM32param with starting frame zero and 120 at 5, 30 and 70 in comparison the ground truth.
16

---

Under review as a conference paper at ICLR 2022
A.4.4
GENERATOR WITH EIGHT LAYERS WITH LSTM WITH MORE FEATURE MAPS
UTe
UnetLSTTM64:
Starting frame 0
Starting frame 120
50
50
40
PSNR [68]
PSNR (d8)
30
20
20
mean
mean
min
min
20
72
80
20
40
46
60
50
start
Figure 9: Performance chart of un6 with respect to PSNR. The dashed lines are highlighting the value range while the gray area shows the standard deviation around mean value (solid line). The iteration step where the mean value falls below the value of 30 dB is marked with the dotted line.
uneILSTM64: Starting frame
un6tLSTM64: Starting frame
ground
ground
truth
prediction
prediction
ground Inuth
ground Inith
= 30
t=
prediction
prediction
ground muth
ground truth
t=70
t=
prediction
prediction
Figure 10: Predictions of unetLSTM64 with starting frame zero and 120 at = 5, 30 and 70 in comparison the ground truth.
17


Reviews:
[{"review_id": "vgfkm8cvhfJ", "reviewer": "Reviewer_BH97", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of fluid simulation as an image-to-image translation task is really interesting. The authors put in considerable work evaluating various architectures for this specific task, and come up with very nice looking fluid flow predictions that score well on PSNR.\n\nInteresting the idea may be, I think the execution of the idea needs some work. The paper is missing important related work, relevant comparisons and baselines, and information about the architecture (although this info can be inferred from the supplementary material). Moreover, the experiments are conducted on a small dataset, are likely to be overfitting, and are unfair to the selected baseline. The writing in the paper is also poor. \n\nMy first concern regards baselines. FEM and LBM are not the only numerical solvers for fluid flow. There has been high demand for fast and only mildly accurate fluid simulations for a long time, mainly from video games and graphics. This body of work is missing from the related works section, and in my opinion from the comparisons and baselines of the experiments in this paper. \"Real-Time Fluid Dynamics for Games\" for example is a 2003 paper describing a simple fluid dynamics solver that is capable of producing Kelvin-Helmholtz and Rayleigh-Taylor instabilities, in addition to Karman vortex sheets, in real-time on a 2000's CPU. Given that these \"fast and only mildly accurate\" methods exist, I feel the authors must justify why using a neural network to simulate fluids would be useful or interesting. Are they faster and more accurate than existing methods? Are ouputs more realistic looking?\n\nThe experiments in the paper are also poorly described and executed. The architecture of the U-net is described in detail, yet I could not find a detailed description of how an LSTM was combined with the U-net and the GAN setup, and how exactly the inflow velocity was given to the U-net. I also could not find info on the training and test data, except that there were 33 simulations, split 80-20, exclusively of the vortex shedding phenomena. With such a small dataset size, there is likely overfitting occuring (a possibility described but not addressed by the authors: \"overfitting effect is probably certainly present\" in section 3.4). I am further concerned that overfitting is occurring because I do not believe the network is ever given enough information to fully predict fluid flow perfectly. The Navier-Stokes equations require both pressure and velocity to be known globally, yet the network is only given pressure and local inflow velocity. The trained network may simply be a \"Karman vortex sheet simulator.\"\n\nFinally, the results of the experiments are lacking nuance. The authors claim their method trades off accuracy for speed, yet there is no explicit analysis of this tradeoff. The authors compare against a single FEM solver, which is slower but more accurate than the proposed neural network approach, but it is not established quantitatively how much more accurate. The baseline itself is rather unfair as well, as the method they compare against is actually a multiphysics tool which can solve general BVPs from electrodynamics, fluid dynamics, heat transfer, and acoustics, whereas the proposed method only works to simulate Karman vortex sheets for very specific boundary conditions. \n\nAdditional Questions:\n\n- What is the LSTM architecture?\n- How is the inflow velocity given to the network?\n- How large is the dataset?\n- Can we be sure there's no overfitting?\n- What is the \"approximation property\"? (sec 1.1)\n- What is the \"false streaming data\"? (sec. 2.1)\n\nAdditional Feedback\n\n- Including the image processing time of paraview seems unfair\n- PSNR likely needs no elaboration in sec. 2.1, as it is a widely used metric\n- In section 2.2, last sentence of second paragraph seems to imply cGANs are very different from GANs.\n- There is, in my opinion, a disproportionately large emphasis on the architecture used (number of features, layers, LSTMs). I think just presenting the best results could be sufficient\n\n- The writing is sort of poor at points in the paper. As a result, it's sometimes hard to understand the ideas being communicated. The paper will probably benefit greatly from a round of revision and editing.\n- Various typos/poor writing:\n    - \"manly\" -> \"mainly\" (sec. 1.2)\n    - Many different spellings of u-net (\"u-net,\" \"UNET,\" \"U-NET\")\n    - eqn 3 typo \"A_G\"\n    - \"Karman vortex street\" (sec 3.1)", "summary_of_the_paper": "The authors pose fluid simulation as an image-to-image translation task. From this perspective, approximating fluid flow using a cGAN can potentially improve speed (at the expense of accuracy) over FEM methods. This can be useful in situations where accuracy can be traded off for speed (e.g. video games). The authors show a U-net can produce good future predictions for Karman vortex sheets, and is faster than an off the shelf multiphysics simulator. In addition the authors ablate various architectural design choices.", "summary_of_the_review": "The idea is interesting, but the execution of the paper is poor. The experiments lack rigor, there is likely massive overfitting, and the baselines are incomplete and weak. The writing in the paper could also use some work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "sULyuLWs2dK", "reviewer": "Reviewer_9rKJ", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposal to treat simulation and visualization jointly by considering the whole problem in image space is a novel aspect of the paper, which the authors clearly note. However, this is never motivated well and reading the paper, I was not convinced that it makes sense. In fact, the authors themselves note clear drawbacks of this approach (\"the very limited resolution of the color spaces alone results in high errors\"), and other motivating factors are unclear (\"the Elmer FEM solver produces a non-intuitive data format that must be processed by Paraview\"; what is non-intuitive about it? what about simulator frameworks that have an integrated visualization component?)\n\nThe paper does not cite or compare to relevant prior work. There is now a large body of literature using neural networks as a replacement for or extension of fluid simulators, including works utilizing GANs and various convolutional architectures and black-box approaches attempting to model the flow field without relying on knowledge of the underlying physics (e.g. doi:10.1109/ICTAI50040.2020.00057, doi:10.1017/jfm.2019.700), This appears to have been completely ignored here. Papers investigating simulation accuracy from the point of view of a human observer (e.g. https://arxiv.org/abs/1907.04179) are also not discussed.\n\nThe authors state that a motivating factor for their project is speed, but the proposed approach does not really deliver on this front. In Table 1, CPU cost of inference (not taking training into account) is always comparable to a FEM solver, and GPU speedup is not particularly relevant due to the lack of a GPU-accelerated FEM baseline. The authors also state that \"there is no mathematical way to decrease this correctness in cases where only approximate predictions are needed.\". It is unclear what counts as \"mathematical\" in this context, but this seems too broad a statement, and approaches like subgrid models or adaptive mesh refinement could be discussed in this context. \n\nThe paper contains very little discussion about generalization -- there is just a brief mention of experiments with flow around a cylinder instead of a rectangle, but no further details. Insufficient information about the training and testing set makes it impossible to say how well the model works e.g. as a function of the Reynolds number. In section 3.4, variations in the placement of the rectangular obstacle are discussed, but again no details are provided.\n\nTechnical comments:\n- \"streaming data\" is mentioned multiple times in the text, but it's unclear what this means. Is it the velocity field?\n- The text mentions \"33 random-picked simulations\" for training. How were the parameters varied? Which Reynolds numbers were covered?\n- The details of the FEM simulation are missing. What mesh resolution & type and time step size were used?\n", "summary_of_the_paper": "The paper casts the problem of 2d fluid flow simulation as an image to image-to-translation task. A cGAN with standard architectures (U-net generator, PatchGAN discriminator) is trained to advance the visualization of the simulation to the next time step, and an extension with an LSTM block is explored. The model is evaluated in an autoregressive setting on the problem of fluid flow around a rectangle, and the results are evaluated with image metrics (PSNR).", "summary_of_the_review": "The lack of comparison to prior work, the usage of non-standard metrics, and lack of novelty (beyond the general framing, which I found poorly motivated) make it impossible for me to recommend acceptance of this paper in the present form.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "FtTQXI0sMGZ", "reviewer": "Reviewer_1Jbg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Although the authors reformed the problem as image translation, using a cGAN (instead of GAN, or other architecture) to generate PDE's result seems to be a trivial idea. Indeed, explicitly declaring the problem as image translation could be a new perspective. Still, it did not provide many novel insights, nor did it help derive a novel machine learning method that outperforms the practices in other related works. In addition, there are other critical issues in the paper.\n\nThe paper is poorly written with fancy words. In the abstract, the authors claimed, \"we developed an approach based on Neural Networks that is much less time-consuming but nearly as accurate as the numerical model for a human observer.\" What kind of human observer? Someone with sharp eyes? On a monitor or a piece of paper? Such statements are vague and erratic in the context of solving a PDE.\n\nThe PSNR is just a new form of loss, and using a PSNR as the loss is irrelevant to other fancy claims (e.g., human observers). The paper also didn't provide a convincing reason that PSNR is more reliable than different loss formulations. \n\nBesides, the method is poorly validated. The authors only test their approach on a single example with a fixed resolution, showing no comparison with other state-of-the-art methods. Hence I am not convinced that the method in this paper is superior to any other related works in solving PDEs. In the extent of \"a human observer\", the authors didn't show any user studies to justify their result is indiscernible to sharp eyes. \n", "summary_of_the_paper": "This paper reform the task to solve a PDE into an image translation problem and used cGAN to solve the PDE.", "summary_of_the_review": "I am not convinced that this paper is contributive (against the bar of ICLR) to either the machine learning or numerical analysis communities. Hence I would oppose its acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "9uFwLXkbhey", "reviewer": "Reviewer_jvzR", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper uses the cGAN method in [Isola et al 2017] on image sequences generated by a fluid solver. These images correspond to 8-bit discretizations of the velocity and pressure fields.\nThere is no novelty in the method (this is 1:1 reimplementation of Isola et al), nor the task (Karman vortex street is a standard benchmark task for both classical, and ML-based physics prediction papers).\nIt's unclear to me what benefits an 'image-based' approach would bring. There are lots of papers modeling flow fields on regular grids with CNNs (e.g. de Bezenac \"Deep Learning for Physical Processes\" or Thuerey \"Deep learning methods for Reynolds-averaged Navier–Stokes simulations\"). The only difference in the encoding is that these papers don't discretize the flow fields to 8-bit values.\nThe paper does not contain any baseline comparisons to other methods, or further analysis.", "summary_of_the_paper": "The paper uses a cGAN to model the fluid flow of a Karman vortex street in image-space. This is intended as a more efficient method for creating 'visual' simulations compared to classical numerical simulation.", "summary_of_the_review": "This paper uses a standard implementation of a cGAN on a standard benchmark task for physics prediction. There is no significant innovation, comparison to any of the many other ML approaches solving this problem, or interesting analysis and discussion.\nI don't think this paper brings any new insights to the community.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper formulates fluid simulation as an image to image prediction task and proposes to solve the problem using a cGAN formulation. The objective is to develop fast approximate solutions for the modeling of fluid dynamics, here Navier Stokes for incompressible flows. The images correspond to the discretization of velocity and pressure fields. Experiments are performed on a simulation for a Karman vortex street. 

All the reviewers expressed concerns w.r.t. the absence of references and comparisons with closely related work in the recent but abundant literature on NN for modeling PDE dynamics, the lack of novelty and the insufficient experimental design, description and discussion.
Decision: Reject