Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper shows a method that combines a convolutional neural network with a multi-scale physical computational fluid dynamics (CFD), in the scope of predicting turbulent flows. The authors proposed a new network, TF-Net, that is based on a multi-scale CFD: a temporal and a spatial filters are learned, prior to 3 separate 'encoder' networks, that are then grouped in a unique 'decoder' part. The method is tested on a synthetic example, showing interesting results and comparing with a large set of baselines.

Overall, I found the paper interesting in that it really starts from modeling and CFD aspects, in order to derive a neural network architecture. However, I am a not totally convinced since (i) the authors said that their goal is to emulate numerical simulations given 'noiseless observations': how can this be used on a real case then? and (ii) the developed (complex) architecture is still not sufficient to constrain the physical problem, as they had to manually include a regularization term (on the divergence) in the loss: what is the purpose of the 'physically-based network' then, and how can this method be extended to other dynamics?

The paper is quite well written, the method compared with a lot of state-of-the-art methods, and performs well on this noise-free problem.

Remarks/ questions:
- In this paper, we suppose that the equations are known. What if it is not the case (at least partially)? Would it still be applicable? In the reality, there are other components of the dynamics that might be not fully understood, some noise, ect...
- As said previously, and also as you mentioned in the introduction, the literature 'focused [...] on regularization being ad-hoc and difficult to tune'. But in the end, you also need a regularization term in your loss to prevent a non-zero divergence. So what is different then?
- The total size of the image is a rectangle composed of 7 square sub-regions, each of them being used separately as the input of the network. But in the result, we do see clearly some boundary effects. Would it be possible to also learn the 'intermediate' squares, in between the n. 1 and the n. 2 for example, and then to reconstruct the full picture with less boundary effects? Otherwise, I see a limitation: since your work is based on known equations without noise, the goal is only the speedup of the computation. But if the results are too degraded, I don't see how it can be used.
- Do you think that your model is able to learn the underlying physics? If so, do we have access to the latent variables, that would be the state of your system (see Learning Dynamical Systems from Partial Observations, I. Ayed, https://arxiv.org/pdf/1902.11136.pdf)? Such as p and T. 
- Similarly, is there a way to understand each component of your network? Such as (i) represent the spatial and temporal filters learned; (ii) understand the 3 U-nets: to what inputs do they respond, what are their respective outputs (before the 'decoder')? Is it doing what we wanted, are they all useful?
- Since you are not using recurrent networks, why don't you automatically predict at 60 frames from the start? 

Small remarks:
- 'quantifies': few times, did not not mean 'quantities'?
- 'N. Thuery (2019) also found the U-net architecture is quite' -> a 'that' is missing
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: ## Summary

The authors propose use learned spatio-temporal filtering and a convolutional model to predict the behavior of a turbulent fluid flow. Turbulent flow is a very difficult problem, of great interest for engineers and physical scientists, so the topic of the paper is certainly compelling.

This paper has several significant issues. The baselines the authors compare to are quite weak. Many of them were designed for other purposes, such as video prediction. The authors claim significant improvements ("64.2% reduction in divergence") that I believe are calculated in a (unintentionally) misleading way. In fact, I think they missed the most important baseline---the ground truth simulation itself. Unless they can argue that the learned model is superior to the classical simulation in a significant way, it is hard to see the benefit of using something like TF-NET.

## Specific Comments

* Page 2: The claim "64.2% reduction in flow divergence, compared to the best baseline" seems misleading. In Figure 5, the constrained TF-NET is as low as ~590 but the ResNet is between ~610 and ~810. I am guessing that the authors meant a 64.2% reduction in *difference* from the Target model, but this should be clarified.
* Page 4: Is 'T' supposed to appear in the denominator of Equation 3? I would have expected this to be a normalizing factor resulting from integrating the filter G over the whole domain.
* Page 4: I would expect filters used for LES are symmetric. Are any symmetry requirements being enforced on the learned filters?
* Page 4: It's not clear what TF-NET is outputting. I would expect it to output the time derivative of the velocity field, but this should be spelled out explicitly.
* Page 5: Since you are using incompressible Navier-Stokes equations, I don't think the Mach number is relevant. IIUC this is only relevant for the propogation of shock waves in compressible flows.
* Page 7: It looks as if the Target model has significant divergence. Why is this? Should we expect this much divergence in the ground truth data?
* Page 8: In homogeneous isotropic turbulent flow, the energy spectrum is governed by Kolmogorov, so we know what the spectrum should look like. Is there an analytic result for RB convection? If so, could you include that on the plot?
* Page 8: The lines for U-Net and TF-NET are nearly indistinguishable in Figure 7. Could you change them?
* Page 8: It seems like it would be worth including the ResNet in the energy spectrum plots as well.
* What did the learned spatial and temporal filters look like? How do they compare to typical 'hand-chosen' filters?
* IIUC, the training, validation and test data all have identical Rayleigh number. Does the learned model generalize to higher/lower energy? This seems critical to making this sort of model useful.
* The paper doesn't describe a tuning process for any of the models. I would expect this to lead to significant improvement. In particular, how do the models' performance change as the weight on the divergence loss term is increased?

### Baselines
* I think the objective should be to show that TF-NET is superior to the ground-truth method in some way. Can it match the results of the ground truth simulator but do it faster, or with fewer resources?
* The authors say "we compare our model with a series of state-of-the-art baselines for turbulent flow prediction." However, most of these models are intended for video prediction or other tasks unrelated to turbulent flows. I wouldn't expect any of the baselines considered to do perform well in this context.
* Since TF-NET gets the benefit of a loss related to divergence, I would have expected the other architectures to get the same treatment. In fact, without this extra loss term, the ResNet architecture is competitive with TF-NET.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This work targets learning flow fields for two-dimensional Rayleigh-Benard convections inspired by hybrid RANS-LES turbulence models. 

Internally, the approach employs three U-nets which are each trained to predict a 32 dimensional feature space from three variants of filtered velocity fields. These three feature spaces are (probably) concatenated and translated to an output velocity via a fourth U-net. While the overall architecture is intuitive, details are missing in the text. 

I also assume that the network outputs w for timestep t+1, which is the sole input for the next evaluation of the network? A regular flow solver is not used in conjunction with this network, but the results (i.e. sequences) shown are purely inferred by the network? (I guess G2 uses multiple frames - what is used for T here, btw.?)

These details, plus specifics of each layer should be written out (e.g. in the appendix), together with operations such as the merging of the three encoder outputs to make the work reproduicble. I hope the authors can also clarify these points in the rebuttal. 

For a future version, I'd also recommend to rephrase equations (2) and (3). The split into w, w-bar and w-tilde is not compatible with figure 2. The figure uses the split from equations (6,7), so it would be good to make this clear via the notation.

As training data, the model uses single RBC data set produced with a lattice boltzmann method. It's a pity only a single case is shown, as the method claims to learn a general turbulence model. Do the authors have a second data set on which they could demonstrate the method? This would show help to show generality of the approach.

The RBC test case is used to train a nice range of different methods, from a simple ResNet to approaches from previous work, and the results are evaluated with a good range of turbulence metrics. These evaluations show nice improvements, e.g., the RMSEs over time in figure 4 are consistently lower. Unfortunately, it's not made clear which data is evaluated - is this a single training case, or e.g. averaged for the whole test set? Likewise for figure 6 and 7.

Very minor, but I'd recommend to rephrase the last sentence of the fluid animation discussion. Those works are part of computer science, which arguably also counts as science.

Overall, I found the split into temporally and spatially filtered components of the flow field is an interesting one, and it's nice to see how well this seems to work. The paper certainly does not aim for new insight for deep learning methods in general, but provides an interesting application for turbulent flows that is evaluated with a nice amount of detail. If the authors can address the unclear points mentioned above, and maybe include a second test case that is evaluated on a subset of the different models, I think this paper could be included in the ICLR program.

review_title: Official Blind Review #3

Meta-review: The reviewers all agree that this is an interesting paper with good results. The authors' rebuttal response was very helpful. However, given the competitiveness of the submissions this year, the submission did not make it. We encourage the authors to resubmit the work including the new results obtained during the rebuttal.
Decision: Reject