Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper focuses on the problem of neural surrogates of high-resolution PDEs. To this end, it applies the principles of multi-grid domain decomposition to exploit the local structure in such data as well applies tensor factorization with low-rank regularaization to reduce the number of parameters. The method is evaluated on vorticity form of Navier-Stokes equations and shown to improve performance. Although it's unclear what was the resolution of data used in experiments.
strength_and_weaknesses: **Strengths**

Given GPU memory bottlenecks, different compute-memory tradeoff methods need to be explored as we keep increasing data sizes in deep learning. FNO based methods scale even worse with respect to memory when number of tracked modes increases. Proposed approach based on tensor decomposition methods makes sense for lower rank representation of model weights. 

**Weaknesses**

Whether this is _hardware-efficient_ is not explored in this work. Something like Monarch [1] is the appropriate comparison for modern hardware. Matching derivatives during training makes the method even more expensive. Some timing comparisons would be useful.
Using % for reporting test error makes it harder to understand whether the underlying errors should be considered good. Similarly it's unclear what is "high" resolution in the paper because the experiments don't mention the spatial or temporal resolution. Moreover it would be useful to actually clarify if an external PDE solver was used or if the solver was hand written for datagen. 

Some of the claims about "deep learning models being orders of magnitude faster than conventional solvers" are based on incorrect comparisons; running on GPUs vs. CPUs, not using state-of-the-art PDE solvers rather comparing against hand written solvers, not taking into account approximation errors etc. 

It's also a bit surprising that older neural operator methods like DeepONets [2, 3] are missing from related works.

[1] https://arxiv.org/abs/2204.00595
[2] https://arxiv.org/abs/1910.03193
[3] https://arxiv.org/abs/2111.05512

clarity,_quality,_novelty_and_reproducibility: Many details are missing from the paper including the resolution of data used in the experiments. But many details can be more clear from the code and data release. The paper otherwise is easy to follow. The idea of using lower ranked factorization with alternative decomposition is common but specific applications to FFT based architectures is less explored and novel.
summary_of_the_review: There are important details missing from the experiments in the paper to make a clear judgement but the idea of using lower ranked tensor factorization and domain decomposition  makes sense for the PDE domain.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a neural operator that scales to large resolutions by leveraging local and global structures through decomposition of both the input domain and the operator’s parameter space.
This is achieved by writing an FNO operation with a single tensor, and subsequently using low-rank tensor factorization. 

strength_and_weaknesses: Strengths:
-	The paper presents a very interesting approach to a well-known problem: how to scale neural PDE surrogates efficiently to large resolutions
-	Writing the FNO basic block as single parameter tensor, and then applying matrix factorization on that is really neat. I think the idea has great potential.
-	The multi-grid domain decomposition makes a lot of sense too.  


Weakness:
-	The biggest concern are the experiments. I cannot find any information on the grid, and any ablation on the baseline method. I could very well be that FNO with much less modes would perform equally well, which would make the achieved compression rates obsolete. It is really hard to judge the results without knowing the grid sizes and having some visual image of the data. Looking at Figure 3, the problems however don’t really seem to have a very high resolution. What is the temporal resolution? Which solver is used to obtain the data? What are the boundary conditions, stating Reynolds numbers and not the boundary condition is problematic. 
-	There is no mentioning of other operator learning methods (e.g. DeepONet from Lu et al.) as well as a discussion of many other neural PDE surrogate methods. There is a huge body of neural PDE solvers/surrogates and omitting 95% of that is not good practice. I however admit that not everything is relevant for this paper. DeepOnet, Adaptive Fourier Neural Operators, multigrid methods, Vision Transformers are.	    
-	Connecting to the previous point: I am also very curious how this relates to Vision Transformers or Adaptive Fourier Neural Operators. Also recently UNets get tremendous performances in high-resolution image generation tasks, how are they doing, or even more interestingly can this approach be applied to e.g. UNets/DeepONets as well?
-	What do I see in Figure 5 right? What is H1 loss there? Both y axes state L2 loss? That fact that there is such a gap between FNO train vs test can very well be due to the rather basic training scheme (no warmup, lr annealing can surely be improved) or can be a sign that better regularization for the baseline is needed.

Lu, Lu, Pengzhan Jin, and George Em Karniadakis. "Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators." arXiv preprint arXiv:1910.03193 (2019).

clarity,_quality,_novelty_and_reproducibility: The paper is written clearly, up to the experiment section. There very interesting details are missing, which make it really hard to judge the impact. In the current version, the experiments are not reproducible
summary_of_the_review: The paper presents a strong idea, which is also practically very relevant. I highly encourage the authors to work on the experiment section and to work out the benefits of the methods as well as relation to other methods in more detail.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to predict solutions of PDE locally and unite them into a global solution. The proposed neural operator scales to large resolutions by leveraging local and global structures by decomposing both the input domain and the operator's parameter space. The proposed method can be trained with much fewer samples than previous approaches, outperforming the FNO when trained with just half the samples. 
strength_and_weaknesses: Strength

MG-TFNO outperforms the FNO with a fraction of the parameters and memory complexity requirements.

Figure 1 clearly illustrates the structure of the network.

Weaknesses

How to handle the Fourier transform of non-periodic problems?

Will the computational cost of the Fourier transform be large?

How is the differentiation of the Fourier transform implemented?

Can this method make a 3D prediction?

clarity,_quality,_novelty_and_reproducibility: The proposed method involves the Fourier transform, which can be complicated.
summary_of_the_review: The paper proposes a new approach but solves only a few simple problems. There are already many similar methods, and it is difficult to say which is better and which is more practical. Studying large-scale examples in three dimensions may be more critical.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The focus of this paper is on using neural networks to predict solutions to partial differential equations (PDEs) locally and combining these predictions into a global solution. The proposed method, called the neural operator, is able to handle large resolutions by taking advantage of both local and global structures in the input domain and the operator's parameter space. It is also able to achieve better performance than previous methods with significantly fewer training samples, outperforming the FNO when trained with only half as many samples.

The reviewers raised concerns on various aspects of the paper, and expected the authors to upload a revised manuscript to address (or partially address their concerns). Unfortunately, the authors did not make such a revision, and the authors' rebuttal was insufficient for them to raise scores. Moreover, Reviewer zgdt has the concern on no significant improvement over existing methods.
justification_for_why_not_higher_score: NA
justification_for_why_not_lower_score: NA
Decision: Reject