Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This is an interesting paper that seeks to propose novel machine learning methods to extract information within continuous and infinite fidelities to bolster the prediction accuracy. The key novelty here is the means to develop a infinite fidelity coregionalization method by introducing a low-dimensional latent output and multiply it with a basis matrix for solution output prediction. The experimental data is comprehensive. Overall, the work is well done and provides some new methodology to this field.
strengths_and_weaknesses: Strength:
A novel infinite fidelity coregionalization method is proposed, which improves upon existed models with finite and discrete fidelities. The paper is well-written. Empirical results on different types of PDE problems are provided.

Weakness:
One of the main advantage of multi-fidelity models is are the improve learning and sampling efficiencies. Therefore, besides the comparison on accuracy, it would be helpful to see the training time comparison for at least 1 of the problems. It would also help the readers to assess the method if the authors can present a comparison when using different amount of training data.
questions: 1. The interpolation and extrapolation study was only provided on Poisson's and Heat equations, which are problems with relatively smooth solutions and this is possibly why the model can extrapolate to even high fidelities. Can the authors perform interpolation and extrapolation studies on Burger's equation and CFD examples?
2. What is the Reynolds number in the CFD example? How does the algorithm perform on high Reynolds number (such as transient flow and turbulence flow) regimes?
3. The authors have commented that a shallow NN (2 hidden layers) is sufficient. Does the number of required layers, as well as other hyperparameters, vary for different examples?
limitations: The discussions on limitations are adequate.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose two novel methods for coregionalization, i.e., projecting multiple grid resolutions onto a common grid, similar to superresolution. While previous works assume a discrete number of resolutions, the proposed work contributes the first coregionalization with a *continuous* change in resolution. The methods combine neural-ODEs and GPs to project low-resolution inputs, parameters, and BCs onto a higher-resolution grid. The main idea is by training a neural ODE to interpolate the meshes, the NODE can learn from other meshes in the latent sstate and outperform discrete methods that learn one model per mesh. The authors provide background, methodology, and support the claims with extensive empirical results. The empirical results confirm that the proposed methods, IFC-ODE and IFC-GPODE, both outperform discrete methods for coregionalization.

strengths_and_weaknesses: 1. Strength:
1.1 The broader research topic of ML-based surrogate modeling of PDEs is significant to computational fluid dynamics, climate modeling, chemistry, biology, etc. The narrow topic of coregionalization, i.e., projecting data from various resolutions to a common grid, or learning from data of various resolutions is relevant to practical settings. The more narrow topic of infinite resolutions would allow for higher flexibility in using the method in practice.   

1.2 The authors provide extensive empirical results on five problem settings and compare to five relevant methods. While the comparative methods could have been selected broader, e.g., including GAN-, Flow-, or Diffusion-based superresolution, the selection seems sufficient. The empirical results support the claims of the paper.

1.3 The author choose advanced methodology to handle GPs in high-dimensional settings. 

2. Weaknesses:
2.1 It is still a bit unclear what exactly the contributions are if Zhe et al., 19 and Li et al., 21 have already provided scalable GPs for regression of latent outputs and bases. The authors acknowledge this in L218. I am assuming that the paper is the first paper to do learning-based coregionalization with continuous fidelities. Section 4 needs to clarify that matrix Gaussian distribution is taken from a different paper and only applied to coregionalization, here, as far as I understood. The clarity of the paper could be improved by adding a 'list of contributions' at the end of the intro.  

2.2 The related works section is very detailed with respect to the most similar works. The related works section could be improved by mentioning a practical use-case of coregionalization, and deterministic and stochastic superresolution methods with GANs, Flows, or diffusion-based methods. 

2.3 The method is quite complicated as it mixes matrix GPs and neural ODEs. I am not sure if introducing active learning to the mix would make for a very interesting paper as it might become really complicated to use this method in practice. 

questions: 3.1 What are the stochastic variables, parameters, ICs, or BCs that vary in between samples in train and test dataset and what are their distributions? It remained unclear to me whether the proposed work is an emulator of a single PDE solution at different resolutions or an emulator of the PDE solver that works at different resolutions and parameters, ICs, BCs. L249 indicates that the proposed work emulated a PDE solver, but I would need confirmation by the authors.

3.2 Can the authors add a complete in-/output diagram to the camera-ready version? It is unclear to me if the state with lowest- or 1-lower fidelity is used as input. The quality of the results would positively surprise me if, e.g., in the case of Navier Stokes, an 8x8 solution is the input and the output is an accurate 64x64 solution.  

3.3 What are the assumptions on the grid that are being made?

limitations: 4.1 The authors do not address limitations or potential negative social imapcts. 

4.2 It is not mentioned on which kind of grids or finite element topologies the proposed model would work. I am assuming that it would only work for equispaced grids. 

4.3 It would be helpful to explain the implications of Gaussian assumptions during modeling.

4.4 The authors claim that "the surrogate model [is trained] only using low-fidelity examples, but we can still expect to obtain high-fidelity predictions, i.e., more accurate than the training data" [L389-391]. It seems to me that there is not sufficient evidence to support this statement and I would alter or explain it for a camera-ready version. I do not fully understand how the proposed model could create, for example, higher-order frequencies that have been seen in the training phase.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduced IFC, an infinite-fidelity coregionalization method for physical simulation. They designed ODE-based modeling to capture information for continuous fidelities and combined it with  Gaussian processes or another ODE to estimate the fidelity-varying bases. The result shows their method outperforms the baselines among several benchmark prediction tasks at the highest fidelity level.
strengths_and_weaknesses: Strengths:
1. Well-written paper in general.
2. the experiment is extensive. Five benchmark study included. 
3. The performance of the proposed model seems good.

Weaknesses:
1. I disagree with the statement "in practice, the fidelity choice is often continuous and infinite". For all five experiments in the paper, the data has finite and discrete fidelities. In practice, people pre-generate the simulation data. The fidelity level is also pre-selected, which is also finite and discrete. Although the performance of the proposed method is good.
2. The definition of fidelity $m$ seems confusing and not consistent. In the background, $m$ looks like a discrete value 1,2,3,.... but starting from the model section, $m$ becomes a continuous value. There's no explanation of how to map a fidelity level to $m$ value. 
3. Only one evaluation metric nRMSE is included. 

questions: 1. What's the definition of fidelity $m$ (continuous case)? 
2. Figure 4 result is interesting, especially the extrapolation part. It will be great if you can add similar figures for the rest three experiments to see whether this trend happens in general.
3. I don't see the case that the dataset has continuous and infinite fidelity. But your method considering continuous and infinite fidelity improves the performance. It will be great if you can explain more about the reasons.
4. baselines are missed [1, 2, 3]. Including them will make the experiment results more strong.

[1] Wang, Yating, and Guang Lin. "MFPC-Net: Multi-fidelity Physics-Constrained Neural Process." arXiv preprint arXiv:2010.01378 (2020).
[2] Wu, Dongxia, et al. "Multi-fidelity Hierarchical Neural Processes." arXiv preprint arXiv:2206.04872 (2022).
[3] Xing, Wei W., et al. "Residual Gaussian process: A tractable nonparametric Bayesian emulator for multi-fidelity simulations." Applied Mathematical Modelling 97 (2021): 36-56.

5. It will be great if the author could include other metrics for accuracy (negative log-likelihood) and uncertainty quantification (Continuous Ranked Probability Score [4] or mean interval score [4,5]).

[4] Gneiting, Tilmann, and Adrian E. Raftery. "Strictly proper scoring rules, prediction, and estimation." Journal of the American statistical Association 102.477 (2007): 359-378. 
[5] Wu, Dongxia, et al. "Quantifying uncertainty in deep spatiotemporal forecasting." arXiv preprint arXiv:2105.11982 (2021).

6. Does the proposed model support varying input and output dimensions at different fidelity levels?
7. Does the proposed model support non-subset multi-fidelity data?



limitations: No potential negative societal impact I can see.
review_title: null

Meta-review: metareview: The paper tackles the multi-fidelity simulation problem by modeling the grid variation with NODE, coupled with a GP. Experiments on multiple physical simulators show better performance compared to baselines. Please also report computational efficiency and sample complexity in the final version.  
confidence: Less certain
Decision: Accept