Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content:  The paper proposes to modular PDE solutions in networks into spatial and temporal modules, where spatial modules are rapidly adapted by meta learning.

Strengths

This paper is clear and relatively easy to read.

The paper does a reasonable of citing related work and the approach seems novel  (although incremental) to my knowledge.


Weaknesses

The overall quantitative benefits of the approach appear to be low. For example in table 2 or table 3, many of confidence intervals overlap. Furthermore, these results are shown in the best setting of the model -- when nearby tasks are drawn almost directly from the same training distribution. Furthermore

The overall technical novelty of approach seems to be low. Modularity with meta learning has been explored before, and the physical learning framework has also been learned before. This work seeks to combine both but does not seem have to have impressive performance. 

Given that a central claim in the paper is that "We extract shareable parameters in the spatial
modules from synthetic data, which can be generated from different dynamics easily." I think it would be good to empirically test this result and  synthetic data is generated more diversely. For example, in Table 1/2, what happens when the underlying uis have different functional forms? What if only the test task has a separate functional form?

There are only three evaluated datasets in the entire paper -- one being a toy problem to visualize the impact of the approach. Since PDEs occur in so many different disciplines, it would be good to evaluate on additional datasets to more throughly validate the utility of this approach. For example, it may also be interesting to test this on additional non graph based settings.

### Post Rebuttal Update

I thank the authors for their response. However, I still have some remaining concerns about novelty, since to me this paper reads as another application of MAML to domain X, to speed up performance in the particular domain. Furthermore, I think it would be good to evaluate generalization to larger extent, for example on different conformations of dynamics as opposed to a fixed set of parameters on the synthetic dataset.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: The key contribution of the paper is not very known to reviewer. Is it employing the PDE-independent knowledge (spatial derivatives) from simulating the PDEs?
Can the authors highlight some of the key contributions of the paper?


review_title: The authors provide a framework for physics-aware meta-learning with auxiliary tasks. 

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a framework for physics-aware meta-learning to tackle the few-shot learning
challenges in physical observations. The authors claim that by incorporating PDE-independent
knowledge from simulated data, the framework provides reusable features to meta-test tasks with
a limited amount of data. The experiments on synthetic and real-world spatiotemporal prediction
tasks demonstrate the effectiveness of the framework. However, there still exist some limitations
to the approach.
Pros:
1. The idea of using meta-learning for modeling physics-related spatiotemporal dynamics is
novel.
2. The paper is well written, and the idea is presented clearly.
Cons:
1. How to choose and construct auxiliary tasks based on synthetic datasets is important for
the performance but there is no systematic approach to addressing such an issue.
2. The authors can add more sensitivity analysis to show how different choices of auxiliary
tasks can influence the experiment performance.
3. The paper focuses on modeling and predicting sensor-based observation in 2-dimension.
However, it is not clear if such an approach can be effective in 3 or higher dimension
space. Higher-dimension space and higher-order derivative can bring up challenges to
constructing useful auxiliary tasks.
review_title: This paper is well written. The idea of physics-awareness in meta learning seems novel and clear. However, some limitations in the approach remain to be improved.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper describes the approach to meta-learning of spatiotemporal predictions for sparse data with auxiliary spatial derivative modules. 

The paper provides novel insights on how to perform physically informed meta-learning and can be useful to the community, but the questions below need to be answered to assure that it is indeed meta-initialisation which improves the few-shot learning performance (the review rating reflects these questions, especially question 2 on the motivation of pre-training, and could be improved based on the answers):
1. It is said that *“While it has been shown that the data-driven approximation of spatial derivatives is more precise than that of finite difference method”*. At the same time, it is said that *'We perform two sets of experiments: evaluate few-shot learning performance (1) when SDM is trained from scratch; (2) when SDM is meta-initialized.’* The reviewer thinks that there might be a third scenario: while data-driven approximation might be more precise, the finite difference method provides for better features for PDE-agnostic methods. Has this scenario been accounted for or could it be used as an additional ablation study?

2. The pre-training has been performed on the synthetic data, and the authors mention the limitations of the approach: *“
Although introducing auxiliary tasks based on synthetic datasets improves the prediction performance, they need to be chosen and constructed manually and intuitively.”* In connection to that, there might be less scarce, non-extreme, events in the same dataset, which could be used for pretraining. For example, in the extreme weather dataset,  the top-10 extreme weather events since 1984 are selected; is it possible to use some of the remaining data for pretraining on non-extreme events? This could remove the bias of synthetic dataset hyper parameter tuning and could serve as  a baseline. Is it possible to say how would the method, pre-trained on such real non-extreme dataset, perform against the synthetic baseline in this case?

3. In Table 3, the experimental results are reported, with an (anticipated) tendency of diminishing return in spatial meta-initialisation as the number of T-shots increase; is there any way to see whether, as the value of T is increased, there will be the value T when pre-training does not improve or even has worse performance? When does it happen, and does it ever get worse than no-pretraining baseline?
review_title: Interesting idea, clarifications needed

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors propose methodology for sharing learned differencing coefficients for estimating spatial derivatives between multiple spatio-temporal modeling tasks. They show that increased number of tasks improves learning. Additionally, the authors propose a meta-initialization procedure by which the differencing coefficients are initialized to values obtained from synthetic data. They show that this initialization procedure improves performance. 

It seems to me that the 'share-ability' of the spatial differencing coefficients requires that each task is operating on the same spatial manifold with same or similar measurement locations. Toward that end, the initial differencing coefficients appear to be chosen using a flat 2D topology. The appropriateness of this choice is probably responsible for the improvements offered by the proposed approach. However, if the initial topology is poorly chosen, or the different tasks have significantly different topologies one would expect poor performance or negative transfer. How to identify these degenerate cases seems to be an open problem.

Aside from my rehashing the perennial question of 'when does task sharing help?' I found this paper to be well written, and a solid contribution to the field of physics-informed machine learning. Certainly, many applications include multiple different types of measurements at common locations which might be expected to transfer.

review_title: A solid combination of PIML and multi-task learning.

Meta-review: This paper was reviewed by 5 experts in the field. The reviewers raised their concerns on lack of novelty, unconvincing experiment, and the presentation of this paper, While the paper clearly has merit, the decision is not to recommend acceptance. The authors are encouraged to consider the reviewers' comments when revising the paper for submission elsewhere.
Decision: Reject