Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a meta-learning mechanism to address current generalization limitations in dynamics forecasting works, in which the majority of works learn deep learning models capable of capturing one dynamics at a time. To this end, they propose *DyAd*, a two-staged approach in which an encoder learns the time-invariant task-specific hidden features of a given observation sequence and influences a forecasting network that aims to learn the shared dynamics of the heterogeneous domain. A time-invariant loss function for the encoder is proposed to encourage time-task disentanglement between encoder and forecaster, leveraging weak-supervision from task-specific parameters. They leverage style-transfer adaptive instance normalization for forecaster adaptation and propose a novel encoder-influenced padding layer called *AdaPad* to address unknown boundary errors. Theoretical proofs are provided to quantify loss term trade-offs in the generalization error and show that there is a relationship between error and task relatedness for the source and target domains. Ablations are provided for each proposed model component and are evaluated on three physical dynamics tasks, including synthetic flows and real-world sea surface temperature and ocean current tasks.
strengths_and_weaknesses: **Strengths:**
- This work tackles a novel task for dynamic systems through the application of meta-learning to enable learning a set of dynamics functions with shared underlying mechanics under one model. It provides ample discussion on common failings of learning dynamical systems and proposes fixes to address them. It is placed well within relevant literature and indeed tackles an insofar uncommon setting.
- The architecture choice of a task-feature encoder that adapts a forecasting network per task via inter-layer controlled padding is intuitive and the rigorous ablations to support the addition of each component strengthens the work.
- Ample baselines are provided from classical video prediction models to applied meta-learning methods used as comparisons.
- The provided code in the Supplementary Material is clean to run and has reproducible results regarding baselines and proposed model. 
- The writing and presentation is clear with little confusion on details concerning architecture or the training setting. The network components are explained well and effectively leverages visualizations in its presentation.

**Minor Code Clean-up:**

README.md:
- A note regarding the base size of the dataset (>150GB) from 'data_generation.py' and how to generate smaller testing versions would help for more approachable evaluation of the provided code.
- There are a variety of unused imports throughout all of the provided scripts which should be cleaned up to reduce unneeded dependencies.

data_generation.py: 
- Has mkdir errors given no path checking for output folders on subsequent runs
- Requires a local module import for phi.geom Sphere not already included
- Variable "Resolution" is undefined and needs removing

**Minor Writing Clean-up:**
- Line 623: 'backpropogation'

questions: **Suggestions:**
- Currently it is hard to interpret the differences in performance of each method through the metrics alone, given that only snapshots of time are shown within the figures. Adding in a figure in the Appendix showcasing full sequence predictions would help the reader understand the quantitative to qualitative mapping for RMSE and ESE.
- The discussion on the Forecaster training and its loss function is a bit short-handed compared to other sections and it can be a bit difficult to interpret how it is exactly formulated. The Appendix highlights that multiple unrolled prediction steps are used in backpropagation while the Forecaster Training section highlights the loss is accumulated over 'different time steps'. Clarification on whether this refers to 1) each training sequence has multiple time windows evaluated on for its pure next step prediction and this is what is accumulated in loss for backpropagation or whether 2) for each window used, multiple steps are predicted out autoregressively and the multi-step sequence is used in the loss function.
limitations: The authors properly address limitations in the claims of their proofs and what information should be gleaned from them. Additionally, limitations in the metrics used to support evaluation are discussed in the Appendix. Potential negative societal impact are not discussed in the work and is denoted as such in the Author's Checklist.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a method to predict dynamical systems when system parameters can differ between training and evaluation.
First, a time invariant network is trained to predict the dynamical "invariants" which could be the number of vortices in a flow or system parameters.
Then a prediction network is trained to forecast the system dynamics, taking the dynamic invariants as an input.

strengths_and_weaknesses: ### Strengths
- Interesting and innovative idea. (I may not be aware of prior work using this approach)
- Good experimental results, although maybe a bit limited datasets.
- Good and clear presentation

### Weaknesses
One potential weakness of the method is that one needs access to the invariants for training.

questions: 1. How do meta-training and meta-testing sets differ? Can you quantify how large the shift is between training and testing dataset (e.g. a shift in the mean number of vortices)?  In particular, do you evaluate on values of the invariant that are not included in the training set?
2. Feeding the invariant as an input to the prediction model is one way to incorporate the information. If one can compute invariants in a differentiable way one could also incorporate this as a loss where context and predictions should not differ in the invariant.
Have you considered this aswell? It probably only works for some invariants, but may be interesting.
limitations: -
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: A physics informed meta-learning architecture is introduced to model different kinds of dynamics. An encoder generates a time-invariant latent state $\hat{z}$ representing physical properties of the observed dynamics. In combination with the dynamics field, a decoder takes this latent state to condition its prediction of the next dynamic steps.
strengths_and_weaknesses: ### Originality
Strengths:
- The AdaPad layer looks intriguing, as it allows the boundary condition (BC) to depend on $x$. In contrast to traditional padding methods (zero/constant or mirror), AdaPad thus seems able to account for dynamic BCs, such as Neumann and Cauchy. However, it might help if it additionally receives $x$ as input. Have the authors experimented with this?
- Fairly simple but highly efficient and well designed architecture to model different kinds of dynamics with one and the same forecaster

### Quality:
Strengths:
- Overall very detailed and well motivated model architecture, data description and evaluation
- Clear demonstration of successful encoding of physically relevant factors (figure 6)
- Exhaustive ablations showing the relevance of the different introduced network components

Weaknesses:
- Apart from the ablations in table 2 and in the appendix, it would be informative to see the effect of different choices of $m$

### Clarity and significance:
Strengths:
- Superiority of the model shown on both synthetic and real-world datasets

Weaknesses:
- I'm not fully clear about the conclusion of the proof and would appreciate an intuition about the result and what it actually means.
- AdaPad operator not explicitly evaluated on changing boundary conditions
questions: - The input and output sequences have different indices compared to the text (lines 71 and 72). It might help readers if the indices were consistent, i.e., changing $w_{1:t}$ and $\hat{w_{t+1:t+k}}$ into $w_{t-l+1:t}$ and $\hat{w_{t+1:t+h}}$.
- According to your reported changes in $\hat{c}$ when shifting the input (over time), how much does $\hat{c}$ actually change (in particular since you enforce it to stay constant with the second term in equation (1))? It would be great if you could show that $\hat{c}$ is fairly constant for one time series encoded at different temporal samples. Certainly, though, $\hat{c}$ should change meaningfully when another task is encoded.
- Since the decoder conditions its prediction on the hidden features, could you also formulate $f_c = f(x|z_c)$ to emphasize this conditioning?
- Could you use something different from $\mu$ for the data distribution in line 84, since you also choose $\mu$ when defining the mean of channels for AdaIN (line 155)?
- In line 156, should it be "For each AdaIN *channel*, a particular style is computed $s_i = $...", or are the styles indeed layer-wise? From figure 2 I assume that you only use one AdaIN layer, though, meaning you only use one style?
- Have you tested recurrent modules in the encoder and forecaster?
- How do you explain the performance of the big U-Net as forecaster compared to the small one? And do you have an intuition why U-Nets seem better on the SST data while ResNet is superior in Turbulent Flows and Ocean Currents?
limitations: Limitations not addressed by the authors. I'd be curious to learn about situations where this model might have difficulties.
review_title: null

Meta-review: metareview: This work proposes a model-based meta-learning method to forecast physical dynamics. The proposed approach is able to generalize across heterogeneous domains as demonstrated in convincing sets of experiments. The reviewers found the work to be well motivated, clear and self-contained. Authors justified the proposed model architecture and the ablation studies conducted showed the importance of the network components. The authors also provided an adequate description of the data and the evaluation strategy, as well as theoretical guarantees on the generalization error in several settings.

confidence: Certain
Decision: Accept