Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper investigates a first order optimization-based framework for context-based meta-learning of static or dynamical systems. In the considered problem setup, the authors look into settings where a large class of models can be expressed as a combination of a function parametrized by shared parameters $\theta$ and task-specific context parameters $c$. The authors propose FOCA as the framework which uses typical first order optimization methods for online optimization for training with a key ingredient being the use of asymmetric configurations of $\theta$ to learn the shared parameters and the setting $\bar\theta$ used to identify the best context parameter set $c^i$ for any task $i$ during training. To maintain stability over training, the authors propose to use a slowly exponentially moving average of $\theta$ as a proxy for $\bar\theta$ instead of drastically moving best values of $\theta$. Finally, the authors test their proposed framework on a toy static regression problem and dynamical system datasets based on well-known ODE/PDE systems.
strength_and_weaknesses: Strengths:
1. The overall idea behind FOCA is well-motivated along with its connections of using a target network in RL and other works in supervised learning. Addressing the stability of bilevel optimization problems, even outside of meta-learning is a common issue, and this work provides empirical evidence of the efficacy of a simple method.
2. The empirical results for FOCA look quite promising on the datasets considered in the paper. However, there are related issues for comparison (see below).

Weaknesses:
1. Overall, despite the empirical focus of this paper, the discussion about weaknesses of other approaches to handle the dependence of best fit $\hat c^i$ is rather weak. Backpropagating through the optimization trajectory in MAML style methods is an issue but various alternatives have been studied in recent years, some of which are even referenced in this paper. Did the authors investigate any such variations empirically on these datasets? Further, implicit gradients 'ideally' would require solving the inner optimization accurately but can be used as an approximate way of solving the $\hat c$ dependency issue. However, I do agree that adding an exponentially moving average with large values of $\tau$ in this case would benefit as well. Also, can a simpler two timescale learning rate scheme for updating the inner variable $c$ at a slower rate work? It will be worth investigating that.
2. In the experiments, it is not clear as to what extent the context variables capture the task specific parameters vs shared parameters capturing the underlying functional form. For instance, is there any result on how the dimensionality of the $c$ affect performance when it is known that the actual cardinality of the context set if small. Similarly, changes in the overall architecture and investigating the distribution of errors across the tasks in the test set will indicate any undesired behavior.
3. Evaluating CAVIA: How is CAVIA run during test inference? Have the authors ran a few gradient steps $K$ during test time as well? If yes, can the alternative of finding the best fit $\hat c^i$ be tested as well? That way, we can actually decouple the effect of meta-training procedure and not induce indirect effects of the meta-test step.
4. Architectural differences with CODA: In the experiments, the authors use $\theta^i = \theta + W\hat c^i$ as the task-adapted parameter for CODA. This is starkly different from how the counterpart for FOCA is structured: $\hat f^i(x) = f_\theta(x,\hat c^i)$. The main difference between CODA and FOCA can be said to be joint training of $\theta,c$ vs interleaved training using EMA respectively. As such, I do not see the linear approximation for the model as anything inherent to CODA. I recommend the authors to have the same setup for comparison.

clarity,_quality,_novelty_and_reproducibility: The paper often seems to lack clarity in terms of the experimental setup and discussion of results. A few comments for example:
1. The axes in most plots are not labels or described which makes it difficult to judge the actual evaluation criteria. 
2. In the static regression example, it is not clear as to what components $\theta$ and $c$ are capturing. Given the dimensionality of the c parameters $64$, it is not even clear if this is just a spurious overparameterization of the network as the range of linear or quadratic functions in Fig 4 doesn't seem to be diverse at all for a given $\theta$.
3. Distinguishing context based adaptation and a few-shot learning based meta-learning objective is important and hasn't been discussed well in the current draft. In the test phase, $\theta$ is kept fixed and only context inference is allowed. However, can the comparison with other procedures be made when a few steps for updating $\theta$ are also allowed?
summary_of_the_review: Overall, the paper investigates an interesting idea and shows promising empirical results. However, I feel that the paper needs improvement on the thoroughness of experiments, adequate comparison with potential baselines and a better ablation study of the key component providing the improvement (if stability is the main issue, it can be combined with other baselines as well). Further, the tests for whether the context captures task specific information is rather weak and needs further evidence.

NOTE: I want to take this space to note that the review is being submitted outside the stipulated timeframe, and I apologize from my end for the delay. I hope that the authors do find my comments helpful.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper investigates the problem of generalizing to new dynamical systems through an efficient first-order context-based algorithm (FOCA). The algorithm is based on a bi-level learning objective obtained through the modeling-then-identification approach. An EMA step is taken to accelerate the training process.
strength_and_weaknesses: The most confusing part of this manuscript is the *dynamical system* that is claimed to be adapting. But throughout the derivation I'm not seeing why it is even a dynamical system and what makes the adaptation of a dynamical system special in the derivation. I suspect that this is the largest weakness of the manuscript. Some detailed questions follow.

Questions:

1. As suggested by the title of this paper, FOCA should be a method that focuses on the generalization of new dynamical systems. The referred related works such as GBML and CoDA are also works that are specific to dynamical systems. However, from the problem setting described in the paper, it is not very clear to me how $f$ is a dynamic system (or perhaps the relationship between $x$ and $y$). This characterization will be very important as it will set apart this problem from the general setting of Meta-learning, where the datasets are often required to have i.i.d. data. 
2. Following the last question, when $f$ represents a general dynamic system where data are not i.i.d. Solving (6) or line 9 of the algorithm with mean squared loss is perhaps non-trivial as one can no longer directly uses ERM which assumes data are i.i.d. It is indeed discussed in the CoDA paper that in general the MSE objective of (6) is intractable and can only be approximated. 
3. In section 3, it is stated that the learning objective for meta-learning is to “minimize the generalization error on $D_{tr}$. This is confusing as usually the generalization error is defined with the testing dataset while $D_{tr}$ denotes the training dataset. The paper actually did not define the test dataset for the generalization problem studied, thus this left me confused. 
4. For the time-series prediction tasks, FOCA is compared against CoDA, CAVIA, and encoder. However, it seems that CAVIA and encoder were originally designed for the general meta-learning setting and thus it is reasonable to have suboptimal performance for generalization to new dynamical systems. Therefore FOCA can be evaluated more comprehensively by comparing it with other algorithms designed specifically for learning dynamical systems, such as [1] and [2]. Besides, in the CoDA paper, two versions of the algorithm are proposed (with $\ell_1$ or $\ell_2$ constraints), could you state which version of the CoDA is used in this work as a baseline?

[1]: Yin, Y., Ayed, I., de Bezenac, E., Baskiotis, N., and Gallinari, P. LEADS: Learning dynamical systems that generalize across environments. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. Advances in Neural Information Processing Systems, 2021a
[2]: Wang, R., Walters, R., and Yu, R. Meta-learning dynamics forecasting using task inference. CoRR, abs/2102.10271, 2021c.

Miscellaneous questions:
1. For Table 1, it is better to include references to each algorithm in the caption. Otherwise, it can be a bit confusing (e.g. encoder can be a general terminology used while here it is used to refer to a specific algorithm?) The use of absolute value ($| \cdot |$) to donate memory usage is also confusing. To my understanding, $\theta$, $\psi$, and $W$ are parameters that can be a vector/matrix. Therefore it is unclear what the definition of the absolute value of a vector/matrix is. 
2. For Figure 5, could you report the standard deviation of the prediction outcome obtained with the algorithms?


clarity,_quality,_novelty_and_reproducibility: The presentation of the manuscript is overall in good quality.
summary_of_the_review: The manuscript is overall in good quality but I'm not aware of its relevance towards dynamical systems and its novelty thereafter.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors  propose FOCA (First-Order Context-based Adaptation), a learning framework to model sets of systems governed by common but unknown laws that differentiate themselves by their context. FOCA  learns to represent the common law through shared parameters and relies on online optimization to compute system-specific context and its training involves a bi-level optimization problem. This paper use an exponential moving average (EMA)-based method that allows for fast training using only first-order derivatives. 
strength_and_weaknesses: Pros: 
This paper is well-written and present a new contextual meta-learning method FOCA.  FOCA jointly learns model and context parameters by solving online bi-level optimization efficiently. Instead of relying on higher-order derivatives, it employs EMA for faster and more stable training. The authors empirically demonstrated in both static and dynamic regression tasks on complex dynamical systems that FOCA outperforms or is competitive to baselines when evaluated on both in- and out-of-distribution contexts.
Cons: 
As the authors say in the last part,  FOCA only models systems that are identifiable from the given dataset. In the setting where the systems are sampled from nearly identical systems, FOCA may not be able to infer context properly. 
clarity,_quality,_novelty_and_reproducibility: All good in this part. 
summary_of_the_review: I would suggest to accept this paper. It certainly is above the average.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper describes a new meta learning technique that avoids second order derivatives by using an exponential moving average trick.  This is quite interesting.

Strengths: 
* The exponential moving average trick is novel and interesting
* The approach avoids second order derivatives while being is farily general and simple

Weaknesses:
* The paper advertises the technique for dynamical systems in general, but the formulation suggests that it is for static supervised learning.  The authors clarified in the rebuttal that by setting $y$ to $x_{t+1}$, we can model a dynamical system.  This is neat and correct.  However, this needs to be explained in the paper.  Also, the class of dynamical systems that can be modeled this way needs to be specified.  For instance it is not clear how to apply the approach to dynamical systems where there is an $x_t$ and $y_t$ at every time step.  Also, it is not clear whether the time between different time steps can vary.  
* It is not always clear how the poposed technique compares to existing techniques.  For example, the empirical comparison wih CoDA uses a different neural architecture so it is not clear whether the better results are due to different neural architectures or the exponential moving average.  An apple-to-apple comparison is needed.  More generally, the take away of the experiments is not always clear.

Overall, this is very interesting and promising work.  If the paper is not accepted the authors are encouraged to continue this work by addressing the above points while taking into account the feedback of the reviewers.

justification_for_why_not_higher_score: * The paper advertises the technique for dynamical systems in general, but the formulation suggests that it is for static supervised learning.  The authors clarified in the rebuttal that by setting $y$ to $x_{t+1}$, we can model a dynamical system.  This is neat and correct.  However, this needs to be explained in the paper.  Also, the class of dynamical systems that can be modeled this way needs to be specified.  For instance it is not clear how to apply the approach to dynamical systems where there is an $x_t$ and $y_t$ at every time step.  Also, it is not clear whether the time between different time steps can vary.  
* It is not always clear how the poposed technique compares to existing techniques.  For example, the empirical comparison wih CoDA uses a different neural architecture so it is not clear whether the better results are due to different neural architectures or the exponential moving average.  An apple-to-apple comparison is needed.  More generally, the take away of the experiments is not always clear.

Since those concerns are not critical, the paper could be accepted for publication if there is a need for more papers to be published.
justification_for_why_not_lower_score: NA
Decision: Reject