Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: In this paper the authors show the advantages of keeping track of (and learning) a shared representation across different 'environments' that are switched, while learning to control a partially unknown system.
main_review: The paper is written well and the math looks solid, but it has some weaknesses that prevent me from accepting it:

- Equation (1) is not general enough unlike the claims of Generality (line 99). For a control-affine system, the most general case is B(x,c) having environment dependence. To model it as part of the disturbances w, you'd need to know the bound W, which in this case is not so mild.

- Having the context c revealed by the environment (in the bi-convex case) is not a reasonable or mild assumption in my opinion.

- The method takes the pseudo-inverse of B (as a 'certainty-equivalent' control approach) for computing the controls, hence decoupling the time steps, whereas the whole idea of control theory is to optimize (given known or partly known models) with the future in mind. Hence I fear that the paper does not make a serious contribution to the adaptive control / RL literature. In fact I think the paper does not do justice to the field of adaptive control, reducing it to a case of bandit / regret minimization. Consequences of bad control (safety, stability) are not considered.

- As for meta-learning literature, I also think that the improvement (keeping track of a shared representation and showing that it improves over methods that disregard it) is not a big contribution, although I am not an expert in this field. 

- The authors only show one experiment (partly known pendulum with switching-environment) and even here there is only one baseline. I wonder if the meta learning / adaptive control / RL literature has other methods that can be applied whenever the dynamics switches regularly?

===== UPDATE =====

After reading the rebuttal and re-reading the paper I decided to keep my rating the same. Basically my main concern about not advancing the state of the art of adaptive control (due to only using a certainty-equivalent greedy pseudo-inverse based approach) remains. The contribution to meta-learning I maintain is also not a big one (as also described more convincingly by Reviewer xzK3).

limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work proposes online adaptive nonlinear control under environment-dependend dynamics by assuming shared representations across environments. 
main_review: The idea of applying representation learning/meta-learning through shared representation to nonlinear control problems is important; also the reviewer appreciates the inclusion of the codes at this point too.
Decoupling shared representations and environment parameters seems to be the main novelty.
Here are the reviewer's concerns.

Major concerns:
1) This work considers control affine system as a nominal model; augmenting the state by control input and regarding the time difference of control input as the new "control", general nonlinear system can be translated into control affine form.  However, this work further assumes that the system is fully actuated.  In such a case, the overall assumptions on the system seem to be too restrictive.  If any strong claim holds for this system, it would have some insights; however, the theoretical analysis seems to be a rather simple application of the existing results. 
(basically, based on the assumptions (stability, rank of B, minimum eigenvalue...), exploration is not required)

2) DNN is usually used for scaling up problems; however, the experiments considered in this paper seems to be rather too simple.  If there is no theoretical claims under the use of DNN or the applications to more complex dynamics, it is hard to see the main messages delivered through the experiments.  It is mentioned that decoupling shared representations and environment has several (computational and other) benefits, and this point should be important for this work; however it is not really discussed in example cases etc. 

Minor concerns:
1) When the author(s) say "multi-task", what does it mean?  For model-based scenarios, even if the environment (or the system) does not change, it would be able to deal with different tasks using the learned dynamical system model.

2) there are some minor typos  (e.g. in Alg 1  The environment select -> selects), please recheck them.

If the significance of decoupling Theta and c becomes clearer in terms of applications, the reviewer is happy to raise the score.

--------------------
After initial response

The reviewer still has some concerns about the significance and applicability of the proposed framework; however, the reviewer agrees with the author(s)' arguments "We showed the first end-to-end theoretical results for multi-environment control in nonlinear systems...." and now believes it can be beneficial to place this work as one of the initial attempts of meta learning via shared representations of dynamical systems under variety of environments.  
So, the reviewer raises the score to 6 from 5.
The reviewer hopes the author(s) continue to build on this work to offer more significant insights in theory and applications too.
limitations_and_societal_impact: Limitations may be stated in the concluding remarks in a indirect way; however, it is hard to see how restrictive those limitations are.
Also, please describe limitations of the assumptions on the class F, and limitations when DNN is used (e.g. how restrictive they are).  These will help readers think about how the work can be scaled up or generalized.

Although there is no direct social impact, it is recommended to discuss any possible misuse etc. of the proposed work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This manuscript investigates the online multi-task adaptive control of nonlinear systems subject to unknown uncertainty and disturbances. Several instantiations of the proposed approach are investigated with theoretical guarantees on the resulting average control error. A numerical example is used to demonstrate the effectiveness of the proposed instantiations.
main_review: This manuscript proposes a novel multi-task meta-learning approach for the adaptive control of nonlinear systems subject to unknown uncertainties and disturbances. The manuscript is well-organized and easy to follow, and the theorems and proof look rigorous. The reviewer admires the authors' effort in presenting various instantiations with a thorough investigation of their theoretical properties. Nonetheless, the experimental validation is certainly a weak point of the manuscript. And the reviewer has some concerns over the generality of the proposed approach. 
limitations_and_societal_impact: Major comments:

1) The nonlinear systems investigated in this manuscript are assumed to be fully actuated, which excludes a lot of underactuated systems such as autonomous vehicles/boats, legged robots, and even the cartpole system. Nevertheless, the manuscript still claims that generality is a merit of the proposed approach. 

2) The experimental part only provides a 2D numerical example to benchmark the performance of OMAC, without comparison with other existing meta-learning control approaches. In the reviewer's opinion, the provided example can't prove the general applicability of the proposed approach nor benchmark its performance against existing meta-learning control approaches. 

Minor comments:

1) Some notations are used before proper introduction, e.g. line 162

2) In section 5, which approach does baseline refer to?

3) The emphasis on adversarial disturbances seems unnecessary, as the adversarial disturbances are not particularly considered either in the theorems or the experiments.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes and studies a meta-learning algorithm inspired from nonlinear adaptive control. The algorithm posits that the uncertainty in the model can be decomposed into a shared representation, and an environment specific representation. In each inner loop, the algorithm adapts to the environment specific parameters, whereas in the outer loop the algorithm adapts to the shared representation parameters. A regret-like bound is shown for the “average control error”, which decays not to zero, but to a ball scaling as the maximum size of the disturbances. A simple experiment is run which shows that meta-learning outperforms a baseline adaptive controller which does not attempt to reuse features across tasks.
main_review: The paper is well written and addresses a relevant question in the learning-to-control/RL literature. However, I am recommending a reject because the theoretical results are not very strong; the results are a simple extension combining the recent work of Boffi et al. [12] and Agarwal et al. [24]. Specifically, Boffi et al. [12] show how the e-ISS assumption on the nominal dynamics yields regret bounds for a single task, and the nested OCO framework of Agarwal et al. [24] gives a very clear path to extending the single task result to the multi task setting. Given knowledge of both these works, piecing them together (as is done in the paper) is a relatively straightforward task. I’m open to changing my mind here if the authors could point out what the challenges were in deriving the results. 

Some comments / questions / nits: 

In the definition of ACE, the OMAC algorithm is not actually competing with the omniscient controller, but rather an *upper bound* for the ACE of the omniscient controller. Thus, when the omniscient controller upper bound is not saturated, this means that the results do not imply OMAC approaches the omniscient controller performance even in the limit of infinite data. This is dealt with in Boffi et al. [12] by restricting the size of the noise. At the very least, I would recommend the authors make a short remark addressing this issue.

In Equation 1, are both f and c^{(i)} unknown, or is just c^{(i)} unknown (but the functional form of f known)? I assume that the functional form of f is known (otherwise how does one even take its gradient). Please clarify the exposition around lines 64-65, which can be interpreted as f itself is unknown.

Small nit: consider calling the “element-wise convex” model “bi-convex” instead. 

On Line 247, it is noted that Y_1 \in R^{30} and Y_2 \in R^{20}. This is a typo, as Y_1 and Y_2 should be matrices. 

My main takeaway from Figure 1 was not that meta-learning was a significant win, but rather I was surprised at how well the baseline adaptive algorithm performed. An error of 0.2 radians seems fairly negligible. Perhaps the authors can try to make the learning task harder by increasing the dimensionality of the shared parameters? Or better yet, include a plot that captures the performance degradation of all algorithms as the number of shared parameters increases.

If possible, please include a version of Figure 1 where “no adaptation” is plotted, so we can see how difficult the task is without any adaptation. 
limitations_and_societal_impact: Yes
review_title: null

Meta-review: Overall there is not enough support here from reviewers for me to recommend acceptance, even though one reviewer raised their score from 5 to 6.

Reviewers agreed on some real strengths to the paper, including that it is well-written and addresses an important topic:
* Reviewer 72B6: "The manuscript is well-organized and easy to follow"
* Reviewer xzK3: "The paper is well written and addresses a relevant question in the learning-to-control/RL literature"
* Reviewer vw5a: "The paper is written well and the math looks solid"

The main weakness is described by Reviewer xzK3: "the theoretical results are not very strong; the results are a simple extension combining the recent work of Boffi et al. [12] and Agarwal et al. [24]" and "Given knowledge of both these works, piecing them together (as is done in the paper) is a relatively straightforward task." (discussed in detail in an exchange during the discussion period, with more details). Reviewer vw5a concurred after the discussion: "The contribution to meta-learning I maintain is also not a big one (as also described more convincingly by Reviewer xzK3)." A second concern is around performance shown in experiments, though these were improved to some degree during the discussion period.

Given these issues and the overall scores, I suggest rejection. However, I encourage the authors to improve the paper given the feedback from the reviewers, especially xzK3.
Decision: Accept (Poster)