Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper describes an adaptive controller which includes a learned component. The paper proves that this controller is asymptotically stable. Importantly the paper develops theory which proves that it is not necessary for the learned component to achieve bounded error on the task of predicting the control input given the state, but instead only that the output of the combined system be bounded by a simple function of the current state. The method incorporates a proposal for generating training data for the learned component by sampling trajectories (open-loop) of the system which conform to an SITL specification. The method is evaluated on several common simulation environments which comprise "reach-like" motion planning tasks. The results demonstrate that the proposes adaptive controller converges to the desired reference trajectory, sometimes faster than adaptive methods which do not incorporate the learned component. 
main_review: Strengths
1. The approach presented in this paper is novel and differs significantly from past proposals.
2. The approach is demonstrated on robotic applications.
3. The concept of using a learned component within a control task is of interest to this community. 

Weaknesses
1. The paper claims that the proposed method can be used to achieve behaviors of the controlled system which satisfy SITL specifications. This appears to be subtely incorrect. Theorem 1 describes asymptotic performance guarantees, not performance of the system during a finite horizon. Nothing can be said about the satisfaction of the "prefix" portion of the SITL specification. 
2. The paper correctly claims that the theorem makes no special requirements about the error of the learned component, and moreover that the selection of training data does not affect the claim. This is not the same thing as saying that these elements "don't matter", what is promised is that things won't go terribly wrong, not that the outcomes will be good. The experimental section could be improved by ablations of the off-policy data both in terms of variations of the system dynamics, the quantity of the data, and the diversity of tasks.
3. The learning problem in which the learned component predicts the next control input seems like it could have issues (which do not affect the theoretical guarantees) in the case that there are two tasks with the same "prefix" trajectory but a different end trajectory. It seems that the network would then be forced to predict the average of the two outcomes. Does the methodology seek to avoid sampling such training data. What are the implications?
4. In general the paper oversells some of the claims. See above. it could be improved by taking a less adversarial approach in the writing and acknowledging the trade off in relaxing the approach to ask that the learned component doesn't add arbitrarily large errors. In addition more discussion could be added about the controllability requirement and how limiting this is; I acknowledge that the paper has made some efforts to be precise here. 
5. A sketch of the proof in the main body could be useful. It is extremely technical. Moreover, it could be useful for the author's to discuss whether the learned component must be a neural network and specific properties of neural networks were necessary for the proof. It seems that the setup could be more general. 
6. The use of SITL seems needlessly complex and not the main point of the paper. Beyond its use as a specification language for generating training examples it seems ancillary. See point 1. The paper could be improved by incorporating specifications that use more properties of the SITL formalism for task specification (e.g. beyond reach like tasks). 
7. In the related work some discussion of work like: https://arxiv.org/pdf/1903.11239.pdf could be included. In this reference the authors also learn a component to augment control inputs. They do not have any similar guarantees but it is a better motivated setting. 
summary_of_the_review: This paper describes an approach to adaptive control incorporating a learned component. The novelty relies in the development of a proof technique which doesn't require the learned component to have Lipschitz like properties or exhibit universally bounded error on a space of inputs. The paper is interesting to the community and combines theory with experiment in a coherent manner.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose an adaptive controller that can be applied to a certain class of dynamical systems in order to fulfill Signal Interval Temporal Logic (SITL) tasks. The proposed controller leverages a nominal feedback law learned from past trajectory data collected on possibly many different SITL tasks and system parameters. The authors test their method in simulation on a manipulator arm, unicycle robot, and a pendulum with varying physical parameters and external forces.
main_review: *** STRENGTHS ***

(1) The paper is generally well-written, and is timely given a recent surge in works at the intersection of parametric/non-parametric learning and adaptive control theory. 

(2) The experiments in the paper are extensive; it is good to see multiple different robotic systems included. The experimental results seem detailed and reproducible.

*** WEAKNESSES ***

The core algorithmic content of the paper is spread across: (1) learning a nominal feedback law as a neural network to bound the system dynamics, (2) computing open-loop trajectories satisfying SITL formulae, and (3) adaptive controller design.

Regarding (1): It seems naive to train a single feedback law on data from many different tasks and models (e.g., different controllers are often used for different system dynamics). Moreover, the authors provide no convincing evidence (rigorous, intuitive, or otherwise) that their learned neural network feedback law should satisfy their Assumption 2. Even if the neural network is trained on bounded trajectories, this does not guarantee that the closed-loop dynamics satisfy the particular affine bound in Assumption 2. Overall, the neural network learning content of the paper is not substantive and not convincingly conducive to the adaptive control theory presented later on.

Regarding (2): the paper's narrative is muddied by its focus on SITL. In section 3.2, the authors describe that, using _existing_ work, a smooth open-loop trajectory in R^n satisfying all of the requirements of a SITL task can be computed. Given that the authors do not make any contributions to SITL work and just reduce any SITL task to a trajectory tracking problem, sections 2.1 and 3.2 should be removed entirely. If desired, the authors could simply include a comment on applications of adaptive trajectory tracking control to SITL tasks in a motivation or discussion section.

So far, the possible algorithmic contribution of the paper has been narrowed down to the design of the adaptive controller on pages 5-6 (assuming any nominal controller that bounds the dynamics according to Assumption 2). However, the adaptive controller design contribution is not adequately contextualized against the adaptive control literature. Other than citing an adaptive control textbook from Krstic, et al, the authors need to cite related works and establish the novelty of their design. For example, Equations (3)-(4) seem reminiscent of well-known sliding-mode adaptive controllers, e.g., from:

    J.-J. E. Slotine and J. A. Coetsee. "Adaptive sliding controller synthesis for non-linear systems". International Journal of Control, 1986.

    J.-J. E. Slotine and W. Li. "On the adaptive control of robot manipulators". International Journal of Robotics Research, 1987.

Indeed, the authors' "e_v" is exactly Slotine, et al's sliding variable "s", and the authors' "v_d" is Slotine, et al's reference velocity "q_r". The authors need to discuss how their design differs from such well-known existing work.

Finally, the content on the adaptive control design for a unicycle system, which spans over a page, seems ill-fitted to the main content of an ICLR paper submission.
summary_of_the_review: Of the three sections in the "Main Results" of the paper, only the adaptive control design section is substantive; the proposed neural network training is just naive regression on state-input tuples, and the inclusion of SITL needlessly complicates the paper's exposition before being reduced to a trajectory tracking task. Adaptive control theory alone may be an unsuitable fit for ICLR. Regardless, the novelty of the adaptive control design is questionable, as it appears similar to well-known previous work and the authors' have not contextualized their design amongst the adaptive control literature.

%%
Note: I have raised my score after the authors' rebuttal.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a learning + adaptive control approach to synthesize control policies that fulfills STL specifications. The open-loop trajectory is generated using (Lindemann & Dimarogonas (2020)), the neural network is trained to replicate the expert demonstration consisting of state-action pairs with trajectories collected following multiple tasks (including the target task). This input is used as the open-loop input and a closed-loop trajectory tracking controller is added to correct the trajectory deviation using adaptive control techniques.


main_review: In my opinion, here are the strenght and weakness of the paper.

Strength: 
1) by combining control techniques and supervised learning, the learned controller can potentially give a good prior to the online adaptive tracking
2) the presentation of the proposed method is clear.

Weakness:
1) I'm not convinced that the supervised learning is necessary as given the open loop trajectory, there are plenty of existing methods to track it.
2) Theorem 1 does not mean much as the transient tracking can be really bad, thus violating the STL formula.
3) Infinite control input should not be allowed.
4) It seems that if you drop the learning controller, the open-loop trajectory and online adaptive control can still satisfy the system task, making the learning obsolete.

Minor comments:
1) Assumption 1, do you mean the matrix is PSD, or the entries are positive definite? g may not be square
2) Assumption 2 seems trivial as any bounded controller with bounded f and g satisfies that
3) When mixing trajectories under different tasks, there could be states with totally different inputs (for different tasks), does that make the learned controller inferior to even some heuristic controller? Why not include the task information in the learning process?
summary_of_the_review: It seems to me that the open-loop trajectory and the online adaptive tracking controller are doing the heavy-lifting here and the learning is not that important. The authors should demonstrate the benefit of learning.
review_title: null

Meta-review: The reviewers acknowledge that the paper is well written and contains interesting ideas to combine adaptive control and learning. However, they identified issues regarding the claims about transient tracking and STL formula. Moreover, the significance of the presented learning rule was unclear regarding one reviewer. While the authors could respond well to the identified transient tracking issue, they also needed to weaken their claims, limiting the contribution of the paper. The reviewers therefore stayed with a a reject rating.
Decision: Reject