Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper formulates a novel approach for robust model-based learning based on control lyapunov functions (CLF) and control barrier function (CBF) in a unified framework that allows to produce a  robust lyapunov-like function that certifies stability and safety for a system with unknown but bounded parameters. The construction of the Lyapunov-like function is automatically generated from a supervised learning framework. The authors show the performance of the controller compared against robust MPC formulations on 7 different plants. They show that the robust control lyapunov function based QP (rCLBF-QP) offers more reliability on the safety rate and it outperforms robust MPC in most of the cases. 
strengths_and_weaknesses: The paper has an incremental and very clear presentation of the rCLBF, expressing theorems that provide a sound formulation of the controller. 
The idea of providing a lyapunov-like function that produces a certificate for stability and safety together provides a better framework for learning and optimization.
Moreover, the conditions of the theorems are used to construct a Loss function for the neural network approximator of the lyapunov-like function, leaving less design work to hand tuned parameters or functions. 

summary_of_recommendation: The paper proposes a novel formulation of a unifying lyapunov-like function that provides a robust certificate for stability and safety. The formulation allows an automatic way to produce such functions via supervised learning. 
issues: The paper might benefit with more discussion about the implementation details of the neural network, like the number of train points, or the parameters a1 - a3, for instance.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a somewhat novel definition of a Control Lyapunov-Barrier function. Such a function is approximately synthesized by learning it as a neural network using samples drawn from a robot's state space as training data. The method is demonstrated successfully on a variety of examples of increasing complexity.
strengths_and_weaknesses: STRENGTHS
- The first three sections are really well-written and easy to follow, much better than many other papers I've read
- The core idea is clear and well-founded in controls theory; I love seeing this effort to merge controls and learning literature.

WEAKNESSES
- The proposed CLBF is not very surprising, and the paper doesn't really address the hardest parts of synthesizing such an object; I don't feel that the idea is novel enough on its own to warrant such a "light" discussion of the challenges (which are, to the paper's credit, clearly stated in the conclusion!)
- The experiments section is severely lacking in explanation of how exactly the CLBF is synthesized; since this is a paper about learning a CLBF, it should explain more about learning, not just about application.
- The proposed method could probably be compared against a couple other approaches, besides just the rather naive robust MPC in the current comparison. I'd recommend comparing against an HJB PDE method and, if possible, a GP-based safe RL method.

summary_of_recommendation: I personally don't see how this is especially different from learning a potential field or other generic barrier function style approach. Given that the core idea is the CBLF, it was surprising to me that, just using the paper, I had no clear idea of how I'd implement the proposed method. Perhaps that is just my own ignorance! But I think there are many clarifications that could be added about the proposed method to make it a much more convincing paper.

Update (I'm leaving my previous comments unedited). The authors have addressed my concerns both in their comments and in the paper. I think the paper is much stronger now, and demonstrates an interesting advance in RL. I also must admit that I didn't carefully read the supplementary material while originally reviewing the paper! So now I will switch from "reject" to "accept."
issues: Here I've listed notes that I took while reading, most of which are "issues" but some of which are also just comments about what I enjoyed when reading the paper :)

Abstract
- Are the safety and stability guarantees DURING training, or only AFTER training?
- Does the proposed method consider actuator limits? How does one ensure there is always a feasible safe control action available?

Introduction / Related Work
- Fig. 1 would benefit from a bit of organization (maybe create a grid?) and some additional subtitles or labels explaining what each problem is (e.g., drone, segway, car)
- The discussion of robust MPC is really helpful and clear
- It's a bit odd to have just one subsection in Sec. 1
- An additional challenge to note with CBFs and CLFs is that it's unclear if a feasible safe control is always available (i.e., is the QP always feasible?)

Preliminaries and Background
- Thank you for clearly stating the problem! So many papers fail to do this!
- The CLBF definition is nice, but sounds very non-convex and hard to satisfy when synthesizing $V$ -- I'm really curious to see how you do it!
- Minor typo: the subscript "CLBF" in the paragraph under (2) shouldn't be italicized
- Give that CLBFs are basically a type of potential field, how do you actually ensure reachability? That is, how do you ensure the controlled system doesn't get, e.g., "stuck" behind an obstacle when the gradients cancel out in (1e)?

Robust CLBF Certificates for Safe Control / Learning rCBLFs
- I've had lots of reviewers complain when I put core proofs in supplementary materials. I personally think this is fine, but just be prepared for other reviewers to complain ;)
- Is it really reasonable to consider a *convex hull* of parameters? The idea of convexity here needs some further explanation to guide reader intuition and justify this idea.
- Fig. 2 is... not really helpful at all. I mean, there's a state, a neural network or two, and some outputs.
- Ah, I love this idea that $\pi_{\mathrm{NN}}$ is used to prove that the feasible set of the QP is nonempty! Very curious to see how that works.
- How would this method ever sample enough points in a high-dimensional state space to enable making strong claims that $V$ satisfies (1) (besides the "easy" claim of satisfying (1b))?
- It might be better to use "split" rather than "subequations" for (5), since it's a single equation continued onto a second line.
- I prefer the convention of using "\cdot" instead of "\circle" to represent an arbitrary argument (e.g., in where the paper explains the ReLU notation)
- OK, so it seems sneaky that $\pi_{\mathrm{NN}}$ is supposed to ensure the QP is always feasible, but, at the end of the day, you have to relax the QP to make this possible.
- It seems terrible to have to verify $V$ after training it. Was this verification actually implemented? I guess I'll find out in the next section!

Experiments
- What does "steadily" increasing complexity mean? Just say "increasing complexity" and avoid empty words!
- I really appreciate that you also tried to implement a SOS method. They're really hard to use!
- What hardware did you use to train your neural networks? How long did training take? Did you apply a verification method afterwards to check that the Lyapunov conditions hold? It's really odd to write a neural network training paper and then include no training results or discussion!
- For the simpler models, it would also be good to compare against, e.g., a grid-based HJB PDE solver approach, since such methods would have similar online running rates as your proposed method.
- Fig. 5 is fun and helpful

Discussion / Conclusion
- Cool, the method works!
- There are huge open questions, though! I think the conclusion does a good job of stating these questions -- but they aren't very surprising. I'm a little sad this paper doesn't do a better job of trying to address these questions, since the underlying CLBF idea isn't incredibly new or surprising.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The main idea of the paper is to design feedback control systems with safety and stability guarantees by fitting robust Lyapunov/Barrier functions and policies, and then applying a standard pointwise min-norm control. The resulting controller is compared in simulation to robust linear MPC on a number of interesting examples.

strengths_and_weaknesses: Strengths:
1) The authors are clearly well-versed in control theory. Material in Sections 1-4 provides a well-written overview of the main ideas around control Lyapunov functins (CLFs) and their close cousins the control barrier functions (CBFs). 
2) The main technical novelty is a proposed method for computing CLFs, CBFs, and policies as a supervised-learning problem, given in Section 5, although this builds fairly naturally on the contributions of citation [8] in the paper. 
3) The simulation results are interesting and demonstrate clear potential for the method. Experiments would be nice of course, but in the year of COVID it is understood that this was extremely difficult.

Weaknesses:
1) This is minor, but I think it could be more clearly stated in Definition 3 or Theorem 2 that the dynamics must be affine in \theta for this approach to work. This is briefly mentioned at the start of Section 3 and is dismissed as not restrictive, however in many cases of interest in robotics, in order to achieve an affine parameterisation of uncertainty it is necessary to use quite conservative overparameterisations.
2) Similarly, for parameter-dependent problems it is known that have parameter *independent* Lyapunov functions can be very conservative, however this is required for the proposed approach. The conservatism of this is not discussed.
3) I think the main contribution of the paper is really the sampling and fitting scheme in Section 5, but the discussion of sampling is not very deep, it just states that N_train points are sampled uniformly at random from X. 
- Is X bounded? If not, how does one sample uniformly? If it is, how do we know X is robustly forward-invariant? Usually a Lyapunov function would be used to verify such a property, but this is assumed before the search for a Lyapunov function begins. 
- Why uniform sampling? The method proposed is analogous to an actor-critic algorithm in RL with the "critic" in this case being a CLBF, and the Lyapunov condition replacing the Bellman equation. In RL uniform sampling of the state space is rare, and sampling design is indeed fundamental to the problem of "exploration vs exploitation", as well as classical experimental design.
4) I'm a little uneasy about claiming safety based on condition (3a) being satisfied only on a finite collection of sample points. The paper says "once trained, V can be verified using..." and gives some references. But these are quite general in nature, it is not entirely clear how they would be adapted to the present problem. Would they prove that (3a) holds globally?
summary_of_recommendation: Overall I enjoyed reading the paper and think it has the beginnings of a very useful approach. I have some reservations about points of detail, as discussed above, at least if the objective is for feedback systems that are really provably safe.
issues: As above, I think the paper would be significantly improved if:
- The technical requirements of affine parameterisation and parameter-independent Lyapunov functions were discussed more clearly
- The question of sampling training points was clarified and explored in more detail
- It was clarified how the learned CLBF could be rigorously verified.
review_title: null

Meta-review: This paper proposes a novel learning-based approach to synthesizing robust nonlinear feedback controllers. While the reviewers agree that this paper contains some interesting ideas, they also raise a number of important concerns. The authors should carefully address the reviewers' comments in their rebuttal. 

UPDATE POST DISCUSSION PHASE: I would like to thank the authors for their comments and clarifications during the discussion phase. The reviewers agree that this paper provides a valuable contribution, and I concur with this assessment. 

Decision: Accept (Poster)