Learning Performance-oriented Control Barrier Functions Under Complex Safety Constraints and Limited Actuation

Authors: lakshmideepakreddy  manda
shaoru  chen
mahyar  fazlyab
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 02:42:44

Abstract: Control Barrier Functions (CBFs) offer an elegant framework for constraining nonlinear control system dynamics to an invariant subset of a pre-specified safe set. However, finding a CBF that simultaneously promotes performance by maximizing the resulting control invariant set while accommodating complex safety constraints, especially in high relative degree systems with actuation constraints, remains a significant challenge. In this work, we propose a novel self-supervised learning framework that holistically addresses these hurdles. Given a Boolean composition of multiple state constraints defining the safe set, our approach begins by constructing a smooth function whose zero superlevel set provides an inner approximation of the safe set. This function is then used with a smooth neural network to parameterize the CBF candidate. Finally, we design a physics-informed training loss function based on a Hamilton-Jacobi Partial Differential Equation (PDE) to train the PINN-CBF and enlarge the volume of the induced control invariant set. We demonstrate the effectiveness of our approach on a 2D double integrator (DI) system and a 7D fixed-wing aircraft system (F16).

Main paper: 
 introduction: 
 CBFs are a powerful tool to enforce safety constraints for nonlinear control systems [1] , with many successful applications in autonomous driving [2] , UAV navigation [3] , robot locomotion [4] , and safe reinforcement learning [5] . For control-affine nonlinear systems, CBFs can be used to construct a convex quadratic programming (QP)-based safety filter deployed online to safeguard against potentially unsafe control commands. The induced safety filter, denoted as CBF-QP, corrects the reference controller to remain in a safe control invariant set.While Control Barrier Functions (CBFs) provide an efficient method to ensure safety, finding such functions can be challenging. Specifically, there is no guarantee that the resulting safety filter will remain feasible throughout operation. This is primarily because the "model-free" construction of the filter only incorporates the constraints. Complex constraints, high relative degree, and bounded actuation exacerbate the challenge of ensuring feasibility. Various techniques have been proposed to address these challenges such as CBF composition for complex constraints [6, 7, 8, 9] , higher-order CBFs for high relative degree [10] [11] , and integral CBFs [12] for limited actuation. Despite significant progress, these approaches can make the filter overly restrictive, thus limiting performance.Contributions We propose a novel self-supervised learning framework for CBF synthesis that systematically addresses all the above challenges. First, we handle complex safety constraints and high relative degree in CBF synthesis by encoding the safety constraints into the CBF parameterization with minimal conservatism. Second, we design a physics-informed training loss function based on Hamilton-Jacobi (HJ) reachability analysis [13] to satisfy bounded actuation while maximizing the learned control invariant set volume. We evaluate our method on the double-integrator and the high-dimensional fixed-wing aircraft system and demonstrate that the proposed method effectively learns a performant CBF even with complex safety constraints. We call the proposed framework Physics-informed Neural Network (PINN)-CBF [14] .Complex safety constraints For a safe set described by Boolean logical operations on multiple constraints, [6] composes multiple CBFs accordingly through the non-smooth min/max operators. [7] introduces smooth composition of logical operations on constraints, which was later extended to simultaneously handle actuation and state constraints [8] [9] using integral CBFs [12] . [15] proposes an algorithmic way to create a single smooth CBF arbitrarily composing both min and max operators. Such smooth bounds have been used in changing environments [16] . Notably, [17] ensures the input constrained feasibility of the CBF condition while composing multiple CBFs.High-order CBF High-order CBF (HOCBF) [10] and exponential CBF [11] are systematicly approach CBF construction when the safety constraints have high relative degree. However, controlling the conservatism of these approaches is a challenge. To reduce conservatism or improve the performance of HOCBF, various learning frameworks have been proposed [18, 19, 20] that allow tuning of the class K functions used in the CBF condition.Learning CBF with input constraints Motivated by the difficulty of hand-designing CBFs, learning-based approaches building on the past [21] have emerged in recent years as an alternative [22, 23, 24, 25] . Liu et al. [26] explicitly consider input constraints in learning a CBF by finding counterexamples on the 0-level set of the CBF for training, while Dai et al. [27] propose a dataefficient prioritized sampling method. [28] explores adaptive sampling for training HJB NN models favoring sharp gradient regions. Drawing tools from reachability analysis, the recent work [29] iteratively expands the volume of the control invariant set by learning the policy value function and improving the performance through policy iteration. Similarly, Dai et al. [30] expand conservative hand-crafted CBFs by learning on unsafe and safe trajectories, and Qin et al. [31] applies actor-critic RL framework to learn a CBF value function while [32] learns a control policy and CBF together for black-box constraints and dynamics. Our method differs by being controller-independent and basing its learning objective on the HJ partial differential equation (PDE), which defines the maximal control invariant set, without requiring trajectory training data.HJ reachability-based methods The value functions in HJ reachability analysis have been extended to construct control barrier-value functions (CBVF) [33] and control Lyapunov-value functions [34] , which can be computed using existing toolboxes [35] . Tokens et al. [36] further apply such tools to refine existing CBF candidates, and Bansal et al. [37] learn neural networks solution to a HJ PDE for reachability analysis. Of particular interest to our work is the CBVF, which is close to the CBF formulation and provides a characterization of the viability kernel. Our work aims to learn a neural network CBF from data without using computational tools based on spatial discretization.Notation An extended class K function is a function α : (-b, a) → R for some a, b > 0 that is strictly increasing and satisfies α(0) = 0. We denote L + r (h) = {x ∈ R n | h(x) ≥ r} as the rsuperlevel set of a continuous function h(x). The positive and negative parts of a number a ∈ R are denoted by (a) + = max(a, 0) and (a) -= min(a, 0), respectively. Let ∧, ∨, ¬ denote the logical operations of conjunction, disjunction, and negation, respectively. For two statements A and B, we have ¬(A ∧ B) = (¬A) ∨ (¬B) and ¬(A ∨ B) = (¬A) ∧ (¬B). 
background and problem statement: 
 Consider a continuous time control-affine system:EQUATIONwhere D ⊆ R n is the domain of the system, x ∈ D is the state, u ∈ U ⊂ R m is the control input, U ⊆ R m denotes the control input constraint. We assume that f : D → R n , g : D → R n×m are locally Lipschitz continuous and U is a convex polyhedron. We denote the solution of (1) at time t ≥ 0 by x(t). Given a set X ⊆ D that represents a safe subset of the state space, the general objective of safe control design is to find a control law π(x) that renders X invariant under the closed-loop dynamics ẋ = f (x) + g(x)π(x), i.e., if x(t 0 ) ∈ X for some t 0 ≥ 0, then x(t) ∈ X for all t ≥ t 0 . A general approach to solving this problem is through control barrier functions.Suppose the safe set is defined by the 0-superlevel set of a smooth function c(•) such that X = {x | c(x) ≥ 0}. For X to be control invariant, the boundary function c(•) must satisfy max u∈U ċ(x, u) ≥ 0 when c(x) = 0 by Nugomo's theorem [38] . However, since c(•) does not necessarily satisfy the condition, we settle with finding a control invariant set contained in X through CBFs. Definition 1 (Control barrier function). Let S := L + 0 (h) ⊆ X ⊆ D be the 0-superlevel set of a continuously differentiable function h : D → R. Then h(•) is a control barrier function for system (1) if there exists an extended class K function α such thatEQUATIONwhereL f h(x) = ∇h(x) ⊤ f (x) and L g h(x) = ∇h(x) ⊤ g(x) are the Lie derivatives of h(x).Given a CBF h(•), the non-empty set of point-wise safe control actions is given by any locally Lipschitz continuous controller π(x)∈ K cbf (x) = {u ∈ U | L f h(x) + L g h(x)u + α(h(x)) ≥ 0}renders the set S forward invariant for the closed-loop system, which enables the construction of a minimally-invasive safety filter:EQUATIONwhere u r (x) is any given reference but potentially unsafe controller. Under the assumption that U is a polyhedron, K cbf (x) is also a polyhedral set and problem (3) becomes a convex QP.While the CBF-QP filter is minimally invasive and guarantees x(t) ∈ L + 0 (h) ⊆ X for all time, a small L + 0 (h) essentially limits the ability of the reference control to execute a task. To take the performance of the reference controller into account, we consider the following problem. Problem 1 (Performance-Oriented CBF). Given the input-constrained system (1) and a safe set X defined by complex safety constraints (to be specified in Section 3.1), synthesize a CBF h(•) with an induced control invariant set S = L + 0 (h) such that (i) S ⊆ X and (ii) the volume of S is maximized. Formally, this problem can be cast as an infinite-dimensional optimization problem:EQUATIONsubject to L + 0 (h) ⊆ X (safety) sup u∈U {L f h(x) + L g h(x)u + α(h(x))} ≥ 0, ∀x ∈ D (control invariance)where H is the class of scalar-valued continuously differentiable functions.i ∈ L + 0 (c)} N i=1 . 
proposed method: 
 In this section, we present our three-step method of learning PINN-CBF :(S1) Composition of complex state constraints: Given multiple state constraints composed by Boolean logic describing the safe set X , we equivalently represent X as a zero super level set of a single non-smooth function c(•), i.e., X = L + 0 (c) = {x | c(x) ≥ 0}. (S2) Inner approximation of safe set: Given the constraint function c(•) obtained from the previous step, we derive a smooth minorizer c(•) of c(•), i.e., c(x) ≥ c(x) for all x ∈ D, s.t. L + 0 (c) ⊆ X . (S3) Learning performance-oriented CBF: To approximate the largest control invariant subset of L + 0 (c), we design a training loss function based on control barrier-value functions and HJ PDE [33] . We propose a parameterization of the CBF and a sampling strategy exploiting the structure of the PDE.Suppose we are given N setsS i := L + 0 (s i ) = {x | s i (x) ≥ 0}where each s i : R n → R is continuously differentiable and the safe set X is described by logical operations on {S i } N i=1 . Since all Boolean logical operations can be expressed as the composition of the three fundamental operations conjunction, disjunction, and negation [6, 15] , it suffices to only demonstrate the set operations shown below:1. Conjunction: x ∈ S i AND x ∈ S j ⇔ x ∈ S i ∩S j = {x | s(x) := min(s i (x), s j (x)) ≥ 0}. 2. Disjunction: x ∈ S i OR x ∈ S j ⇔ x ∈ S i ∪ S j = {x | s(x) := max(s i (x), s j (x)) ≥ 0}. 3. Negation: NOT x ∈ S i ⇔ x ∈ S ∁ i = {x | s(x) := -s i (x) ≥ 0} (complement of S i ).The conjunction and disjunction of two constraints can be exactly expressed as one constraint composed through the min and max operator, respectively. Furthermore, negating a constraint s i (x) ≥ 0 only requires flipping the sign of s i . These logical operations enable us to capture complex geometries and logical constraints as illustrated in the following two examples.Example 1 (Complex geometric sets). Consider x = [x 1 x 2 ] ⊤ ∈ Rs 1 (x) = -x 1 + c 1 , s 2 (x) = x 1 -a 1 , s 3 (x) = -x 2 + d 1 , s 4 (x) = x 2 -b 1 , s 5 (x) = -x 1 + c 2 , s 6 (x) = x 1 -a 2 , s 7 (x) = -x 2 + d 2 , s 8 (x) = x 2 -b 2 ,and let c(x) = max(min(s 1 (x), s 2 (x), s 3 (x), s 4 (x)), min(s 5 (x), s 6 (x), s 7 (x), s 8 (x))). Then, we haveL + 0 (c) = O 1 ∪ O 2 .Example 2 (Logical constraints). Consider the three constraints s i (x) ≥ 0, i = 1, 2, 3, at least two of which must be satisfied. This specification is equivalent to the constraint c(x) ≥ 0, where c(x) = max(min(s 1 (x), s 2 (x)), min(s 2 (x), s 3 (x)), min(s 1 (x), s 3 (x))).In summary, by composing the min and max operators, we can construct a level-set function c : R n → R such that x ∈ X ⇔ c(x) ≥ 0, i.e., X = L + 0 (c). Being an exact description of X , however, c(•) is not smooth. Next, we find a smooth lower bound of c(•) that facilitates CBF design.To find a smooth lower bound for c(•), we utilize its compositional structure. We bound the max operators using the log-sum-exponential function as follows [15] (min follows similarly):EQUATIONwith β > 0. As β → ∞, these bounds can be made arbitrarily accurate. We note that both the lower and upper bounds in (5) are smooth and strictly increasing in each input s i . Therefore, to obtain a lower bound c(x) of c(x), it suffices only to compose the lower bounds on each min and max function. In Figure 2 we show the effect of β on the resulting inner approximation. The smooth function c(•), whose 0-superlevel set provides an inner approximation of the safe set X , is not necessarily a CBF. Thus, we aim to find the "closest" CBF approximation of c(•). We learn our Neural Network (NN) model using a Hamilton-Jacobi (HJ) PDE from reachability analysis whose infinite-time horizon solution precisely characterizes the CBF maximizing the volume of L + 0 (h). Our neural network model is trained by minimizing the PDE residual, grounding the method in physics, and justifying its name (PINN)-CBF.Hamilton-Jacobi PDE for reachability Consider the dynamics (1) in the time interval [t, 0], where t ≤ 0 and x are the initial time and state, respectively. Define U [t,0] as the set of Lebesgue measurable functions u : [t, 0] → U. Let ψ(s) := ψ(s; x, t, u(•)) : [t, 0] → R n denote the unique solution of (1) given x and u(EQUATIONwhich is the set of all initial states within L + 0 (ℓ) from which there exists an admissible control signal u(•) that keeps the system trajectory within L + 0 (ℓ) during the time interval [t, 0]. Solving for the viability kernel can be posed as an optimal control problem, where V(t) can be expressed as the superlevel set of a value function called control barrier-value function (CBVF). Definition 2 (CBVF [33] ). Given a discount factor γ ≥ 0, the control barrier-value functionB γ : D × (-∞, 0] → R is defined as B γ (x, t) := max u(•)∈U [t,0] min s∈[t,0]e γ(s-t) ℓ(ψ(s)).For t ≤ 0, we have Proposition 2] . Additionally, B γ is the unique Lipschitz continuous viscosity solution of the following HJ PDE with terminal condition B γ (x, 0) = ℓ(x) [33, Theorem 3]:V(t) = {x | B γ (x, t) ≥ 0} [33,EQUATIONUnder mild assumptions of Lipschitz continuous dynamics and ℓ(x) being a signed distance function, B γ (x, t) is differentiable almost everywhere [39] . Furthermore, taking t → -∞, the steady state solution B γ (x) := B γ (x, -∞) gives us the maximal control invariant set contained in L + 0 (ℓ) [33, Section II.B]:N (B γ , ℓ) := min ℓ(x)-B γ (x), max u∈U L f B γ (x)+L g B γ (x)u+γB γ (x) = 0, ∀x ∈ D (8)We will leverage the above PDE to learn a CBF h whose zero superlevel set approximates the maximal control invariant set.In the context of our problem, we are interested in the viability kernel of L + 0 (c(x)), where c is the smoothed composition of the constraints. Thus, our goal is to learn a CBF h that satisfies the PDE N (h, c) = 0. To this end, we parameterize the CBF candidate asEQUATIONwhere δ θ : D → R ≥0 is a non-negative continuously differentiable function approximator with parameters θ. In this paper, we parameterize δ θ in the form of a multi-layer perceptron:EQUATIONwhere {W k , b k } K k=0 are the learnable parameters, σ(•) is a smooth activation such as the ELU, Tanh, or Swish functions [40] , and σ + (•) is a smooth function with non-negative outputs, i.e., σ + (r) ≥ 0, ∀r ∈ R. We choose σ + (•) as the Softplus function, σ + (r) = 1 β log(1 + exp(βr)), β > 0. The parameterization (9) offers several advantages over using a standard MLP model for h θ . First, it accelerates learning by enforcing the non-negativity constraint c(x) ≥ h(x) in ( 8) by design. Second, complex constraints are directly integrated into the system via c(x), simplifying the learning process. Importantly, h θ (x) is automatically non-positive in regions containing obstacles, allowing the learning process to focus elsewhere.In summary, the parameterization in (9) ensures that h θ (•) is smooth and satisfies Training loss and sampling distribution Given the parameterization (9) , we now propose to train h θ to approximately satisfy the steady-state HJ PDE. This leads to the following physics-informed risk minimizationh θ (x) ≤ c(x) ≤ c(x) for all x ∈ D, implying that L + 0 (h θ ) ⊆ L + 0 (c) ⊆ L + 0 (c) = XEQUATIONEQUATIONEQUATIONThe first loss L HJ (x; θ, γ) is the square of the PDE residual to increase the volume of L + 0 (h θ ), while the second loss L CBF (x; θ, γ) enforces the CBF condition. We note that the latter is implicitly present in the PDE but not enforced by the corresponding loss L HJ (x; θ, γ).Finally, µ represents a sampling distribution over L + 0 (c), meaning no learning is required within obstacle regions. Efficient sampling in these regions can be achieved using intelligent sampling methods like envelope rejection [41] or the random walk Metropolis-Hastings (RWMH) [42] , both of which are well-suited for cluttered environments. Notably, the training process remains selfsupervised, as only domain points are collected for learning.In the following, we demonstrate that jointly enforcing L HJ and L CBF together with intelligent sampling (IS) improves the performance and safety of the learned PINN-CBF. We note that for polyhedral actuation constraint sets, the supremum over u is achieved at one of the vertices, yielding a closed-form expression. We summarize our self-supervised training method in Algorithm 1.Input: Constraints {c i }, smoothing parameter β > 0, discount factor γ > 0, regularization parameter λ ≥ 0 Output: Modules δ θ (•) and c(•) s.t. h θ = c(•) -δ θ (•)1. Following sections 3.1 and 3.2, compose constraints {c i } using β to form c.θ randomly with W L , b L = 0. 4. ERM: minimize θ Ĵ(θ) = 1 N N i=1 L HJ (x i ; θ, γ) + λL CBF (x i ; θ, γ) 2 
experiments: 
 We demonstrate the efficacy of PINN-CBF on a 2D double integrator system and a 7D fixed-wing aircraft system. Experiments are performed on Google Colab, where the PINN-CBF training is performed on a T4 GPU with 15GB RAM and its validation on a CPU with 52GB RAM. The code is available at https://github.com/o4lc/PINN-CBF.Double Integrator First, we consider the double integrator benchmark ẋ = v v = u, with x ∈ R denoting the position and v ∈ R denoting the velocity. The action u ∈ R represents the acceleration of the system and directly operates on v.We generate a complex obstacle configuration consisting of rotated rectangles and unit walls bordering the state space D = {(x, v) | -1 ≤ x ≤ 11, -6 ≤ v ≤ 6} shown in Fig. 2 . Then, we parameterize PINN-CBF following Section 3.3 and train it with N = 10 4 uniform samples. At each state in the training data, we compute the values and gradient of c using JAX [43] flexible automatic-differentiation and hardware acceleration. The acceleration u is bounded by u ∈ [-1, 1]. The nominal controller is given by a PID controller meant to stabilize the system to a target position (see Appendix A.1). We evaluate the combination of loss functions (11b) and (11c) with intelligent sampling (IS) from Algorithm 1, as shown in Table 1 . All proposed ablations and baseline methods use the same architecture (9) for a fair comparison, accommodating complex input-space constraints. The MLP for δ θ has layer widths of 2 -50 -50 -1, while DeepReach [37] includes time as an additional input and is applied over the full horizon with t = 0.Using three metrics, We validate at 10 5 grid samples covering the effective obstacle-free space L + 0 (c). The residual error |N (h, c)| denotes how close the learned CBF is to the maximal-invariant solution of the HJ PDE, L CBF measures safety violation, and Vol(L + 0 (h θ ))/Vol(L + 0 (c)) represents the volume ratio between the learned invariant set and the smoothed safe set, which serves as an upper bound enforced by the architecture.Mean In the ablations, scalarization with λ = 0.2 reduces safety violations but decreases volume. In contrast, intelligent sampling lowers residuals and safety violations while increasing volume, suggesting it mitigates the tradeoff of scalarization. Among the baselines, directly enforcing the CBF condition (2) with L 2 CBF results in a highly safe but ineffective filter with zero volume. NCBF and DeepReach achieve high volume but with significantly higher safety errors. Next, we will examine how these methods perform on a more complex fixed-wing plane system.|N (h, c)| over L + 0 (c) Mean L CBF over L + 0 (c) Vol(L + 0 (h θ ))/Vol(L + 0 (c)) L HJ 0.The same comparisons conducted for the Double-Integrator are also performed with the following adjustments: all methods utilize a 7-100-100-1 MLP architecture. DeepReach includes an additional input for time. Additionally, validation is carried out on 10 7 uniformly random sampled states instead of grid samples.Mean Training with L 2 CBF is ineffective, yielding nearly zero volume. The performance of NCBF severely declines due to the increased difficulty of sampling for counterexamples in higher dimensions. In contrast, DeepReach achieves higher volume at the cost of increased safety violations. Our method, PINN-CBF, achieves both low conservatism and safety. To illustrate this, we visualize the performance of PINN-CBF, NCBF, and DeepReach in filtering a nominal trajectory representing takeoff.Fixed-Wing Aircraft Example In Fig. 3 , the fixed-wing aircraft starting from the given position collides with a 3D obstacle, while in Fig. 4 , obstacles are avoided with the PINN-CBF safety filter.  
conclusion and limitations: 
 We introduced a self-supervised framework for learning control barrier functions (CBFs) for limitedactuation systems with complex safety constraints. Our approach maximizes the control invariant set volume through physics-informed learning on a Hamilton-Jacobi (HJ) PDE characterizing the viability kernel. Additionally, we proposed a neural CBF parameterization that leverages the PDE structure.A key limitation of learning-based CBFs is the lack of guarantees on the control invariant set, as well as the need for many samples, similar to physics-informed methods. Our intelligent sampling (IS) strategy mitigates this to some extent, but the approach remains restricted to specific domains and obstacle configurations. Future work will explore domain-adaptive CBFs that generalize across different obstacle settings, building on recent advances in physics-informed techniques and further investigating links between CBF learning, HJ reachability analysis, and Control Lyapunov Value Functions [44] . 
 
 Appendix: 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 


Decision: The paper introduces an approach to learning control barrier functions focused on improving the performance of the system by maximizing the volume of the control invariant set. They utilize HJ-PDE inspired training loss to achieve this. The proposed formulation handles multiple safety constraints by utilizing a composition of safety constraints followed by smoothening to get an inner approximation of the safe set.

Strengths
- The paper is well-written and easy to follow. The technical content is clearly presented.
- The proposed framework is well-motivated by theoretical analysis of the structure of the CBVF.

Limitations

- Several reviewers have raised concerns regarding the novelty and contribution of the proposed approach over existing methods. 

- The experiment results do not demonstrate a clear advantage achieved by the proposed modifications over existing approaches. Further quantitative comparison is required to establish the utility of the proposed approach.

- The experiment results presented as it is somewhat confusing. It will help to separate out the baseline comparisons and ablation studies, clearly marking the winning approach.

- The authors should discuss the limitations of the proposed approach, such as the effect of using a learning-based approach for safety.

**Comments post rebuttal**

The authors have successfully addressed reviewers' concerns. The new experiments conducted by the authors were particularly useful in demonstrating the utility of the proposed method. The authors should incorporate the additional experiment results into the revised paper to make it more solid and convincing. In addition, the authors should include a metric for comparison that is more interpretable than a loss term (e.g., some sort of % safety rate?) in the final version.

Reviews: review: title: title: Official Review of Paper404
paper_summary: paper_summary: The paper introduces a novel approach towards learning control barrier functions focused on improving the performance of the system by maximizing the volume of the control invariant set. They utilize HJ-PDE inspired training loss to achieve this. Their formulation handles multiple safety constraints by utilizing composition of safety constraints followed by smoothening to get an inner approximation of the safe set.
main_review: main_review: Overall, the presentation of the paper is very clear but there are typos at multiple places. The motivation of the paper is also reasonable as there is a need of control barrier functions focused on improving performance of the system as CBFs usually tend to be conservative in nature and hamper system's performance. Until now, researchers have mostly focused on tuning class-K function to achieve this. The paper is theoretically sound and is the first to combine HJ reachability theory and control barrier functions theory when trying to learn neural network inspired control barrier functions. In table 1, the Neural Control barrier function (NCBF) method still outperforms the proposed method in terms of volume maximization which is a bit concerning as this was the main focus of the paper. Also, there are no hardware results for the proposed formulation. Since the CBF formulation is neural network inspired, 100% safety cannot be guaranteed and therefore the paper should address the safety rates for different formulations in Table 1 (see questions).
questions: questions: 1. Since the paper utilizes a neural network based control barrier functions, 100% safety cannot be guaranteed. It would be great if the paper compares the safety rate of the different formulations listed in table 1. Does performance oriented CBF have lesser or more safety rate compared to other methods?

2. What effect the does the discount factor $\gamma$ have on the learned CBF. what value was chosen for implementation and was that kept the same throughout the training?

3. In section 3.2 what does each input $y_i$ correspond to. This is a bit confusing. They should clearly specify it and relate it to terms in section 3.1.

4. In Table 1, the volume of ${L_0}^{+}(h_{\theta})$ is higher for NCBF as compared to $\mathcal{L_{HJ}}$. Could you provide a reason for that?

5. The limitations of the proposed method are not mentioned. Please add that as well.

6. There are multiple typos in the paper which should be addressed.
limitations: limitations: 1
review_summary: review_summary: Please read the review above.
title: title: Official Review by 6drv
paper_summary: paper_summary: Synthesizing CBFs that maximize the resulting control invariant set for nonlinear high-dimensional systems remains an open challenge. The authors propose several useful modifications to existing learning-based reachability frameworks to stabilize and improve the training of a specially parameterized neural CBF. The efficacy of the proposed framework is demonstrated on a 2D double integrator system and a 7D fixed-wing aircraft system.
main_review: main_review: $\textbf{Quality and Clarity:}$ The paper is well-written, and the technical content is clear and concise. The reviewer appreciates the complete section on related works.

$\textbf{Originality:}$ The contributions of the paper include (1) the construction of a smooth minorizer of the constraint function, (2) a special parameterization of the neural CBF and an associated training scheme, and (3) the formulation of a self-supervised training loss that combines the HJ reachability and CBF objectives. (1) The authors note that the smooth composition of constraint functions is taken from [1], so the originality is unclear to the reviewer, apart from being applied in this specific framework. (2) As far as the reviewer understands, the proposed parameterization of the neural CBF and the proposed training scheme (e.g., initializing weights of last layer to zero, sampling training data only for nontrivial states), constitute a novel contribution. (3) The augmented training loss is only a slight modification of the well-established self-supervised HJ PDE loss used in, e.g., DeepReach [2], which lacks originality in the reviewer's opinion. Overall, the originality of the work is modest.

$\textbf{Significance:}$ The significance of the proposed smooth minorizer of the constraint function is unclear to the reviewer, since it is not compared to, e.g., the commonly used minimum signed distance function for HJ reachability problems in the experiments. The advantages of the proposed parameterization of the neural CBF is also unclear, since the results are not compared to existing neural baselines. The improvement achieved by using the proposed augmented training loss instead of the established HJ PDE loss is not clearly demonstrated by the experiment results. 

$\textbf{Strengths:}$
 - The proposed framework is well-motivated by theoretical analysis of the structure of the CBVF.
 - The quality of the text and technical content is high.
 - The experiment results and analysis are well-presented.

$\textbf{Weaknesses:}$
 - The experiments section lacks comparisons to existing state-of-the-art baselines for synthesis of the CBVF.
 - The experiment results do not demonstrate a clear advantage achieved by the proposed modifications versus using the already established self-supervised HJ PDE loss.
 - The paper does not have a limitations section.

[1] Molnar, Tamas G., and Aaron D. Ames. "Composing control barrier functions for complex safety specifications." IEEE Control Systems Letters (2023).

[2] Bansal, Somil, and Claire J. Tomlin. "Deepreach: A deep learning approach to high-dimensional reachability." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021.
questions: questions: - How does using the proposed smooth minorizer of the constraint function compare to using, e.g., the minimum signed distance function commonly used in HJ reachability problems?
 - The reviewer strongly suggests including a comparison of the proposed framework with an existing baseline to demonstrate the improvements achieved by the proposed parameterization of the neural CBF and proposed training scheme.
 - What key advantages does the CBVF formulation have compared against the HJ reachability value function formulation?
 - How are the algorithm hyperparameters $\lambda, \beta, \gamma$ selected?
limitations: limitations: 1
review_summary: review_summary: Although each proposed modification to the learning-based synthesis of the CBF is modest, the combined framework will likely serve as an illustrative baseline for future works. The paper is also greatly strengthened by high-quality writing, presentation, and context. For these reasons, I weakly recommend that the paper is accepted.
title: title: Review of CoRL submission 404
paper_summary: paper_summary: This paper proposes to learn a neural network control barrier function to achieve maximal performance by solving the HJ PDE using CBVF. To find the inner approximation of safe set, logical operators are adopted to construct state constraints. Experiments are conducted on two simulation examples.
main_review: main_review: The paper is well-written and is easy to follow. Most of the notations and equations are self-contained and detailed. I like the CBF parameterization of Eq (10). But there are some weaknesses below.

- Some of the contributions seem to be incremental. The inner approximation is actually the log-sum-exp softmax in literature [14]. HJ PDE is also based on CBVF in previous work [32]. Moreover, in terms of PINN, the HJ loss in Eq. (12b) is the typical PDE residual error and it seems that there is no boundary condition loss to solve a well-posed problem, as it usually does in learning to solve PDE.

- I did not see the evidence of claimed "self-supervised training" because it seems that in Alg. 1, the pipeline still follows data collection and model training using ERM, which is typically supervised learning.

- In the experiments, Table 1 and Table 2 are confusing, and it is hard to tell which method performs the best. More importantly, I cannot find evidence that the proposed method significantly outperforms the baseline methods. It is expected to separate the ablation study with the comparison with baselines to better show the superiority of the proposed method.

- The fixed-wing aircraft example just gives a qualitative result and it is expected to have more quantitative results for comparison. Also, it is better to submit a video to show the results more clearly instead of Figure 7 and 8.

- Figure 1 needs more legends and notations to show the meaning of each component. There is a wrong reference on Line 231 with "??".
questions: questions: See weakness above.
limitations: limitations: 1
review_summary: review_summary: Incremental contribution, unconvincing experiment, but more convincing after rebuttal