A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Authors: Manan Tayal
Aditya Singh
Shishir Kolathaya
Somil Bansal
Venue: ICML 2025 poster (2025)
Keywords: Safety-Performance co-optimization
Safety-critical controls
Physics-informed ML
conformal prediction
Primary area: applications->robotics
TL;DR: This paper presents a novel approach to co-optimize safety and performance in autonomous systems using Physics- Informed Machine Learning

Abstract: As autonomous systems become more ubiquitous in daily life, ensuring high performance with guaranteed safety is crucial. However, safety and performance could be competing objectives, which makes their co-optimization difficult. Learning-based methods, such as Constrained Reinforcement Learning (CRL), achieve strong performance but lack formal safety guarantees due to safety being enforced as soft constraints, limiting their use in safety-critical settings. Conversely, formal methods such as Hamilton-Jacobi (HJ) Reachability Analysis and Control Barrier Functions (CBFs) provide rigorous safety assurances but often neglect performance, resulting in overly conservative controllers. To bridge this gap, we formulate the co-optimization of safety and performance as a state-constrained optimal control problem, where performance objectives are encoded via a cost function and safety requirements are imposed as state constraints. 
We demonstrate that the resultant value function satisfies a Hamilton-Jacobi-Bellman (HJB) equation, which we approximate efficiently using a novel physics-informed machine learning framework. In addition, we introduce a conformal prediction-based verification strategy to quantify the learning errors, recovering a high-confidence safety value function, along with a probabilistic error bound on performance degradation. Through several case studies, we demonstrate the efficacy of the proposed framework in enabling scalable learning of safe and performant controllers for complex, high-dimensional autonomous systems.

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Manan Tayal !
Adiyya Singh"
Shishir Kolathaya ! Somil Bansal 2
Abstract
1. Introduction
As autonomous systems become more ubiquitous in daily life, ensuring high performance with guar- anteed safety is crucial. However, safety and per- formance could be competing objectives, which makes their co-optimization difficult. Learning- based methods, such as Constrained Reinforce- ment Learning (CRL), achieve strong perfor- mance but lack formal safety guarantees due to safety being enforced as soft constraints, lim- iting their use in safety-critical settings. Con- versely, formal methods such as Hamilton-Jacobi (HJ) Reachability Analysis and Control Barrier Functions (CBFs) provide rigorous safety assur- ances but often neglect performance, resulting in overly conservative controllers. To bridge this gap, we formulate the co-optimization of safety and performance as a state-constraiied optimal control problem, where performance objectives are encoded via cost function and safety re- quirements are imposed state constraints. We demonstrate that the resultant value function sat- isfies a Hamilton-Jacobi Jaco---lloo-i (HJB) equation, which we approximate efficiently using a novel physics-ifformed machine learning framework. In addition, we introduce a conformal prediction- hased verification strategy to quantify the learning errors, recovering a high-confidence safety value function, along with probabilistic error bound on performance degradation. Through several case studies, we demonstrate the efficacy of the proposed framework in enabling scalable learning of safe and performant controllers for complex. nigh-dimensional autonomous systems.
Autonomous systems are becoming increasingly prevalent across various domains, from self-driving vehicles and robotic automation to aerospace and industrial applications. Designing control algorithms for these systems involves balancing two fundamental objectives: performance and safety. Ensuring high performance is essential for achieving efficiency and task objectives under practical constraints, such as fuel limitations or time restrictions. For instance, warehouse humanoid robot navigating to destination must optimize its route for efficiency. At the same time, safety remains paramount to prevent catastrophic accidents or system failures. These two objectives, however, often conffict, making it challenging to develop control strategies that achieve both effectively.
A variety of data-driven approaches have been explored to integrate safety considerations into control synthesis. Con- strained Reinforcement Learning (CRL) methods (Altman, 1999; Achiam et al., 2017) employ constrained optimization techniques to co-optimize safety and performance where performance is encoded as a reward function and safety is formulated as constraint. These methods often incorporate safety constraints into the objective function, leading to only soft imposition of the safety constraints. Moreover, such formulations typically minimize cumulative constraint vio- lations rather than enforcing strict safety at all times, which can result in unsafe behaviors.
Another class of methods involves safety safetyfiltering (Hsu et al., 2024), which ensures constraint satisfaction by modifying control outputs in real-time. Methods such as Control Bar- rier Function (CBF)- quadratic programs (QP) (Ames et al., 2017) and Hamilton-Jacobi (HJ) Reachability filters (Borquez et al., 2024; Wabersich et al., 2023) act as correc- tive layers on top of a (potentially unsafe) nominal controller, making minimal interventions to enforce safety constraints. However, because these safety filters operate independently of the underlying performance-driven controller, they often lead to myopic and suboptimal decisions. Alternatively, online optimization-based methods, such as Model Predic- tive Control (MPC) (Gareía et al., 1989; Grüne et al., 2017) and Model Predictive Path Integral (MPPI) (Williams et al., 2018; Streichenberg et al., 2023), can naturally integrate
'Equal contribution 'Center for Cyber Physical Systems, Indian Institute of Science, Bangalore, India DDepartment of Aeronautics and Astronautics, Stanford University, USA. Correspondence to: Manan Tayal <manantayal wwwpoaallllisicciiiii.i. Aditya Singh <adityas- ingh@iisc.ar inn..isc....i...
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

A Physics-Informee Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
safety constraints while optimizing for performance objec- tive. These methods approximate infinite-horizon optimal control problems (OCPs) with receding- horrzon frame- work, enabling dynamic re-planning. While effective, solv- ing constrained OCPs online remains computationally ex- pensive, limiting their applicability for high-frequency con- trol applications. The challenge is further exacerbated when dealing with nonlinear dynamics and nonconvex (safety) constraints, limiting the feasibility of these methods for ensuring safery and optimality for real-world systems.
formulation, we leverage physics-informed machine learn- ing (Raissi et al., 2019a; Li et al., 2022) learn solution to the resultant HJB-PDE by minimizing PDE residuals. This enables us to efficiently scale epigraph computation for higher-dimensional autonomous systems, leading to safe and performant policies. To summarize, our main contribu- tions are as follows:
We propose a novel Physies-Informed Machine Learn- ing (PIML) framework to learn policies that co- optimize safety and performance for high-dimensional autonomous systems.
A more rigorous approach to addressing the trade-off be- tween performance and safety is to formulate the problem as a coate-constrained optimal control problem (SC-OCP), where safety is explicitly encoded as hard constraint, while performance is expressed through reward (or cost) func- tion. While theoretically sound, characterizing the solu- tions of SC-OCPs is challenging unless certain control- lability conditions hold (Soner, 1986). To address these challenges, (Altarovici et al., 2013) proposed an epigraph- based formulation, which characterizes the value function of an SC-OCP by computing its epigraph using dynamic programming, resulting in a Hamilton-Jacobi-eellaa Par- tial Differential Equation (HUB-PDE). The SC-OCP value function as well as the optimized policy are then recovered from this epigraph. However, dynamic programming suffers from the curse of dimensionality, making it impractical for high-dimensional systems with traditional numerical solvers (Mitchell, 2004; Wang et al., 2024). Furthermore, the epi- graph formulation itself increases the problem's 's dimension- ality, exacerbating computational complexity further. Many techniques for speeding up the computation of solutions to the HJB PDE put restrictions on the type of system allowed (Chow et al., 2017). However, solving the HJB PDE for general nonlinear systems remains a key challenge.
We introduce conformal prediction- safety veri- fication strategy that provides high-confidence proba- bilistic safety guarantees for the learned policy, reduc- ing the impact of learning errors on safety.
We propose a performance quantification framework that leverages conformal prediction to provide high- confidence probabilistic enror bounds on performance degradation.
Across three case studies, we showcase the effective- ness of our proposed method in jointly optimizing safety and performance, while scaling to complex, high-dimensional systems.
2. Problem Setup
Consider a nonlinear dynamical system characterized by the state x E X S R" and control input u E u S IR"" governed by the dynamics (t) f(x(t), u()). where the function f : R x R RM is locally Lipschitz contin- uous. In this work, we assume that the dynamics model f is known; however, it can also be learned from data if unavailable.
Recent advances in Deep Learning have enabled the devel- opment of physics-informed machine learning approaches (Raissi et al., 2017; 2019b) for solving partial differential equations (PDEs) with neural networks. These methods have demonstrated notable effectiveness in addressing high- dimensional PDEs while ensuring that the learned solu- tions adhere to the governing physical laws. In particular, DeepReach (Bansal & Tomlin, 2021) proposes a frame- work for solving nammitoon-acobi-Bllllle (HJB) PDEs in safety-critical settings using thysics-informer machine learning. However, its exclusive focus on safety neglects performance considerations, resulting in overly conservative control strategies.
We are given a failure set F C X' that represents the set of unsafe states for the system (e.g., obstacles for an au- tonomous ground robot). The system's 's performance is quan- tified by the cost function C(t, x. u), given by:
ff" ((r(s)) ds + ((x(T)), (I)
C(t,z(t),u) =
where : X + R20 and Φ X + Rg are Lipschitz continuous and non-negative functions, representing the running cost over the time horizon [t, T) and the terminal cost at time T, respectively. u : [t, T) + U is the control signal applied to the system. Using this premise, we define the main objective of this paper:
In this work, we propose a novel algorithmic approach to co-optimize safety and performance for high-dimensional autonomous systems. Specifically, we formulate the prob- lem as an SC-OCP and leverage the epigraph formulation in (Altarovici et al., 2013). To efficiently solve this epigraph
Objective 1. We aim to synthesize an optimal policy n" : [t, x X + U that minimizes the cost function C' while ensuring that the system remains outside the failure set F all times.
2

---

A Physics-Informan Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
2.1. State-Constrained Optimal Control Problem
Furthermore, facilitate solving the epigraph reformula- tion, can be treated as state variable, with its dynamies given by ²()) = -I(x(t)). This implies that the trajec- tory progresses over time, the minimum permissible cost, 2, decreases by the step cost ((x: at each time step. This allows us to define an augmented system that evolves according to the following dynamics:
To achieve the stated objective, the first step is encode the safety constraint via function g : R" + R such that, :=
== {z E X g(x) > 0}. Using these notations, the ob- jective can be formulated as the following contrroostt Optimal Control Problem (SC-OCP) to compute the value function V:
v(t) = min J. ((a(s))ds+ ((x(T))
== f(, 2r u) = f(t, x, u)
-(xx) (
(5)
u
(2)
s.t. t = f(z,u),
where == = 2] represents the augmented state. With the augmented state representation and under assumptions A1- of (Altarovici et al., 2013), the auxiliary value func- tion (t,,=(() (()) is unique continuous viscosity so-
g(x(s)) = v [t
This SC-OCP enhances the system's performance by mini- mizing the cost, while maintaining system safety through the state constraint, gxx) < 0, ensuring that the system avoids the failure set, F. Thus, the policy, n², derived from the solution of this SC-OCP co-optimizes safety and performance.
lution satisfying the following Hamillon-JacoiiBellle (HJB) PDE:
min (ann-O.V 0.V V ((2,,,
v- )(x)) = 0.
(6)
2.2. Epigraph Reformulation
VI E (0, and & E X x IR, where (.--) denotes the dot product of vectors. The boundary condition for the PDE given by:
Directly solving the SC-OCP in (2) presents significant challenges due to the presence of (hard) state constraints. To address this issue, we reformulate the problem in its epigraph form (Boyd & Vandenberghe, 2004), which trans- forms the constrained optimization into more tractable two-stage optimization problem. This reformulation allows us to efficiently obtain a solution to the SC-OCP in (2). The resulting formulation is given by:
VTT = max -(,((T))2,g((z))) E R. (7)
Note that by slight abuse of notations, we have replaced the arguments x, z for V with the augmented state ă.
3. Methodology
(tt, (1)) = min 2
To solve the SC-OCP in Equation (2), we aim to compute the optimal value function V, which minimizes the cost while ensuring system safety. In this section, we outline a struc- tured approach: first, we learn the auxiliary value function V using a thysics-informed machine learning framework. Then, we apply conformal prediction-based method to ver- ify safety and correct for potential learning errors in V. The final value function V is obtained from the safety-corrected V using the epigraph formulation in (3). Lastly, we assess the performance of V through a second conformal predic- tion procedure. Figure gives an overview of the proposed approach. The following subsections provide detailed ex- planation of each step, beginning with the methodology for learning V.
ZER
s.t. 2t
where z is non-negative auxiliary optimization variable, and V represents the auxiliary value function. Here, V is defined as (Altarovici et al., 2013):
(t, = min max (atx(((ttx(t)))))))) - 2, max (((())) (E(T.T)
(4)
Note that if (1, O, it implies that g(x(s)) < 0
for all E [t, In other words, the system must be outside the failure set at all times; therefore, the system is guaranteed to be safe whenever (tt, 2) < O.
In this reformulated problem, state constraints are effectively eliminated, enabling the use of dynamic programming to characterize the value function, as we explain later in this section. Intuitively, optimal z (z") can be thought of as the minimumm permissible cost the policy can incur without compromising on safety. From Equation 3, it can be inferred that if 2 > 2", the safety constraint dominates in the max term, resulting in a conservative policy. Conversely, if 2 < 2", the performance objective takes precedence, leading to a potentially aggressive policy that might compromise safety.
3.1. Training the Auxiliary Value Function (V)
The auxiliary value function, V. satisfies the HJB-PDE in Equation (6), as discussed in Section 2.2. Traditionally, numerical methods are used to solve the HJB-PDD over grid representation of the state space (Mitchell, 2004; Schmerling, 2021), where time and spatial derivatives are approximated numerically. While baidd methods are accurate for dimediinlll problems, they struggle with
3

---

A Mmysics-Informee Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Step 1: Training
Step 2: Safety Verification
SC-OCP
Training
A Induced Safe Policy
Ri
Backwoadd Propagalion
Rollouts
(1k ak [
plrte)) Yee
Value currection level
Ausillory Valte Furntion
Step 4: Performance Quantification
Step 3: Inference
T
Safe and Performant Value Function Vo(t.a)
Induced Policy
Rollouts
Performant Policy
(V, ) Vs.
s.t. Volt, 2) &
V8
Crnaz
Performance Qumtification level.
(o(t,a) arg
Figure Overview the proposed approach: The methodology is organized into four steps. The first step involves training the auxiliary value function, Ve. using physics-informed informed machine learning framework. The second step applies conformal prediction approach for safety verification of the learned V.. In the third step, the final value function Ve and the optimal safe and performant policy ma are inferred. The fourth step quantifies the performance of Va through second conformal prediction procedure.
the curse of dimensionality- theii computational complex- ity increases exponentially with the number of states lim- iting their use in digh-dimensiona systems. To address this, we adopt physics-informed machine learning framework, inspired by (Bansal & Tomlin, 2021), which has proven effective for high-dimensional reachability problems.
where Ht,, 2)) min (t, 2), f(ă, u)) and (ii) sat- isfaction of the boundary condition in (7), using boundary condition loss, given by:
Live (tk, (x) = ||max (Ø(xx) 2
2k. ((*k))
(9)
Vo (tx, an) (t+ -T).
The solution of the HJB-PDE inherently evolves backward in time, the value function at time t is determined by its value + At. To facilitate neural network training, we use a curriculum learning strategy, progressively expanding the time sampling interval from the terminal time [T, ' to the full time horizon [0, T]. This approach allows the neural network to first accurately learn the value function from the terminal boundary conditions, subsequently propagating the solution backward in time by leveraging the structure of the HJB-PDE.
These terms are balanced by trade-off parameter X, leading to the overall loss function:
L (tx, St ( = Lpde (tx. 2x () + ALse #x
(10)
Furthermore, we use the adaptive loss re-balancing scheme proposed in (Wang et al., 2021) to reduce the impact of X on the learned value function. Minimizing the overall loss function provides a self-supervised learning mechanism to approximate the auxiliary value function.
Specifically, the auxiliary value function is approximated by neural network, Vo. where denotes the trainable pa- rameters the network. Training samples, (tk, Tky zι N are randomly drawn from the state space based on the cur- riculum training scheme. The proposed learning framework utilizes loss function that enforces two primary objectives: (i) compliance with the PDE in (6), using the PDE residual error given by:
3.2. Safety Verification
The learned auxiliary value function, Vo. induces policy, ita that minimizes the Hamiltonian term H (t, 2) in the HJB-PDE. The policy is given by:
at .)) = arg min min (VVet, 2), f(t u)). aell
(1I)
The rollout cost corresponding this policy is defined as:
Vz, (t. max(C(t,xx()) u) max g(z(s)))
Lepde (lx, tx (0) = min {-0. (1,, xx)) H(1,,
sE(t.T]
u=
(12)
Vo (ta, (xx) ll.
Ideally, the rollout cost from a given state under म. should match the value of the auxiliary value function at that state.

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Algorithm 1 Safety Verification using Conformal Prediction
Algorithm 2 Performance Quantification using Conformal Prediction
Require: S, Ne. Sss cs. (á 0). V*o (ē, 0), M (number of 5-levels search for 8).
Require: S", Np, 3p. (z 0), V (zx, 0) I: D + Sample Np IID states from x E 8*
1: Du Sample N₂ IID states from S5=0
2: ón < no (V, .t)) Va, (0, (0, 2 0)
2: for i = 0, 0.1.....
do
3: P. t (),
3: 60 - (14) (using a6=0)
4: A + Ordered list of M uniform samples from [ōo. 0]
end for
5: P P sorted in decreasing order
5: for 1,,,..., M
do
6: ap f N,+T 40 f- Po. 6o f (18)
6: while 6, S E, do
7:
8; + A;
for = 10.1..... Np. do
8: Update as, from 6;
8: while 6 S c do
9: < (14) (using (ss.)
Op No v Pa. C G (18)
i+1
10: end while
10: end while
11: end for
11: end for
12:
return 5g 8,
12: return 2b 1/
However, due to learning inaccuracies, discrepancies can arise. This becomes critical when state, 2t, is deemed safe by the auxiliary value function (V& (t, .t) < 0) but is unsafe under the induced policy (V=, (t, .) > 0). To address this, we introduce a uniform value function correction margin, 6, which guarantees that the sub-ò level set the auxil- iary value function remains safe under the induced policy. Mathematically, the optimal 5 (6*) can be expressed as:
Algorithm I presents the steps to calculate d using the ap- proach proposed in this theorem.
3.3. Obtaining Safe and Performant Value Function and Policy from Vi
Using the d-leell estimate from Algorithm (1), we can fi- nally obtain the safe and performant value function, (t,,,,;, by solving the following epigraph optimization problem:
8 = min (V (0, VR (0,4) 200 0}
(13)
SEX
Vo(t, min =ER
Intuitively, 5t identifies the tightest level of the value fune- tion that separates safe states under Aoa from unsafe ones. Hence, any initial state within the sub-ö* level set is guar- anteed to be safe under the induced policy, ft. However, calculating 5* exactly requires infinitely many state-space points. To overcome this, we adopt a conformal- prediction- conformal-prediction- based approach approximate 8* using a finite number of samples, providing probabilistic safety guarantee. The following theorem formalizes approach:
(16)
Vo(1, 5
Note that (t( trivially xo for the states where (t,,,,, 22, > 8, since such states are unsafe and hence do not satisfy the safety constraint.
In practice, we solve this optimization problem by using binary search approach on z. The resulting optimal state- feedback control policy, To X x [t, + U. satisfying Objective (1), is given by:
Theorem 3.1 (Safety Verification Using Conformal Predic- tion). Let Sy be the set of states satisfying (o 2) S 6, and let (0, d.).......N. be N₂ i.idd. samples from Sz. Define as as the safety error rate among these N, samples for a given 8 level. Select a safety violation parameter e, 10, and confidence parameter 3r (0, 1) such that:
To (l, arg min (VV, f("", u)),
(17)
where 4" is the augmented state associated with the optimal
2* obtained by solving (16), i..,, x- = [x, 2" Intuitively, we can expect no to learn behaviors that best tradeoff the safety and performance of the system.
é (M))
Σ
1-1
No
No SB..
(14)
İ=U
3.4. Performance Quantification
Then, with the probability of at least 1 - Ba, the following holds:
where 1 = [(N, + 1.
In general, the learning inaccuracies in the auxiliary value function Vo. may lead to errors in the value function Vo. These errors, in turn, can lead to performance degradation under policy mo-
(V(0, S 5)) 1-
P aeS;
(15)
The proof is available in Appendix 1.1. The safety error rate as is defined as the fraction of samples satisfying Vo S 5 and Vz, 2 0 out of the total N₂ samples.
To quantify this degradation, we propose a conformal prediction-basee performance quantification method that provides probabilistic upper bound on the error between
5

---

A Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Our Method
MPPI
MPPI-CBF
SAC-Lag
PPO-Lag
CPO
Mean higher cost compared to our method
50
100
150
200
56
108
159
200
50
100
150
200
Safety Rate
6
70
100 70
80
98 100 70
100
Boat Navigation
Evader Chasing
Multi Agent Navigation
Figure This figure presents comparative study between all the methods based on our evaluation metries. The top plot illustrates the mean percentage increase in cumulative cost relative to our method for each baseline, demonstrating that our approach consistently incurs lower costs, with gap widening as system complexity grows. bottom plot depicts safety rates, showing our method maintains 100% safety rate, while baselines that encourage safety rather than enforcing (like MPPI and C-SAC) achieve lower rates. MPPI-CBF also attains 100% safety at the expense performance. Overall, our method uniquely balances both safety and performance, whereas the baselines compromise least one aspect.
the value function and the value obtained from the induced policy. The following theorem formalizes our approach:
4. Experiments
The objective of this paper is to demonstrate the co- optimization of performance and safety. To achieve this, we evaluate the proposed method and compare them with baselines using three metries: (I) Cumulative Cost: This metric represents the total cost Je [xx(s))ds+ ((x(T)), ac- cumulated by a policy over the safe trajectories. (2) Safety Rate; This metric defined as the percentage of trajecto- ries that remain safe, i.e., never enter the failure region FF at any point in time, (3) Computation Time: This metric compares the offline and online computation times of method and the baselines.
Theorem 3.2 (Performance Quantification Using Confor- mal Prediction). Suppose S* denotes the safe states satis- fying (0, < oo (or equivalently (0, it) < 5) and (0, (0)=:))=,, are Np i.idd samples from S*. For a user- specified level atp, let be the ([N++1((I-ap) th quantile of the scores (p: := [Vo(0,,.)-V,,(0..) (0.x0) Np on the Np state samples. Select a violation parameter 6p (0, 1) and confidence parameter Bp E (0, 1) such that:
Cmaz
NND) (1:)
Σ
(1 - -0 SBp
(18)
We consider categories of baselines: the first set of methods aims enhance the system performance (i.e., minimize the cumulative cost) while encouraging safety, en- compassing methods such as Lagrangian- basdd CRL algo- rithms like SAC-Lagrangian (SAC-Lag), PPO-Lagrangian (PPO-Lag) (Ray et al., 2019; et al., 2024) and Model Predictive Path Integral (MPPI) (Williams et al., 2018) algo- rithms. The second category prioritizes safety, potentially at the cost of performance. This includes Constrained Pol- icy Optimization (CPO) (Achiam et al., 2017) and safety filtering techniques such as Control Barrier Function (CBF)- based quadratic programs (QP) (Ames et al., 2017) that modify a nominal, potentially unsafe controller to satisfy the safety constraint.
Baselines:
where, 1 [(N, + 1)ap). Then, the following holds, with probability 1- 3p:
Vo x.) VA (0. 2:)) I {Φ 2 1111p. (19)
.ESS
Cmaz
where Cmaz normalizing factor and denotes the maxi- possible cost that could be incurred for any E St
The proof is available in Appendix A.2. Note that Omea can be casily calculated by calculating the upper bound of the cost function C(t, (t)), u)vz E S
Intuitively, the performance of the resultant policy is the best when the v value approaches while the worst performance occurs 1/ Algorithm 2 presents the steps to calculate ab using the approach proposed in this theorem.
4.1. Efficient and Safe Boat Navigation
In our first experiment, we consider 2D autonomous boat navigation problem, where boat with coordinates (ara, yb)
6

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Method
MPPL-CHF
SAC-Lag
Our Method
MPPI
MPPI-CBF
SAC-Log
2
Evader Trajectory
y/b
y
-2
-3
C
Figure 3. Trajectories from two distinct initial states shown, with dark grey circles representing obstacles and the green dot indicating the goal [1.5, 0)"". Notably, method the only one that successfully approaches the goal while adhering to safety constraints.
Figure 4. Trajectories from two distinct initial states are depicted. with dark grey circles representing obstacles and purple trajectories indicating the evader's path, with arrows showing its direction motion. Our method successfully tracks the evader while avoid- ing collisions, whereas other methods either fail to maintain safety, struggle to track the evader both
navigates river with state-dependent drift reach island. The boat must avoid two circular boulders (obstacles) of different radii, which corresponds to the safety constraint in the system (see Fig. 3). The cost function penalizes the distance to the goal. The system state, v, evolves according to the dynamics:
Set Toolbox (Mitchell, 2004) (results in Appendix B.1.1).
Additional baselines include: (1) MPPI, a sample-based path- both-planning algorithm with safety as soft constraints, (2) MPPI-NCBF, where safety is enforced using a Neural CBF- based QP with MPPI the nominal controller (Dawson et al., 2022; Tayal et al., 2024b), and (3) Constrained RL methods like SAC-Lag, PPO-Lag, and CPO.
I = [rb ybl, i [4 + 0.5y wel
(20)
where [u1, u2] are the bounded control inputs in the and y/b directions, constrained by the control space U = {[u1, u2] I2² [[[u,, uglll < 1}. The term 0.5y, introduces stateddepndeent drift, complicating the control task as the actions must counteract the drift while ensuring safety, which challenging under bounded control inputs. The rest of the details about the experiment setup can be found in the Appendix B.1.
Comparative Analysis: Figure 3 shows that our method effectively reaches the goal while avoiding obstacles, even when starting close them. In contrast, MPPI and CRL- based policies fail to maintain safety, while MPPI-NCBF ensures safety but performs poorly (leading very slow trajectories). Figure highlights that method outper- forms all others. SAC-Lag attains a mean cost that is 7.5% higher than ours, while exhibiting the lowest safety rate at 76%. The remaining CRL methods display comparable trends, highlighting their inability to jointly optimize for safety and performance. MPPI, with more competitive safety rate of 89%, performs poorly with 32.67% higher mean cost. MPPI-NCBF achieves 100% safety but performs significantly worse, with 50.72% higher mean cost. Addi- tionally, CBF-based controllers sometimes violate control bounds, limiting their applicability. This demonstrates that our method balances safety and performance, unlike others that compromise on one aspect. Moreover, the 100% safety rate of our method aligns closely with at least 99.9% safety level that we expect using our proposed verification strategy, providing empirical validation of the safety assurances.
Safety Guarantees and Performance Quantification: We use N₂ 300K and N, = 300K samples for thorough verification, ensuring dense state space sampling. For this experiment, we s, 0.001 and 8, 10-10 resulting in 5-level of O. This implies that, with 1 10-10 confidence, 0, is safe with at least 99.9%
any state with Vo probability. For performance quantification, we set Ep 0.01 and 3, = 10-10 10 leading to a vi-level of 0.136. This ensures, with 1 - 10 confidence, that any state in S* has a normalized error between the policy value of less than 0.136 with 99% probability. Low 8 and z values with high confidence indicate that the learned policy closely approximates the optimal policy and successfully co-optimizes safety and performance.
predicted value and
4.2. Pursuer Vehicle tracking a moving Evader
This being a 2-dimensional system, we com- pare our method with the ground truth value function com- puted by solving the HJB-PPD BDE numerically using the Level
Baselines:
In our second experiment, we consider an acceleration- driven pursuer vehicle, tracking a moving evader while

---

A Informedd Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
t= Os
t= ls
t t 2s
Figure 5. Snapshots of multi-agent navigation trajectories at different times using the proposed method. Agents are represented as circles with radius R, indicating the minimum safe distance they must maintain from each other. Smaller dots mark their respective goals, The trajectories show that agents proactively maintain long-horizon safety by adjusting their paths to avoid close encounters, rather than enforcing safety reactively, which could lead to suboptimal behaviors. Finally, the agents reach their respective goals within the specified time horizon.
avoiding five circular obstacles (see Fig- 4). This experiment involves 8-dimensional system, with the state x defined as x [Tp V/, u, O Fez Me: Une, Uyel"' where Tp: Mp: v, θ represent the coordinates, linear velocity, and orientation of the pursuer vehicle, respectively, and Te, Ve: Ure, oye rep- resent the coordinates and linear velocities of the evader vehicle. The pursuer vehiele is controlled by linear accel- eration (u1) and angular velocity (u2). The control space U {[uq, u2] E 21?}. The complexity of this sys- tem stems from the dynamic nature of the goal, along with the challenge of ensuring safety in a cluttered environment, which in itself is difficult safety problem. More details about the experiment setup are in Appendix B.2.
NCBF ensures 100% safety but has a 42% higher mean cost. SAC-Lag underperforms both in safety (66% safety rate) and performance (101% % higher mean cost). A similar trend is evident across all other CRL methods, indicating their difficulty in co-optimizing safety and performance in high-dimensional, complex systems.
4.3. Multi-Agent Navigation
In our third experiment, we consider a multi aulti--agett set- ting where each of the 5 agents, represented by x; [Ta, F Vo.l tries to reach its goal while avoiding collisions with others. (Iar Va.) denote the position of the ith agent, while (Ig,, Vg.) represent the goal locations for that agent. The system is 20-dimensional, with each agent controlled by its x and y velocities. The control space for each agent is 24; complexity of this system stems from the interactions and potential conflicts between agents as they attempt reach their goals while avoiding collisions. The rest of the details about the experiment setup can be found in Appendix B.3.
Safety Guarantees and Performance Quantification: Similar to the previous experiment, we set N₂ N, = 300k. We choose 6, = 0.01 and B, 10 yielding 5-level of -0.04 and safety level of 99% on the aux- iliary value function. For performance,
{[e,, "y.l I[[e,, 2/y S 1). The
set Ep 0.01 and B, 10-10, leading to a ri-level of 137. These val- ues indicate the learned policy maintains high safety with www-performance degradation in this cluttered environment.
Safety Guarantees and Performance Quantification: We set N, = N, = 300k, 6, = 0.001, and 3, = 10-10 resulting in a ô-level of 00.09 with safety assurance of 99.9% for the auxiliary value function. For performance set 6p 0.01 and B, 10 10-10, leading to a ve-level of 0.068. It is evident that the 5 and v/ values remain very low with high confidence, highlighting the effectiveness of our method in co-optimizing safety and performance for nigh-dimensional, multi-agent systems.
Baselines: As the previous experiment, we employ MPPI and CRL methods (SaC-Lag, PPO-Lag, and CPO). For safety filtering, utilize QP based on the collision cone CBF (C3BF) (Goswami et al., 2024), chosen for its effec- tiveness in managing acceleration-driier systems.
quantification,
Comparative Analysis: Figure 4 shows that our method ef- fectively tracks the moving evader while avoiding obstacles, even when starting close to them. contrast, other methods have limitations: MPPI and CRL methods attempt to follow the evader but fail maintain their pace, violating safety constraints, while MPPI-C3B sacrifices performance to maintain safety. Figure 2 highlights our method's superior performance in balancing safety and performance. MPPI achieves the best performance among the baselines but with an 18% higher mean cost and only 72% safety rate. MPPI-
Baselines: Similar previous experiments, we have used MPPI, SAC-Lag, PPO-Lag, CPO, and MPPI-NCBF as our basclines for this experiment too.
Figure 5 shows that our method ensures long-horizon safety while enabling all agents to reach their goals without collisions. In contrast, the base- line methods either exhibit overly conservative behavior or
Comparative Analysis:
8

---

A Informedd Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Our Method
MPPI
MPPI-CBF
SAC-Lag
PPO-I
CPO
Offine Computation Time (in min)
200
400
600
200
499
219
400
600
Online Computation Time (in sec)
8/ 0.2
0.6 8/
0.2
8.6 0.0
0,2
0.4 0.6
Boat Navigation
Evader Chasing
Multi Agent Navigation
Figure 6. This figure presents comparative analysis of all methods based on online and offline computation time evaluated on the same computing machine. The top plot illustrates offline computation time for our method and the baselines. Since our method and SAC-Lag involve training value functions, they incur higher oflline computation costs, whereas MPPI-based methods require no offline training. The bottom plot depicts the online computation time, demonstrating that our method and SAC-Lag have minimal online computation requirements, whereas MPPI-based methods exhibit significantly higher online computational costs.
fail to maintain safety, leading to collisions, as detailed in Appendix B.3.1. Figure 2 demonstrates the superior per- formance of our approach, with MPPI, MPPI-NCBE, and SAC-Lag showing mean percentage cost increases of 148%, 192%, and 164%, respectively. Although MPPI and MPPI- NCBF achieve competitive safety rates 90% and 100%, their significant performance degradation highlights their inability to balance safety and performance in complex sys- tems. MPPI's subpar performance stems from its reliance on locally optimal solutions in a finite data regime, leading to several deadlocks along the way and overall subopti- mal trajectories over a long horizon. Furthermore, CRL methods struggle with both safety and performance, further demonstrating their limitations in handling increasing sys- tem complexity and dimensionality. These results confirm our method's s ability to co-optimize safety and performance in high-ding sich-dimmnsional systems, demonstrating its scalability. Additionally, the safety guarantees hold in the test samples, validating the scalability of our safety verification frame- work for multi-agent systems.
sublinearly from 8D to 20D system. This scalability is key advantage of the proposed approach. We finally note that while offline training requires time, our method achieves real-time inference speeds, with optimal policy computed in just 2ms across all systems, making the approach highly suitable for real robotic systems.
5. Conclusion and Future Work
In this work, introduced physics-informed machine learning framework for co-optimizing safety and perfor- mance in autonomous systems. By formulating the problem as conttrinntt optimal control problem (SC-OCP) and leveraging an epigraph-based approach, we enabled scalable computation of safety-aware policies. Our method integrates conformal prediction safety verification ensure high-confidence safety guarantees while maintain- ing optimal performance. Through multiple case studies, we demonstrated the effectiveness and scalability of our approach in high-dimensional systems. In future, we will explore methods for rapid adaptation of the learned policies in light of new information about the system dynamics, en- vironments, or safety constraints. We will also apply our method other high-dimensional autonomous systems and systems with unknown dynamics.
4.4. Computation time Analysis
Figure 6 presents comparative analysis of the offline and online computation times for our method against the base- lines. While traditional grid-based methods suffer from an exponentially scaling computational complexity (and are completely intractable for the 8D Evader Chasing and 20D Multi-Agent case studies), the proposed method scales much better with the system dimensionality. For ple, the computation time increases only minimally from the 2D system to the 8D system, thanks to neural network parallelization. Similarly, the computation time increases
A Acknowledgements
Manan is supported by the Prime Minister's Research Fel- lowship (PMRF), Government of India. This work is par- tially supported by the AI Robotics Technology Park (ARTPARK) IISc, the DARPA Assured Neuro Symbolic

---

A Physics-Informec Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Learning and Reasoning (ANSR) program, and the NSF
Chen, J., Du, R., and Wu, K. A comparison study of deep galerkin method and deep ritz method for elliptic prob- lems with different boundary conditions. arXiv preprint WwXiv:2005.04555 2020.
CAREER award (2240163).
Impact Statement
Chow, Y. T., Darbon, J., Osher, S., and Yin, W. Algo- rithm for overcoming the curse of dimensionality for time-dependent non-convex hamilton- arising from optimal control and differential games prob- lems. Journal of Scientific Computing, 73:617-643, 2017.
This paper presents an approach to co-optimize safety and performance autonomous systems using Physics- Informed Machine Learning. This framework advances the deployment of scalable, provably safe, and high- performance controllers for complex, high-dimensional au- tonomous systems, with potential applications in robotics and autonomous driving.
equations
Darbon, and Osher, S. Algorithms for overcoming the curse of dimensionality for certain hamilton- equa- tions arising in control theory and elsewhere. Research in the Mathematical Sciences, 3(1):19, 2016.
References
Achiam, J., Held, D., Tamar, A., and Abbeel, P. Constrained policy optimization. In International conference on ma- chine learning, pp. 22-31. PMLR, 2017.
Dawson, C.,, Qin, Z., Gao, S., and Fan, C. Safe nonlinear control using robust neural lyapunov functions. In Conference on Robot Learning, pp. 1724-172 1135. PMLR, 2022.
Altarovici, A., Bokanowski, O., and Zidani, H. A gen- eral hamilton-jacobi framework for non-linear state- constrained control problems. ESAIM: Control, Optimi- sation and Calculus of Variations, 19(2):337-357, 2013.
Fotiadis, F. and Vamvoudakis, K. G. A physics-informed neural networks framework to solve the infinite-horizon optimal control problem. In 2023 62nd IEEE Conference on Decision and Control (CDC), pp. 6014-6019, 2023. doi: 10.1 1LL11///DCA973732222.....82
Altman, E. Constrained Markov Decision Processes. Stochastic Modeling Series. Taylor & Francis, 1999.
ISBN 9780849303821. URL https: /books. googl e. co. www.bokksssdie11sniioeee
García, C. E., Prett, D. M., and Morari, M. Model predictive control: Theory and practice-a survey, Automatica, 25(3))335- 348, 1989. ISSN 0005-1098. doi: http:///bioo////0000660600009099 URL http:s:/www..ccieccddieeetttt saience//rticaee/p///00118888999
Ames, A. D., Xu, X., Grizzle, J. W., and Tabuada, P. Control barrier function based quadratic programs for safety crit- ical systems. IEEE Transactions on Automatic Control, 62(8):3861-3874 2017. doi: 10. 2019/tac.201.26633
Angelopoulos, A. N. and Bates, S. A gentle introduction to conformal prediction and distribution-free uncertainty quantification, 2022. URL https: http///axxvvvo//// org/ abs/2107.0755
Goswami, B. G., Tayal, M., Rajgopal, K., Jagtap, P., and Kolathaya, S. Collision cone control barrier functions: Experimental validation on ugvs for kinematic obsta- cle avoidance. In 2024 American Control Conference (ACC), pp. 325-331, 2024. doi: 10.23919/ACC60939, 2024. 1064.36644338.
Bansal, S. and Tomlin, J. Deepreach: A deep learning approach to liigh-dimensiona reachability. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 1824, 2021. doi: 10.1109// 2021.9561944
Grüne, L., Pannek, J., Grüne, L., and Pannek, J. Nonlinear model predictive control. Springer, 2017.
Hsu, K.CC., Hu, H., and Fisac, J. F. The safety filter: A unified view of safety-critical control in autonomous systems. Annual Review of Control, Robotics, and Autonomous Systems. 7(Volume 7, 2024):47-72, 2024. ISSN 2573-5144. doi: https://do..rgg/0..146/
Borquez, J., Chakraborty, K., Wang, H., and Bansal, S. On safety and liveness filtering using hamilton-jacee jacobi reachability analysis. IEEE Transactions Robotics, 40: 4235-4251, 2024. doi: 10. 1..11099RRO... 201..3.00202.44444400
Boyd, S. and Vandenberghe, L. Convex optimization. Cam- bridge university press, 2004.
annneey-ontrro-077722--0004 URL https: annualreviews. htwpa//wweeeste..e.c..eetee////a
1 1146/ 1146/ htt5://nnnee--ccoro--01111111111
Chakrabarty, A., Jha, D. K., Buzzard, G. T., Wang, Y., and Vamvoudakis, K. G. Safe approximate dynamic program- ming via kernelized lipschitz estimation. IEEE Transac- tions on Neural Networks and Learning Systems, 32(1): 2021. doi: 10.1 10.1100 10.1109/TNI.S 20200.0022002222 2978859
Ji, J., Zhou, J., Zhang, B., Dai, J., Pan, X., Sun, R., Huang, W., Geng, Y., Liu, M., and Yang, Y. Om- nisafe: An infrastructure for accelerating safe reinforce- ment learning research. Journal of Machine Learning
10

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Research, 25(285):1-6, 2024. URL http://jmlr. arg/papers/ w25/23-
Soner, H. M. Optimal control with state-space constraint
html.
i. SIAM Journal Control and Optimization, 24(3):
552-561, 1986. doi: 10. 10.1137/0324032. URL https:
Li, Z., Zheng, H., Kovachki, N. B., Jin, D., Chen, H., Liu, B., Stuart, A., Azizzadenesheli, K., and Anand- kumar, A. Physics-informed neural operator for learn- ing partial differential equations, 2022. URL https: h //openreview net/ / for urmm? i d=dtt YnHcmQQeM.
//doi. //dbi.org/10.113100 11w./ 03
Streichenberg, L., Trevisan, E., Chung, J. J., Siegwart, R., and Alonso-Mora, J. Multi-agent path integral control for interaction-aware motion planning in urban canals. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 1379-1385, 2023. doi: 10.1 109/ (CRA48891.20233 221.220.....1111.. 10161511.
Mitchell, I. A toolbox of level set methods. http://ww.. cs. ubc. cahntcchel//ooloooSSttooobooo
2004.
Olver, F. W. J., Daalhuis, A. B. O., Lozier, D. W., Schneider, B. I. Boisvert, R. Clark, C. W., Miller, R., Saunders, B. V., Cohl, H. S., and M. A. McClain, NIST Digital Library of Mathematical Functions. National Institute of Standards and Technology, 2023. URL https::///// dlmf. nist. gov/. Release 1.1.11 of 2023-09-15.
Tayal, M., Singh, A., Jagtap. P., and Kolathaya, S. Semi- supervised safe visuomotor policy synthesis using barrier certificates. arXiv preprint arXiv:2409. 12616, 2024a.
Tayal, M., Zhang, H., Jagtap, P., Clark, A., and Kolathaya, S. Learning formally verified control barrier function in stochastic environment. In Conference on Decision and Control (CDC). IEEE, 2024b.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics informed deep learning (part i & ii): Data-driven solutions of nonlinear partial differential equations, 2017.
Vovk, V. Conditional validity of inductive conformal predic- tors, 2012. URL https: /awpiviivvv.////////a//////aav org/ abs/ 1209. 2673.
Raissi, M., Perdikaris, P., and Karniadakis, G. Physies- informed neural networks: A deep learning frame- work for solving forward and inverse problems involv- ing nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019a. ISSN 0021-9991. doi:
Wabersich, K. P., Taylor, A. J., Choi, J. J., Sreenath, K., Tomlin, C. J., Ames, A. D., and Zeilinger, M. N. Data- driven safety filters: Hamilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems. IEEE Control Systems Magazine, 43(5):137~ 177, 2023, doi: N0.1109/MCS.20
045. URL aeiiaeee/articeee/l////029999999
com/
Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707) 2019b.
Wang. H., Dhande, A., and Bansal, S. Cooptimizing safety and performance with control-constraine formulation. IEEE Control Systems Letters, 8:2739-217 2744, 2024. doi: 10.1 109/LCSSYS2004.333 14299.
Wang, S., Teng, Y., and Perdikaris, P. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing,
Ray, A., Achiam, J., and Amodei, D. Benchmark- ing safe exploration in deep reinforcement learn- ing, 2019. URL https:/ //tt:://cd... owwnai..c..mmm com/ safexp-short.ppe
4335):A3055-A308 2021.
Williams, G., Drews, P., Goldfain, B., Rehg. J. M., and Theodorou, E. A. iifommation-thoorete model predictive control: Theory and applications to autonomous driving. IEEE Transactions on Robotics, 3406)160 2018. doi: 10.1109/TRO.200 2011 2865891.
Schmerling, E. reachabbb;;; Hamilton- Jacobi reachability analysis in JAX. reachability. 2021.
Singh, A., Feng, Z., and Bansal, S. Exact imposition of safcty boundary conditions in neural reachable tubes. In 2025 IEEE International Conference on Robotics and Au- tomation (ICRA), 2025. URL httpss://arxivorgg abs/ 44044.400004 0.814.
So, O., Ge, C., and Fan, C. Solving minimum-cost reach avoid using reinforcement learning. In The Tiirth- Annual Conference on Neural Information Processing Systems, 2024. URL battps:// http:://///eeeeiee.//// net. forum? d=jzngdJJ221..

---

Physics-Informee Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Contents
A. Notations
13
A.I Proof of Theorem (3.1)
13
A.2 Proof of Theorem (3.2)
14
A.3 Relationship between 8, and
14
B. A Additional Details the systems in the experiments
15
B.1 Efficient and Safe Boat Navigation
16
B.2 Pursuer vehicle tracking an evader
17
B.3 Multi-Agent Navigation
18
C. Implementation Details of the Algorithms
20
C.1 Experimentation Hardware
20
C.2 Hyperparameters for the Proposed Algorithm
20
C.3 Hyperparameters for MPPI
21
C.4 pyprraaraeeters for SAC-Lag
21
C.5 Hyperparameters for PPO-Lag
21
C.6 Hyperparameters for CPO
21
12

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
A. Proofs
A.1. Theorem (3.1)
Theorem 3.1 (Safety Verification Using Conformal Prediction) Let Sj be the set of states satisfying (o à) = 6, and let (0, df))==1.....N be N₂ i.i.d. samples from Sg. Define og as the safety error rate among these N, samples for a given à level. Select a safety violation parameter eą (0, 1) and confidence parameter [3 (0, 1) such that:
Σ
c - -a) S 38
where = [(V, + 1. Then, with the probability of at least 3.. the following holds:
(v(0, .f)) 5 0) 2 Cz.
SS
Proof. Before proceed with the proof of the Theorem (3.1), let us look at the following lemma which describes split conformal prediction:
Lemma (Split Conformal Prediction (Angelopoulos & Bates, 2022)). Consider set of independent and identically distributed (i.i.d.) calibration data, denoted as {(X,, Y=) along with new test point (X Yee) sampled independently from the same distribution. Define a score function s(x, y) E R. where higher scores indicate poorer alignment between x and y. Compute the calibration scores $1 = s (X1, Yi)...., Sn s(x,, Y,). For a user-defined confidence level a, let & represent the [nn 1)(1 a) quantile of these scores. Construct the prediction for the test input xes
C(Xtess) {y (X y) S a).
Assuming exchangeability, the prediction set ((1ea) guarantees the marginal coverage property:
P(ee E C(Xiess)) 2
Following the Lemma I, we employ a conformal scoring function for safety verification, defined as:
s(X) = V= (0, F), Vi E Sz:
where Sş denotes the set of states satisfying (0, :) S 8 and the score function measures the alignment between the induced safe policy and the auxiliary value function.
Next, we sample N₂ states from the safe set Sü and compute conformal scores for all sampled states. For a user-defined enror rate E j0, 1). let ; denote the (N,+1)a th quantile of the conformal scores. According to (Vovk, 2012), the following property holds:
P #ES;
(É., S <)) Beta(N, - 1 1,
(21)
where = [(N, + 1)a.
Define F., as:
E, :- P aEs,
(V,, 0) 5 =
Here, E. is Beta-distributed random variable. Using properties of cumulative distribution functions (CDF), we assert that E, 2 6, with confidence B3 if the following condition is satisfied:
1i-e, (N 1, < 8.,
(22)
where (a, is the regularized incomplete Beta function and also serves the CDF of the Beta distribution. It is defined as:
1
[
(a, 6)
ga=1(11 6)6-1 dt,
B(a, b)
13

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
where Baa, b) is the Beta function. From (Olver et al., 2023)(8.11)))) (7.17.),, can be shown that k.nnkkkkt Et= (1)x(1
Then (22) can be rewritten as:
Σ
No i {
- No) S Bar ) (1
(23)
==1
Thus, if Equation (23) holds, we can say with probability 1 - Bg that:
I #ES3
(vo (Zi, 0) < 2 1 Eg.
(24)
Now, let denote the number of allowable safety violations. Thus, the safety error rate is given by A+1 Let q
N,+1
represent the [N,+1)as th quantile of the conformal scores. Since & denotes the number of samples for which the conformal N.
score is positive, the (N,-1)as th quantile of scores corresponds to the maximum negative score amongst the sampled states. N.
This implies that i S 0. From this and Equation (24), we can conclude with probability 8g that:
VH. (0, 24, s $0) 2 -
aES₁
From Equation (4), can be inferred that (1, (0, S Vi (Fis, 0). Hence, with probability - Bs the following holds:
P #ES,
(v(),..)) 50) 2 1 Co.
A.2. Theorem (3.2)
Theorem 3.2 (Performance Quantification Using Conformal Prediction) Suppose S denotes the safe states satisfying V0(0, < oo (or equivalently (O, (8)) < 8) and (0,æ:))=1,...,,,, are Np i.i.d. samples from S For a user- specified level Op, let vé be the [(N,+1)(1-p]]] (h quantile of the scores (p: == Voo Cmaz
)i.,,...,NN, N, on the Np state samples. Select violation parameter Ep (0, 1) and confidence parameter B, E (0, 1) such that:
N₂
Σ
N₂ (1 20) Bp
1=0
where, - [(N, + 1)apl. Then, the following holds, with probability 1 - Bp:
(Vo(O, Vi (0, x0,
IP
2 1 Ep.
rest
Cmaz
where Cmaz is a normalizing factor and denotes the maximum possible cost that could be incurred for any E S*.
Proof. To quantify the performance loss, we employ conformal scoring function defined as:
p(x)= Vo x,) VZo (0, x:s) Vo E S*
Cmaa
where the score function measures the alignment between the induced optimal policy and the value function.
Next, we sample Np states from the state space S* and compute conformal scores for all sampled states. For user-defined
error rate ap [0, 1], let denote the (N,+1)e quantile of the conformal scores. According (Vovk, 2012), the following property holds:
N,
Vo xTI - VV.o (0,x:) S 1b) ~ Beta Np - -1+ 1,1),
I z
Cmaz
14

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
where / L(Np + 1)apl-
Define Eip as:
-vo
Ep:=
P #ES*
S 5
Cmax
Here, Ep is a Beta-distributed random variable. Using properties of CDE, we assert that Ep 2 Ep with confidence 1 - 8p the following condition is satisfied:
4--e (N - / 1,0) 5 Spp
(25)
where L (a, b) is the regularized incomplete Beta function. From (Olver et al., 2023)(8.17.),, can be shown that (1-x)==x)) x)" Hence, Equation (25) can be equivalently stated as:
(n k, k 1) Σi= x1)x² -
Σ
No 7 - -pp) -p S Bp
(26)
Thus, Equation (26) holds, we can conclude with probability 1 Sp that:
(0,;;;;; VIN (0,
S 8/1 2 cp.
xES
Cmaz
A.3. Relationship between a, 3, and E
1k
N 10k
N 100k
0.75
0.75
0.75
6.50
0.50
3-65
0.60
0.00
0.00
α
Figure 7. This figure shows the a-e plots for different numbers verification samples, N, and different values of B.
The work (Vovk, 2012) states that smaller number of samples leads to greater fluctuations in the conformal prediction calibration, meaning that if we redraw N samples and repeat the conformal prediction process, we might get different calibration result. This variance decreases as N increases. Similarly, in our work. small N means that the value correction term 8 might fluctuate each time the verification algorithm is executed. Therefore, to ensure stable estimate of ö, it is desirable to select a sufficiently large value of N.
Figure 7 presents the a - c plots for varying numbers verification samples N and different values of 3. From the figure, we observe that as increases, effect of 33 diminishes, and the curve approaches the a line. Ideally, user-specified safety eиror rate (a) should closely match the safety violation parameter (e) while maintaining high confidence 3 close to 1). Thus, selecting larger N enables smaller / while ensuring the alignment of a and Conversely, if N is small, one must either compromise on the confidence parameter or accept that will be lower than resulting in more conservative upper bound on the safety rate.
B. Additional Details the systems in the experiments
In this section, we will provide more details about the systems we have used in the experiments section 4.
15

---

A Physics-Informel Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
B.1. Efficient and Safe Boat Navigation
The states, a of the 2D Boat system are [ x2]", where, 21, 21,22 22 are the a and y coordinates of the boat respectively, We define the step cost at each step, xt) as the distance from the goal, given by:
((1,x) = || (1.5,0)
The cost function (()) is defined as:
c(t, S: ((,,(())) dt + +((((T)
(27)
where T is the time horizon (28 in our experiment), 1(1, xx))) = ||x(1) - (1.5, 0) represents the running cost, and ((x(T)) = [Tx(T) - (1.5,0)²) is the terminal cost. Minimizing this cost drives the boat toward the island.
Consequently, the (augmented) dynamics of the 2D Boat system are:
01 01 + 0.5x
22 U2
ltt, xz)
where u1, U2 represents the velocity control in xı and z2 directions respectively, with uf + u2 < and 2 - -0.5x3 specifies the current drift along the x -axis..
The safety constraints are formulated as:
g(z) == m0x || - (-0.,,.,,,,,)) - - (100-000000000------11-1111-1111 (-12,,,112))
(28)
where g(x) > 0 indicates that the boat is inside boulder, thereby ensuring that the super-level set of g(z) defines the failure region.
B.1.1. GROUND TRUTH COMPARISON
We compute the Ground Truth value function using the Level-Set Toolbox (Mitchell, 2004) and use it as a benchmark in our comparative analysis. To facilitate demonstration, unsafe states are assigned a high value of 20 instead of The value function in this problem ranges from 0 to 14.76.
As illustrated in Figure 8, the value function obtained using our method closely approximates the ground truth value function. Notably, the unsafe region (highlighted in yellow) remains identical in both cases, confirming the safety of the learned value function. Furthermore, the mean squared error (MSE) between the two value functions is 0.36, which is relatively low given the broad range of possible values.
It is also worth mentioning that computing high-fidelity ground truth value function on 210 210 210 grid using the Level Set Toolbox requires approximately 390 minutes. In contrast, proposed approach learns the value function in 122 minutes, achieving substantial speedup. This demonstrates that even for systems with relatively low-dimensional state space, our method efficiently recovers an accurate value function significantly faster than grid-based solvers.
1.2. EMPIRICAL VALIDATION OF THE a-3-e RELATIONSHIP AND CALCULATION OF SAFETY LEVELS
We conducted an experiment to empirically validate the theoretical relationship between a, 3, and e. The results are presented in Figure 9. The figure visualizes the relationship between theoretical and empirical safety metrics across varying levels of 8, and includes the following elements:
Safety error rate (o, purple line) function of different 5 levels. Computed on the calibration dataset as k+1 where is the number of allowable safety violations and N, is the number of calibration samples.
N
Theoretical safety violation probability (c, orange line) as a function of Derived using the theoretical relation in Equation (14).
16

---

Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
17.5
15.0
12.5
Yt
7.5
5.0
2.5
-3
b
2
-3
Eb
2
Figure 8. Heatmap of the value function for the ground truth (left) and our method (right). The yellow region represents the unsafe area. method successfully captures most the safe set, indicating not overly conservative while completely recovering unsafe regions.
Empirical safety violation probability (green points) as a function of ó. Computed by sampling 3M initial states from the d-sublevel set of the learned value function, simulating rollouts, and measuring the observed safety violation rate. This serves as a practical estimate of system safety.
For this experiment, we set = 300k and B = 10-10_ As shown in the figure, the empirical violation rate remains consistently below the theoretical bound (c) across all values ò. This demonstrates that our method provides conservative and valid safety guarantees, confirming the soundness of the theoretical relationship practice.
Additionally, from the 8 plot, we can observe that the ô level approaches the values approach the chosen safety level of 0.001. Hence, say that the sub-level set of the auxiliary value function, V 2) is safe with probability of 1 1 0.001 0.999.
B.2. Pursuer vehicle tracking evader
The state, of ground vehicle (pursuer) tracking moving evader is [2pp Vps v. O Ze. Ves Uze: Uye where, Te: Me, v, θ are position, linear velocity and orientation of the pursuer respectively, Mey Uze: Uye are the position and the linear velocities of the evader respectively. We define the step cost at each step, I(t,x), as the distance from the goal, given by:
1(t, 2) == (((p((t) (t) (t)
and the terminal cost is ((x(())) ((l,TT)))=1(,((()83___ ( The cost function C(t, ((t) is defined as:
((t, = J. ((t, dt + ((x(T))
(29)
where T is the time horizon (1s in this experiment). Minimizing this cost aims to drive the pursuer toward the evader. Consequently, the (augmented) dynamics of the system as follows:
T cos( (e),
Up = v (iin))),
u1.
e 122
Ures
ye Uyer
Uiee
0,
Vie -l(t, x),
17

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
0.0008
Empirical Safety Violation Rate
0.0007
0.0007
0.0006
0.0005
0.0005
0.0004
0.0003
0.0003
0.0002
0.0001
0.0001
0.0000
0.3233
-0.2156
-0.1078
0.0000
8
Figure 9. This figure presents comparative analysis of the relationships between 6- 8, and empirical safety violation rate 6. As observed, the empirical safety violation consistently remains below the theoretical bound, thereby supporting our theoretical guarantees. Furthermore, decreases, the corresponding 8 approaches zero, indicating that the learned value function incurs negligible safety violations.
where U1 represents the linear accelcration control and uz represents angular velocity control.
The safety constraints are defined as:
g(x) :=maz(0.2) || (0.5, 0.5) 1, || (-0., (-0.5, 10.5 0.2 || (0.5, 1I, 0.2 || (0.0,0.0)71.))
which represents 5 obstacles of radius 0.2 units each.
B.3. Multi-Agent Navigation
A multi-agent setting with agents. The state of each agent is represented by x; = [xay, Da, Tg. Uy. tries to reach its goal while avoiding collisions with others. (Zays Na,) denote the position of the ith agent, while (I My.) represent the goal locations for that agent. We define the step cost at each step, l(t, xt))), as the mean distance of each agent from its respective goal, given by:
((t, x()) Eim [1, (t), Me, (t)T (I Ma. (t)
5
The cost function C(t,x(()) u) is defined as:
C(t, := f' ((t,((t)) + +((a(1))
(30)
18

---

A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
where T is the time horizon (2s in this experiment). Minimizing this cost aims drive each agent towards its goal. Consequently, the (augmented) dynamics of the system is as follows:
La; uns, Vi E (1,2,3,4,5)
j/. U21, Vi E 2,4,4,
g. 0, E (1,2,3,4,5)
Vg. 0, Vi (1,2,3,4,5)
=((t,z)
where Uii U2i represents the linear velocity control of each agent i. The safety constraints defined as:
9(x(t)) = (J=(I..,)..4 (R (a Yo. (Ia, Ya, )
(31)
B.3.1. COMPARISON OF AGENI-AGENT NAVIGATION WITH BASELINES
[ Os
t= ls
t 2s
Figure 10. Snapshots of multi-agent navigation trajectories at different time instances using MPPI. The trajectories indicate that the agents adopt a highly conservative strategy to prevent collisions. Consequently, this leads to a reduction in performance, as the agents end up very far from their respective goals.
Os
1 1s
[ 2s
Figure Snapshots of multi-agent navigation trajectories different time instances using MPPI-NCBF. The observed trajectorics demonstrate suboptimal behavior similar to that of the MPPI policy. Consequently, this results in high-performance costs, indicating inability to effectively co-optimize safety and performance.
Figures 10, 11, and 12 illustrate the trajectories obtained by the baseline methods for the Multi-Agent Navigation problem. It can be observed that the trajectories obtained by MPPI and MPPI-SF are highly conservative, implying that these methods prioritize safety to mitigate potential conflicts among agents. In contrast, the policy derived from SAC-Lag fails to maintain safety, resulting in agent collisions. This indicates that as system complexity increases, the baseline methods tend to prioritize either safety or performance, leading to suboptimal behavior and safety violations. Conversely, the proposed approach effectively co-optimizes safety and performance, even in complex high-dimensional settings, achieving superior performance while ensuring safety. The visualization of the trajectories can be found on the project website!.
Phttps://taye: http:///aaaaalaa2aeea
manan28. ittthb.i//pim1sooo
19

---

Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Os
t= Is
t 2s
Figure 12. Snapshots of multi-agent navigation trajectories at different time instances using SAC-Lag. The trajectories indicate that agents demonstrate less conservative behavior compared to MPPI and MPPI-NCBE, but they lead to collisions. These safety violations are critical and cannot be disregarded, further highlighting the limitations of the baseline methods in simultancously optimizing safety and performance.
C. Implementation Details of the Algorithms
This section provides an in-depth overview our algorithm and baseline implementations, including hyperparameter configurations and the cost/reward functions used in the baselines across all experiments.
C.1. Experimentation Hardware
All experiments were conducted on a system equipped with an 11th Gen Intel Core 19-11900K 3.50GHz x 16 CPU, 128GB RAM, and an NVIDIA GeForce RTX 4090 GPU for training.
C.2. Hyperparameters for the Proposed Algorithm
We maintained training settings across all experiments, as detailed below:
Hyperparameter
Value Multi-Layer Perceptron (MLP)
Network Architecture Number of Hidden Layers
Activation Function
Sine function
Hidden Layer Size
256 neurons per layer
Optimizer
Adam optimizer 10-5
Learning Rate
Boat Navigation
Number of Training Points
65000
Number of Pre Training Epochs
50K
No. of Training Epochs
200K
Pursuer Vehicle Tracking Evader
Number of Training Points
65000
Number of Pre Training Epochs
60K
No. of Training Epochs
300K
Multi Agent Navigation
Number of Training Points
65000
Number of Pre Training Epochs
60K
No. of Training Epochs
400K
Table Hyperparameters for the proposed algorithm
20

---

A Physics-Informee Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
C.3. MPPI based baselines
For all the experiments we consider the MPPI cost term as follows:
CмPP1 = C(t, ( Amax(g(2x,0)
(32)
where, À is the trade-off parameter, C(t, ((),, u), g(x) are the cost functions and safety functions as defined in Appendix B. Following is the list of hyperparameters we have used for MPPI experiments in all the cases:
Hyperparameter
Value
Trade-off parameter (A)
100
Planning Horizon
20
Softmax Lambda
200
No. of Rollouts 8000
Table Hyperparameters for MPPI Baselines
C.4. SAC-Lag hyperparameters
For all the experiments, we consider the reward term as follows:
RSAC-Los Lag = -Ctt, u) -I(x))0 x (100) + +((ta(()))oo. X (100)
(33)
where, C(t, u), g(x) are the cost functions and safety functions as defined in Appendix B. Table 3 provides the list of hyperparameters we have used for SAC experiments in all the cases.
Parameter
Value
Policy Architecture
Multi-Layer Perceptron (MLP)
learning rate
3 x 10-4
buffer size
1, 000, 000
learning starts
10, 000
batch size
256 0.005
Target network update rate (r)
Discount factor (7)
0.99
Boat Navigation
Number of Training Steps
1,000,000
Pursuer Vehicle Tracking Evader
Number of Training Steps
2,500,000
Multi Agent Navigation
Number of Training Steps
1,000,000
Table 3. General Hyperparameters of SAC in experiments
C.5. PPO-Lag hyperparameters
For all the experiments, consider the reward term follows:
RpPO-Lag = -C(t, u) Ig(e)>u (100) + [((,,(())<< (100)
(34)
where, C(t, (t),, u). g(z) are the cost functions and safety functions as defined in Appendix B. Table 4 provides the list of hyperparameters we have used for PPO experiments in all the cases.
21

---

Conjugate Gradient Damping A Physics-Informee Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Parameter
Value
Policy Architecture
Multi-Layer Perceptron (MLP)
learning rate
x 10-4
buffer size
1, 000, 000
learning starts
10, 000
batch size
256
Target network update rate (T)
0.005
Discount factor (7)
0.99
Boat Navigation
Number of Training Steps
1,000,000
Pursuer Vehicle Tracking Evader
Number of Training Steps
2,500,000
Multi Agent Navigation
Number of Training Steps
1,000,000
Table 4. General Hyperparameters of PPO in our experiments
C.6. CPO hyperparameters
For all the experiments, consider the reward term follows:
Rcro -C(t,x((), u) (00 x (100)
(35)
where (t),, u), g(x) are the functions and safety functions defined in Appendix For the CPO implementation. have used the training settings used in (Chen al., 2020). Table 5 provides the list of hyperparameters we have used for CPO experiments in all the cases.
Parameter
Value
Policy Architecture
Multi-Layer Perceptron (MLP)
Batch Size
128
Target KL Divergence
0.01
Entropy Coefficient
0.0
Reward Discount Factor (7)
0.99
Cost Discount Factor (%a
0.99
GAE Lambda (A)
0.95
Cost GAE Lambda (Aa)
0.95
Critic Norm Coefficient
0.001
Penalty Coefficient
0.0
0.1
Conjugate Gradient Iterations
15
Actor Hidden Sizes
[256, 256]
Critic Hidden Sizes
[256, 256]
Critic Learning Rate
0.001
Boat Navigation
Number of Training Steps
10,000,000
Pursuer Vehicle Tracking Evader
Number of Training Steps
25,000,000
Multi Agent Navigation
Number of Training Steps
10,000,000
Table CPO Hyperparameters from OmniSafe Configuration used for our experiments
22


Reviews:
[{"review_id": "s9dX2uUEq8", "reviewer": "Reviewer_Rqif", "summary": "The paper addresses the challenge of simultaneously optimizing performance and safety in autonomous systems by formulating it as a state-constrained optimal control problem. The key contribution is a physics-informed machine learning (PIML) framework that efficiently approximates the Hamilton-Jacobi-Bellman (HJB) equation, capturing the optimal value function under hard safety constraints. By embedding known system dynamics into the learning process, the approach mitigates the computational challenges of solving HJB equations in high-dimensional settings. Additionally, the authors introduce a conformal prediction-based verification method to quantify learning errors and provide probabilistic safety guarantees. The learned value function is statistically corrected to ensure safety with a high-confidence bound, and a separate conformal prediction step quantifies potential performance degradation. Empirical evaluations on three autonomous control tasks demonstrate that the framework learns controllers that maintain safety constraints while achieving near-optimal performance.\n\n## update after rebuttal\nI thank the authors for their response. My questions have been largely answered. New baselines and experimental results for scalability are appreciated and will make a good addition to the paper. For the new safe RL baselines, please include experimental results for all tasks in the paper. I updated my overall recommendation accordingly.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Clarification regarding the Pursuer Evader Problem\n\nIn the Pursuer-Evader case study (Section 4.2), the objective is for a pursuer robot to chase a moving target and reach as close to this target as possible within the time horizon. The human figure serves only as a placeholder for the evader and can be substituted with any other entity (e.g., another robot). The key aspect of the problem is the interaction between the pursuer and the evader, both of which follow the dynamics outlined in Appendix B.2. The quality of tracking is encoded by the performance cost, which at each time step is equal to the $l_2$ distance between the evader and the pursuer. The lower the cumulative performance cost, the better the quality of tracking. In addition to tracking, the pursuer needs to avoid collisions with obstacles, which is encoded as a safety constraint. Due to this constraint, the pursuer indeed needs to prioritize obstacle avoidance over tracking at times. An example of this is shown in Figure 1 (anonymous link: https://shorturl.at/gWIrw). Here, the pursuer vehicle can achieve a lower tracking cost by cutting through the obstacle; however, due to the collision-avoidance constraint, the pursuer goes around the obstacle and intercepts the evader from behind.\n\n# Comparison with Other Safe RL Baselines\n\nIn response to reviewer suggestions, we benchmarked our method against additional safe RL baselines: CPO [1], IPO [2], and PPO-Lagrangian [3] on the Pursuer-Evader problem. Our results are summarized below:\n\n| Method | Safety Rate (%) | % Cost Higher than Our Method |\n|------------------|-----------------|-------------------------------|\n|CPO|75.11| 123.14|\n|IPO|72.3 |110.34|\n|PPO-Lag|63.33|114.63|\n\nThese experiments reveal that although CPO and IPO achieve marginally higher safety rates compared to C-SAC, all safe RL baselines underperform compared to the proposed approach in both safety and cost metrics.\n\n# Scalability of the Proposed Method\n\nWe assess the computational scalability of the proposed approach by reporting offline and online computation times across 3 case studies:\n\n| Task | Dim | Offline Time | Online Time |\n|------------------------|----------------|--------------|-------------|\n| Boat Navigation|2|122 min|2 ms|\n| Evader Chasing|8|195 min|2 ms|\n| Multi-Agent Navigation|20|323min|2 ms|\n\nTraditional grid-based methods exhibit exponentially scaling computation complexity and become intractable for higher-dimensional problems (8D and 20D). In contrast, our method shows only a modest increase in the offline computation time when going from a 2D to a 20D system while maintaining real-time inference (2 ms) across all dimensions—a key advantage for real robotic applications.\n\n# Relations to Control Barrier Functions (CBFs)\n\nThe learned auxiliary value function, $\\hat{V}$, captures a notion of safety similar to CBFs. In particular, the condition $\\hat{V}(x,z) \\leq 0$ defines the safe region for the system, while $\\hat{V}(x,z) > 0$ corresponds to unsafe states-mirroring the core idea behind CBFs. Therefore, $\\hat{V}(x,z)$ can directly serve as a CBF for the system.\nHowever, importantly, $\\hat{V}(x,z)$ extends beyond standard CBFs by also capturing performance objectives. Through the parameter $z$, one can modulate the trade-off between safety and performance, effectively generating a family of CBF-like functions with different safety-performance characteristics. As a result, $\\hat{V}$ can serve as a CBF in traditional settings or be integrated into CBF-based safety frameworks to encode both safety and performance objectives.\n\n# Tackling Model Uncertainty/Disturbances\n\nWe discuss the extension of the proposed approach to account for model uncertainty and disturbances in our rebuttal to reviewer BKAj. Due to space constraints, we respectfully refer the reviewer to that discussion for further details.\n\n# Transfer to Complex, High-Dimensional Systems\n\nThe primary goal of this work is to lay the foundation for integrating PINNs with HJ Reachability to co-optimize safety and performance in autonomous systems. Our framework already demonstrates strong potential for high-dimensional applications, as evidenced by its success in a 20D multi-agent navigation task. Moreover, the safety constraints in our experiments are also highly complex—often consisting of disconnected, non-convex sets—highlighting the method's ability to handle intricate safety specifications. Further extension and testing of our framework on more complex systems is an exciting future research direction.\n\n# References\n\n[1] Achiam, J. et al. (2017). Constrained Policy Optimization. *ICML*.\n\n[2] Liu, Y. et al. (2020). IPO: Interior-point Policy Optimization under Constraints. *AAAI*.\n\n[3] Ray, A. et al. (2019). Benchmarking Safe Exploration in Deep Reinforcement Learning. *arXiv preprint*.\n\n[4] Bansal, S. et al. (2017). Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances. *IEEE CDC*.", "claims_and_evidence": "Most claims in the paper are reasonable and supported by empirical results.\n\nThe authors claim that their framework scales to complex, high-dimensional systems where classical Hamilton-Jacobi (HJ) or grid-based methods become computationally infeasible. While their experiments include tasks with moderately high-dimensional state spaces, they provide limited discussion or formal analysis on how the approach would extend to even higher-dimensional problems, as encountered in some real-world robotic applications). A deeper investigation into scalability limits, such as computational complexity or the effects of increasing dimensionality on solution accuracy, would strengthen this claim.", "methods_and_evaluation_criteria": "The proposed method is well-motivated for the problem at hand. The authors reformulate the state-constrained optimal control problem in epigraph form, introducing an auxiliary value function that incorporates an additional state variable. This enables the problem to be solved using a Hamilton-Jacobi-Bellman (HJB) approach, leveraging physics-informed machine learning to approximate solutions efficiently.\n\n**Baselines**:\nThe chosen benchmark tasks are reasonable but could be expanded (see \"Experimental Designs or Analyses\" for details).\n\nHowever, the baseline selection could be improved. The current baselines include:\n- C-SAC and MPPI, which are soft-constrained methods that do not enforce strict safety.  \n- MPPI-CBF, which introduces a hard safety constraint but is the only explicitly safe baseline.  \n\nWhile MPPI-CBF provides a useful comparison, a single safe baseline is insufficient to fully assess the method’s safety-performance tradeoffs. Additionally, soft-constrained baselines like C-SAC may not be the best comparison since they treat safety as a penalty rather than a hard constraint, potentially ignoring safety violations rather than actively enforcing constraint satisfaction.  \n\nTo strengthen the evaluation, the authors should consider safe reinforcement learning baselines that explicitly handle hard constraints, such as: Constrained Policy Optimization (CPO), Interior-Point Policy Optimization (IPO).\nThese methods dynamically adjust constraint handling (e.g., by tuning Lagrange multipliers) and would provide a more rigorous safety comparison.\n\n**Evaluation Metrics**: \nThe evaluation is based on cumulative cost (to measure performance), and safety rate (percentage of trajectories that remain safe). While these metrics are relevant, they are not sufficient to fully characterize the safety-performance tradeoff. Some limitations include:\n\n- Cumulative cost does not explicitly account for constraint satisfaction trade-offs, i.e., two methods with similar costs might differ significantly in how well they satisfy safety constraints.\n\n- Safety rate provides only a binary measure so it does not capture the severity or frequency of constraint violations (e.g., how far a trajectory enters an unsafe region).\n\nAdditional metrics could provide deeper insights, such as:\n- Constraint violation magnitude: Quantifying how much and how often safety constraints are violated.\n\n- Worst-case safety violations: To ensure the method does not fail catastrophically in rare but critical scenarios.\n\n- Computational efficiency: Since the approach is meant to scale, measuring inference time or sample efficiency would strengthen claims about scalability.", "theoretical_claims": "The paper presents several theoretical claims, primarily related to the Hamilton-Jacobi-Bellman (HJB) formulation, epigraph reformulation, and conformal prediction-based verification. While I am not deeply familiar with the theoretical aspects of this work, I have outlined some potential areas for further scrutiny. My suggestions may not be fully comprehensive.\n\nThe conformal prediction-based safety verification (Theorem 3.1) and performance quantification (Theorem 3.2) leverage results from distribution-free uncertainty quantification. These rely on assumptions about the distribution of sampled states, and their applicability to high-dimensional control problems could benefit from additional discussion.\n\nThe method learns the HJB solution using a physics-informed neural network. Theoretical guarantees on convergence or approximation error bounds for the learned solution are not explicitly discussed, which could impact scalability and accuracy in high-dimensional settings.", "experimental_designs_or_analyses": "The experiments primarily involve 2D environments, likely due to the assumption of known system dynamics. While these tasks are reasonable for evaluating safety constraints, incorporating higher-dimensional agents (e.g., quadrupeds like Ant from Safety Gym) would better assess scalability and applicability to more complex scenarios.\n\nTask 2 (Pursuer-Evader) is unclear. The description refers to an \"evader vehicle,\" but the visualizations appear to show pursuer vehicles interacting with humans. This raises several questions:\n\n- What exactly is the task? Is the pursuer meant to collide with the evader, follow it at a safe distance, or something else?\n- If the goal is to track the evader, what defines successful tracking?\n- Does the pursuer ever need to prioritize obstacle avoidance over tracking?\n\nAdditionally, the environment complexity could be increased to better challenge the proposed method. Consider scenarios where:\n\n- More obstacles block the direct path to the target.\n- The free space is more constrained (e.g., narrow passages), requiring more precise maneuvering under safety constraints.", "relation_to_broader_scientific_literature": "The use of epigraph reformulation to enforce hard constraints and conformal prediction for probabilistic safety verification appears to be a novel combination that extends prior work in safe reinforcement learning. A deeper comparison with safe RL methods (e.g., Constrained Policy Optimization, Interior-Point Policy Optimization) and HJ-reachability-based safety filtering could help clarify how this approach advances the state of the art.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "__Strengths__:\n\n- The epigraph formulation effectively transforms the constrained problem into an augmented state space, facilitating tractable optimization.\n- Theoretical guarantees for safety verification and performance quantification strengthen the framework’s reliability.\n\n__Weaknesses__:\n\n- Solving high-dimensional HJB Partial Differential Equations (PDEs) remains computationally expensive, limiting real-time applicability.\n\n- Assumes known (or well-learned) system dynamics, restricting applicability to environments with uncertain or evolving dynamics.\n\n- Limited comparison against strong safe control baselines, reducing the robustness of empirical validation.\n\n- Experimental tasks are relatively simple, likely due to the dependence on known dynamics, limiting generalizability to more complex settings.", "other_comments_or_suggestions": "Task 2 description can be rephrased (see my comments above).", "questions_for_authors": "- How does the learned auxiliary value function $\\hat{V}$ compare to Control Barrier Functions in ensuring safety? Can it be directly used as a CBF, or how would it integrate with CBF-based methods? A response clarifying this relationship would help assess whether the approach can be leveraged in existing safety-critical control frameworks.\n\n\n- How well does the proposed framework transfer to more complex, high-dimensional systems with unknown or highly nonlinear dynamics? Have you tested it beyond the benchmark tasks? Demonstrating broader applicability would strengthen the paper’s impact.\n\n\n- If the assumed or learned system dynamics deviate significantly from reality, how does the approach perform in terms of both safety and optimality? Have you evaluated it under model uncertainty or system disturbances? A discussion on this would clarify the method’s robustness in real-world deployment.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oPXonWfCkY", "reviewer": "Reviewer_7cLD", "summary": "This paper proposes a novel Physics-Informed framework to address the co-optimization problem of safety objectives and performance objectives for Constrained Reinforcement Learning (CRL). The paper reformulates the co-optimization problem as a state-constrained optimal control problem (SC-OCP) with epigraph formulation. Moreover, the authors propose an algorithmic approach to learning the SC-OCP value function from which the policy is induced. At last, this paper proposes a conformal prediction-based verification strategy to provide a probabilistic guarantee of safety and performance degradation. The proposed framework is demonstrated on several nonlinear control tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# How is the policy $\\pi_{\\theta}$ synthesized?\n\nThe final policy $\\pi_{\\theta}(t, x)$ is synthesized by first determining the optimal $z^*$ by solving the following optimization problem for any $(t,x)$:\n\n\\begin{equation}\n    \\begin{aligned}\n    z^* = &\\arg\\min_{z \\in \\mathbb{R^+}}  \\; z  \\\\\n    \\text{s.t.} & \\; \\hat{V}_{\\theta}(t,x,z) \\leq \\delta,\n\\end{aligned}\n\\end{equation}\n\nFor efficiency purposes, we solve the above optimization problem via binary search on $z$. \nOnce $z^*$ is determined, the optimal control policy is the policy that minimizes the Hamiltonian ($H(t,\\hat{x})$) [1] at the corresponding augmented state $(x, z^*)$, denoted as $\\hat{x}^*$:\n\n\\begin{equation}\n    H(t,\\hat{x}^*) = \\min_{u\\in \\mathcal{U}} \\langle\\nabla \\hat{V}_{\\theta}(t, \\hat{x}^*), \\hat{f}(\\hat{x}^*, u)\\rangle.\n\\end{equation}\n\nThis results in the final policy $\\pi_{\\theta}$, as provided in Equation (17) of the submitted manuscript:\n\n\\begin{equation}\n    \\pi_{\\theta}(t,x) = \\arg \\min_{u\\in \\mathcal{U}} \\langle\\nabla \\hat{V}_{\\theta}(t, \\hat{x}^*), \\hat{f}(\\hat{x}^*, u)\\rangle.\n\\end{equation}\n\nIntuitively, the policy $\\pi_{\\theta}$ at any state $x$ follows the minimum cost path, which is given by the gradient descent direction of the auxiliary value function.\nThe overall policy inference process takes $2ms$ on our standard GPU system.\n\n# How is the parameter $\\lambda$ determined in equation 10? How does it affect the safety and performance?\n\nFor a given partial differential equation (PDE), multiple solutions may exist, and uniqueness is ensured only when an appropriate boundary condition is imposed. The parameter $\\lambda$ regulates the enforcement of this boundary condition. If $\\lambda$ is too small, the boundary function may not be adequately learned, resulting in an inaccurate solution. To address the sensitivity of the learned solution to $\\lambda$, we employ the adaptive loss restructuring approach from [2], which dynamically updates $\\lambda$ at each iteration. The updates are directly proportional to the ratio of the boundary loss gradient to the PDE loss gradient. Intuitively, a small ratio results in a lower $\\lambda$, prioritizing the PDE loss, whereas a larger ratio increases $\\lambda$, emphasizing boundary condition enforcement.\n\n\n# To achieve a high probabilistic guarantee, the proposed verification methods need to densely sample the state space (300K), which is non-trivial. Can you provide more discussion on the practicality and potential limitations of real-world applications?\n\nThe work [3] by Vovk states that a smaller number of samples leads to greater fluctuations in the conformal prediction calibration, meaning that if we redraw $N$ samples and repeat the conformal prediction process, we might get a different calibration result.\nThis variance decreases as $N$ increases.\nSimilarly, in our work, a small $N$ means that the value correction term $\\delta$ might fluctuate each time the verification algorithm is executed. Therefore, to ensure a stable estimate of $\\delta$, it is desirable to select a sufficiently large value of $N$.\n\nAdditionally, Figure 1 in https://shorturl.at/8vgih presents the $\\alpha-\\epsilon$ plots for varying numbers of verification samples $N$ and different values of $\\beta$. From the figure, we observe that as $N$ increases, the effect of $\\beta$ diminishes, and the curve approaches the $\\alpha = \\epsilon$ line. Ideally, the user-specified safety error rate ($\\alpha$) should closely match the safety violation parameter ($\\epsilon$) while maintaining high confidence ($1-\\beta$ close to 1).  \nThus, selecting a larger $N$ enables a smaller $\\beta$ while ensuring the alignment of $\\alpha$ and $\\epsilon$. Conversely, if $N$ is small, one must either compromise on the confidence parameter $\\beta$ or accept that $\\alpha$ will be lower than $\\epsilon$, resulting in a more conservative upper bound on the safety rate.\n\n\n**References**\n\n[1] S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, \"Hamilton-Jacobi reachability: A brief overview and recent advances,\" 2017 IEEE 56th Annual Conference on Decision and Control (CDC), Melbourne, VIC, Australia, 2017, pp. 2242-2253, doi: 10.1109/CDC.2017.8263977.\n\n[2] S. Wang, Y. Teng, and P. Perdikaris, “Understanding and mitigating gradient flow pathologies in physics-informed neural networks,” SIAM Journal on Scientific Computing, vol. 43, no. 5, pp. A3055–A3081, 2021.\n\n[3] V. Vovk, “Conditional validity of inductive conformal predictors,”\n2012. [Online]. Available: \\\\ https://arxiv.org/abs/1209.2673", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Partially, I checked all equations and theoretical statements except for the proofs for Theorem 3.2", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The proposed contributions are related to Constrained  Reinforcement Learning.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths: \n1. The targeted problem is well-motivated, and the solution is novel.\n2. The reformulation of the CRL problem is insightful, and the derived physics-informed loss function for the learning value function is mathematically sound. \n3. The theoretical analysis and empirical study support the claimed contributions.\n\nWeaknesses: \n1. Some technical details need more clarification. Please see the questions", "other_comments_or_suggestions": "Please see the questions.", "questions_for_authors": "1. How is the policy $\\pi_\\theta$ synthesized? The searched policy should minimize equation 11; however, how the policy is learned is unclear. The authors should provide more details, such as what algorithm is used and how policy is parameterized.\n2. How is the parameter $\\lambda$ determined in equation 10? How does $\\lambda$ affect the safety and performance? It would be nice if the authors could provide more insights.\n3. To achieve a high probabilistic guarantee, the proposed verification methods need to densely sample the state space (300K), which is non-trivial. Can you provide more discussion on the practicality and potential limitations of real-world applications?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FAZcrBSEd0", "reviewer": "Reviewer_a3t3", "summary": "In this work, the Authors propose a novel framework for the certified safety of autonomous agents based on the combination of the epigraph-based formulation of the optimal control problem, DeepReach, and conformal predictions. They test the proposed framework in three simulated environments and show the advantage of the proposed framework over the models that either mostly focus on performance or mostly focus on safety.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Clarification on Curriculum Learning\nAs discussed in Sec. 3.1, we first pre-train the DNN to learn the value function at the terminal time ($t=T$)—i.e., the boundary condition of the HJB-VI—using $\\lambda = 0$. We then apply curriculum learning, gradually decreasing $t$ from $T$ to $0$, so the terminal value function propagates backward per the HJB-VI. At each iteration, we uniformly sample $N$ states and time points from [$t, T$]) and train using the loss in Eq. (10).\n\n# Clarification on Verification Procedure\nWe would like to provide further clarification on the proposed verification procedure. First, we estimate the induced policy $\\hat{\\pi_{\\theta}}$ from the learned value function $\\hat{V_{\\theta}}$. Next, we generate 300K rollouts from $t = 0$ to $t = T$, starting from various initial states $\\hat{x_0}$, and compute the corresponding rollout costs $\\hat{V}_{\\hat{\\pi}}$. We then apply conformal prediction to compute the correction $\\delta$, which represents the safety correction over the **entire trajectory** from $t = 0$ to $t = T$. Thus, the violation probability $\\epsilon_s$ is defined for the full trajectories starting from $\\hat{x_0}$, not individual states.\n\n# Why $\\delta$ Does Not Grow Over Time  \nAs discussed above, the correction term $\\delta$ is based on the rollout cost evaluated over the entire trajectory from $t = 0$ to $t = T$. If, instead, the cost were evaluated starting later in the trajectory, i.e., from $t = t_0$ to $t = T$ for some $t_0 > 0$-the trajectory would typically experience fewer safety violations, since early violations are excluded from the cost computation. Thus, the magnitude of $\\delta$ at the beginning of the trajectory is greater than or equal to its magnitude at any later time. This property ensures that the initial correction term is conservative enough to account for worst-case safety costs, leading to robust safety guarantees.\n\n# i.i.d. Assumption in Theorems\nWe assume i.i.d. sampling over the safe set as we lack prior knowledge of the test-time initial state distribution. This ensures generality in our safety guarantees. If a specific initial state distribution is of interest, it can be used for sampling instead to obtain tighter safety estimates. To reduce reliance on distributional assumptions, one can also adopt a worst-case approach by setting $\\alpha = 0$, as the reviewer suggested. \n\n# Potential bias from sampling near $\\partial S_{\\delta}$ \nWhile states delicately balancing safety and performance often reside near $\\partial S_{\\delta}$, learning errors can occur anywhere in $S_{\\delta}$. Sampling only near the boundary may underestimate violations in the interior, which can be problematic if the initial states of interest during the test time are inside $S_{\\delta}$. In the absence of prior distribution knowledge, we adopt uniform sampling across $S_{\\delta}$ to capture violations comprehensively.\n\n# Experiment Validating $\\alpha$-$\\beta$-$\\epsilon$ Relationship\nTo empirically validate this relationship, we conducted an experiment in the Boat Navigation case study (Sec. 4.1). Figure 1 (https://shorturl.at/VYYai) illustrates:\n\n- Safety error rate ($\\alpha$) computed as $\\alpha = \\frac{k+1}{N_s+1}$ (Purple).\n- Theoretical safety violation probability ($\\epsilon$) derived from Equation (14) (Orange).\n- Empirical safety violation probability, estimated via sampling 3 million initial states and simulating their rollouts (Green).\n\nWe use $N = 300K$ and $\\beta = 10^{-10}$. We note that empirical violation rates consistently remain below theoretical bounds, confirming the validity of our approach.  \n\n# Setting $\\alpha = 1$ and Circumventing Conformal Prediction\nWe acknowledge that setting $\\alpha = 0$ (zero violations) indeed eliminates the need for conformal prediction. To test this, we mark the black point in Figure 1 above that corresponds to $\\alpha = 0$ and $\\delta$ level of $-0.3234$, yielding nearly 100\\% safety. As expected, this approach reduces the volume of the resulting safe set, but we still obtain a sizable safe set that accounts for worst-case violations, as suggested by the reviewer. In contrast, $\\delta = 0$ achieves ~99.9\\% safety, reducing conservatism but at a slight safety compromise. In this sense, our framework can be viewed as a robust generalization of this extreme-case approach, offering users the flexibility to choose their preferred level of conservatism based on the application's safety-performance requirements. \n\n# Relevant $\\epsilon$ and $\\beta$, if quantity in eq(15) Accumulate Over Time\nSince all the analysis is performed for the entire trajectory starting from a given state, $\\epsilon$ and $\\beta$ already account for time accumulation.\n\n# Additional Baselines\nWe kindly refer the reviewer to the discussion on additional baselines with reviewer: Rqif.\n\n# Essential References Not Discussed\nWe appreciate the reviewer’s suggestion and will include [1] in the revised manuscript.  \n[1] Hsu et al. The safety filter: A unified view...", "claims_and_evidence": "The work follows the standard routine in the field of the safety of autonomous agents. As a result, the claims here are supported by the standard means, previously validated in the related research.", "methods_and_evaluation_criteria": "I found this work to present a step in the right direction. Naturally, the next step after having DeepReach, a neural network model for solving the Hamilton-Jacobi Reachability problem, was to extend it with the safety guarantees accounting for the errors potentially made by the deep learning model. I have, however, the following concern regarding the type of the guarantees provided by the conformal predictions, as follows. Equation 15 formulates the probability of the safety violation per state $\\hat{x}$ in $S_\\delta$. It would seem that, for longer trajectories, this quantity would accumulate, making the target values of $\\epsilon$ low. How low exactly and how strong the safety requirements would be in that case is a question that, I believe, is important to investigate and answer. At the same time, I would like to notice that this form of guarantee is a huge step forward compared to the cumulative guarantees used previously with conformal predictions, as they did not preclude the possibility of short-term safety violations.\n\nAdditionally, I would like to note that the very same machinery applied here to the cost estimate in Theorem 2 makes perfect sense to me. As point violations of this estimate are not safety-critical, conformal predictions here offer a nice way to make that estimation that, on average, should be correct.", "theoretical_claims": "I have checked the theoretical claims. While the theory looks mostly correct and follows the standard practices in the field, my biggest question is related to the i.i.d. assumption in both Theorems 1 and 2. While I understand that this assumption is central for using the conformal predictions (e.g. through Lemma 1 of Supplementary Material), I am not sure if its use is justified here for the following reason. While under this assumption $N_s$ is sampled i.i.d. from $S_\\delta$, this is arguable not where the states of interest reside. The states of interest, balancing safety and performance, seem likely to reside on the edge of the set $S_\\delta$ ($=\\partial S_\\delta$); thus the i.i.d estimation may be biased.\n\nIf I am correct, two ways to mitigate that potential issue are:\n\n- to set $\\alpha=1$. If it wouldn’t affect the value of $\\beta$ as a function of $\\epsilon$ terribly, that would allow circumnavigating the entire machinery of the conformal predictions, instead converging to a trivial limit-case result from combinatorics (which was also a nice way for me to check that the equations are correct)!\n\n- to perform an empirical study where the numbers of safety violations would be measured as a function of $\\alpha$ and $\\beta$ to see if the simulated experiment reproduces the theoretically predicted trend.\n\nI suggest trying both options as they are easy to try and have the potential of making the results here stronger.\n\nAnother question that I have is: the framework seems to assume that the deviations of the neural network model solutions from the actual solutions are local and do not accumulate over time. It would be nice to hear the argument regarding why this might be the case.", "experimental_designs_or_analyses": "The testing scenarios and baselines here appeared overly simple to me. While there’s nothing wrong with this, typically the related research in the field has used stronger benchmarks, both simulated and data-driven, and considered stronger baselines. There have been lots of advancements in the field lately; it would be nice to compare some of those – per Authors’ choice – to the proposed model.", "relation_to_broader_scientific_literature": "The work is rooted in the host of prior literature. Specifically, the Hamilton-Jacobi Reachability, the epigraph formulation of the optimal control problem, DeepReach, and conformal predictions are at this point all well-established tools used in the certified safety of autonomous systems. The combination of DeepReach and the conformal predictions that is aimed to reinstate the certifiable safety in the model that otherwise computed a solution through a deep neural network, is a novel and natural step forward for that line of research.", "essential_references_not_discussed": "I feel like some further, newer references could be added to the Introduction to better reflect the current state of the research in the field. Perhaps, review papers such as the one on The Safety Filter by Jaime Fisac’s group could be a good addition here.", "other_strengths_and_weaknesses": "Strength: the text is mostly clearly written. Combined with a well-structured research project, it is a pleasure to read.\n\nWeakness: the methods are not fully documented. It is important to detail, in the Methods section, the details of the training (beyond the structure of the neural network that is present in Appendix).", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Why is the i.i.d. assumption made in both theorems?\n\n- Why is the assumption of stability (i.e. that $\\delta$ does not grow over time) made in the model?\n\n- What would be the relevant values of $\\epsilon$ and $\\beta$ provided that the quantity in Equation 5 would accumulate over time and states traversed?\n\n- What was the training procedure for the curriculum learning?\n\n- Could you perform an experiment to show that the theoretically derived relation between $\\alpha$, $\\beta$, and $\\epsilon$ holds?\n\nI am happy to update my score based on the answers to these questions.\n\n__________________________________\n\nPost-rebuttal: my questions and concerns were mostly addressed; raising the score.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7UHWZCLbVX", "reviewer": "Reviewer_BKAj", "summary": "This work develops a physics-informed learning approach for constrained optimal control problem, where the performance objectives are represented using a cost function and safety conditions are formulated using state constraints. A conformal prediction-based safety verification approach is developed with probabilistic error bound on performance degradation. Three simulation examples are provided to demonstrate the performance of the approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Method application to non-convex OCP\n\nOur method extends beyond convex OCPs. While a convex cost function leads to a convex epigraph formulation that standard optimizers can efficiently solve, our approach is not restricted to them.  By leveraging dynamic programming to solve the epigraph formulation, we obtain a general HJB-VI, which is well-suited for handling both convex and non-convex settings. In fact, our experiments explicitly demonstrate this capability by incorporating non-convex and even disconnected safety constraint sets.\n\n# Tackling Model Uncertainty/Disturbances\n\nThis study pioneers a learning-based approach to jointly optimize safety and performance using HJ Reachability, assuming accurate model information. Extending it to account for model uncertainty is a key future direction. As an initial step, we can follow [3] to handle dynamics uncertainty, modelling it as $\\dot{x}(t) = f(x(t), u(t), d(t))$, where $d(t)$ captures learning uncertainty, unmodeled dynamics, or disturbances. We then define a robust auxiliary value function $\\hat{V}$:\n\n\\begin{equation}\n    \\hat{V}(t, x(t), z) = \\min_{\\textbf{u}} \\max_{\\textbf{d}} \\max \\{C(t, x(t), \\textbf{u}, \\textbf{d}) -z, \\max_{s \\in [t, T]}g(x(s)) \\}.\n\\end{equation}\nThe auxiliary value function optimizes performance cost under worst-case uncertainty while guaranteeing safety. As with the non-robust formulation, it is characterized as the unique viscosity solution of the HJI-VI:\n\\begin{equation}\n\\min\\Bigl(-\\partial_t \\hat{V} - \\min_{u} \\max_{d} \\langle \\nabla_{\\hat{x}}\\hat{V}(t, \\hat{x}), \\hat{f}(\\hat{x}, u, d)\\rangle ,\\hat{V} - g(x)\\Bigr) = 0,\n\\end{equation}\nTo incorporate uncertainty (e.g., model errors or adversarial settings), we solve the PDE using our proposed method. As an initial validation, we apply it to the Pursuer-Evader problem with uncertainty in the evader model, capturing potential variability in human behavior. Figure 1 (https://shorturl.at/sxkvw) illustrates this: dashed lines show undisturbed trajectories, while solid lines depict uncertain cases where the evader moves unpredictably. The pursuer adapts with sharper turns, ensuring robust tracking despite uncertainty. Future work will further explore uncertainty handling.\n\n# Training loss does not have a regularizing term\n\nTo mitigate trivial solutions, we employ the adaptive loss balancing strategy from [5], which dynamically adjusts $\\lambda$ at each training step to balance the boundary loss and PDE loss. This ensures both objectives are enforced jointly. \nAs future work, we plan to incorporate explicit regularization, such as Tikhonov ($L_2$) regularization, to prevent overfitting and improve generalization. Additionally, we will explore hard constraint enforcement techniques within the neural network architecture—such as those proposed in [1,2]—to exactly satisfy boundary conditions and enhance the fidelity of the learned value function.\n\n# Missing Assumptions\n\nWe appreciate the reviewer for highlighting the missing assumptions. In the revised manuscript, we will explicitly state those assumptions. In practice, assuming local Lipschitz continuity is often sufficient, particularly when the value function needs to be computed over a compact state space, as is often the case with PINN training (including our work). Under this assumption, the continuity and the well-posedness of the HJB solution can still be ensured without requiring global Lipschitz continuity. We will revise the manuscript accordingly to clarify the assumptions and better contextualize their relevance in practical settings.\n\n# Differentiability of $\\hat{V}$\n\nWe approximate the value function using a PINN with *sinusoidal* activations, ensuring smoothness and differentiability of the value function. Thus, the feedback control law, derived from the value function gradients, is well-defined and implementable.\n\n# Fewer Training Samples to train PINNs\n\nSince our approach is entirely self-supervised, generating training data is computationally inexpensive. Thus, we use a large number of training points (65K) to promote stability and convergence during training. However, we acknowledge that fewer points may be beneficial in high-dimensional cases to reduce memory and computation. While not explored here, we will investigate trade-offs between sample efficiency and performance in future work.\n\n# Essential References Not Discussed\n\nWe thank the reviewer for the suggestion. We will include [4] as well as an analysis of other PIML approaches for HJ Reachability and optimal control in the revised manuscript.\n\n# References\n\n[1] Gaby et al. (2022). Lyapunov-net for Lyapunov function approximation. IEEE CDC.\n\n[2] Singh et al. (2025). Safety boundary conditions in neural reachable tubes. IEEE ICRA.\n\n[3] Bansal et al. (2017). Hamilton-Jacobi reachability. IEEE CDC.\n\n[4] Fotiadis & Vamvoudakis (2023). PINNs for infinite-horizon control. IEEE CDC.\n\n[5] Wang et al. (2021). Gradient flow pathologies in PINNs. SIAM J. Sci. Comput.", "claims_and_evidence": "The contribution is clear and mostly supported by the mathematical exposition and simulation results. The paper makes an interesting contribution relevant to the machine learning, controls, and robotics communities. However, claiming that the paper develops a novel physics-informed machine learning framework is misleading because the paper does not provide any fundamentally new physics-informed machine learning method; the development is mostly a direct application of well-known physics-informed machine learning methods.", "methods_and_evaluation_criteria": "The methodology and evaluation criteria is appropriate.", "theoretical_claims": "1. The theoretical claims (Theorems 1 and 2) appear correct.\n \n2. However, it is not clear if the cost function and safety constraints need to be convex. Are the authors claiming the method applies to non-convex optimal control problems? Some theoretical exposition in this regard would improve the quality of the paper.\n\n3. The training loss in (8) and (9) do not appear to have a regularizing term (e.g., Tikhonov regularization). In physics-inforrmed learning, having a regularizing term in the loss is important as it can prevent the value function estimate from learning a trivial solutions that would yield a zero PDE residual error and also mitigate overfitting. Furthermore, the authors could also include penalty terms to penalize undesirable values of \\hat{V}_{theta}, use it as a constraint in the training, or encode hard constraints in the NN architecture itself (see Lyapunov-Net reference below).\n\n4. The paper claims the auxiliary value function is a unique continuous viscosity solution to the HJB, citing the reference (Altarovici et al., 2013). However, (Altarovici et al., 2013) makes this claim under multiple assumption (see A1-A4 in that result), one of which is that the dynamics f(x,u) be globally Lipschitz in x which highly restrictive for most practical applications). The paper makes no mention of these assumptions. Furthermore, the viscosity solutions are only guaranteed to be continuous and there are no guarantees on its differentiability. However, the control policy is dependent on the derivative of the auxiliary value function. Without differentiability, the feedback law might not be well-defined.\n\n5. The approach using safety verification with conformal prediction is interesting.\n\nReferences:\n\nGaby, N., Zhang, F. and Ye, X., 2022, December. Lyapunov-net: A deep neural network architecture for lyapunov function approximation. In 2022 IEEE 61st Conference on Decision and Control (CDC) (pp. 2091-2096). IEEE.\n\nAltarovici, A., Bokanowski, O. and Zidani, H., 2013. A general Hamilton-Jacobi framework for non-linear state-constrained control problems. ESAIM: Control, Optimisation and Calculus of Variations, 19(2), pp.337-357.", "experimental_designs_or_analyses": "1. The experimental methodology is mostly appropriate. The results are shown to be applicable for high-dimensional systems (20 dimensional system in the multi-agent navigation case). \n2. The paper assumes known dynamics which by itself is not problematic, but what would happen if the dynamics have modeling uncertainties and/or disturbances? In practice, this is usually the case, so the experiments need to account for such modeling uncertainties to be realistic.\n3. One of the benefits of using physics-informed neural networks is that one can obtain good performance with fewer training samples. The experiments use 65,000 training points. What would happen if fewer datapoints are used.", "relation_to_broader_scientific_literature": "The key contributions are related to approximate dynamic programming (ADP) and physics-informed machine learning research. Notably, value function approximation using neural networks is not new, the authors can discuss more relevant ADP papers/optimal control papers to put contribution in context. The authors can also discuss more physics-informed machine learning literature.", "essential_references_not_discussed": "The following reference is relevant and needs to be discussed in the introduction:\n\nFotiadis, F. and Vamvoudakis, K.G., 2023, December. A Physics-Informed Neural Networks Framework to Solve the Infinite-Horizon Optimal Control Problem. In 2023 62nd IEEE Conference on Decision and Control (CDC) (pp. 6014-6019). IEEE.", "other_strengths_and_weaknesses": "None", "other_comments_or_suggestions": "None", "questions_for_authors": "None, please see the previous comments", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors have presented a compelling approach to the critical challenge of jointly optimizing safety and performance in autonomous systems. The reviewers' concerns have been adequately addressed in the rebuttal. Based on the strength of the proposed framework and the authors' responses, this paper is suitable for acceptance.
Decision: Accept (poster)