V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions
Authors: Mumuksh Tayal
Manan Tayal
Aditya Singh
Shishir Kolathaya
Ravi Prakash
Venue: Accepted by TMLR ()

Abstract: Ensuring safety in autonomous systems requires controllers that aim to satisfy state-wise constraints without relying on online interaction.While existing Safe Offline RL methods typically enforce soft expected-cost constraints, they struggle to ensure strict state-wise safety. Conversely, Control Barrier Functions (CBFs) offer a principled mechanism to enforce forward invariance, but often rely on expert-designed barrier functions or knowledge of the system dynamics. We introduce Value-Guided Offline Control Barrier Functions (V-OCBF), a framework that learns a neural CBF entirely from offline demonstrations. Unlike prior approaches, V-OCBF does not assume access to the dynamics model; instead, it derives a recursive finite-difference barrier update, enabling model-free learning of a barrier that propagates safety information over time. Moreover, V-OCBF incorporates an expectile-based objective that avoids querying the barrier on out-of-distribution actions and restricts updates to the dataset-supported action set. The learned barrier is then used with a Quadratic Program (QP) formulation to synthesize real-time safe control. Across multiple case studies, V-OCBF yields substantially fewer safety violations than baseline methods while maintaining strong task performance, highlighting its scalability for offline synthesis of safety-critical controllers without online interaction or hand-engineered barriers.

Published in Transactions on Machine Learning Research (04/2026)
V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions
Mumuksh Tayal"
mamukshtaayal@iiscccccc
Center for Cyber Physical Systems (CPS) Indian Institute of Science (1ISc) Bengalura, India
Manan Tayal"
bmmnwanaayal@@nnicoooffttomm
Microsoft Research, India
Aditya Singh
singhadiSscas.uupe wupene edu
Department Electrical and Systems Engineering University of Pennsyloania, Philadelphia, USA
Shishir Kolathaya
shiihhikk
Center for Cyber Physicca Systems (CPS) Indian Instätute of Science (IISe) Bengalura, India
Ravi Prakash
ravipr@iisc.uc
Center for Cyber Physical Systems (CPS) Indian Institute of Science (IISe) Bengaluru, India
denotes equal first author contribution
Reviewed on OpenReview: httpps//geaaaeeieeeeeee///////t2
Abstract
Ensuring safety in autonomous systems requires controllers that aim to satisfy state-wise constraints without relying on online interaction. existing Safe Offline RL methods lypically enforce soft expected-cost. constraints, they struggle to ensure strict. slate-wise safety. Conversoly, Control Barrier Functions (CBFs) offer a principled mechanism en- force forward invariance, but often rely on expert-designed barrier functions or knowledge of the system dynamics. We introduce Value-Guided Offline Control Barrier Functions (V-OCBF), framework that learns neural CBF entirely from offline demonstrations. Unlike prior approaches, V-OCBF docs not assume access to the dynamics model; instead, derives a recursive fiile-difference barrier update, enabling model-free learning of a bar- rier that propagates safety information over time. Moreover, V-OCBF incorporates an expectilo-based objective that avoids querying the barrier on out-of-distribution actions and restricts updates to the dataset-supporled action set. The learned barrier then used with Quadratic Program (QP) formulation to synthesize real-time safe control. Across multiple case studies, V-OCBF yields substantially fewer safety violations than baseline methods while maintaining strong task performance, highlighting its scalability for offline synthesis of safety-critical controllers without online interaction or hand-engineered barriers.
1 Introduction
Ensuring the safety of autonomous systems is essential for their reliable and widespread deployment. From houschold service robots autonomous vehicles and aerial drones, these systems increasingly operate complex and unstructured environments where unsafe behavior can lead irreversible consequences. As

---

Published in Transactions on Machine Learning Research (04/2026)
autonomy becomes deeply integrated into transportation, manufacturing and healthcare, guarantecing that such systems operate within well-defined safety boundaries is critical for reliability, and long-term adoption.
Reinforcement learning (RL) has emerged as a powerful paradigm for enabling autonomous systems to acquire sophisticated control behaviors. However, in safety-critica domains, naive RL exploration can be hazardous. Although constrained RI. (CRL) (Achiam et al. 2017 Altman 2021: Alshiekh et al. 2018 Zhao et al.) 2023) methods attempt to incorporate safety constraints during learning, they typically require extensive online interaction with the environment. However, most previous studies focus online RL setting (Liu et al., 2024h, which suffers from serious safety issues in both training and deployment phases, especially for scenarios that lack high-fidelity simulators and require real system interaction for policy learning. As a result, there is a growing interest in synthesizing safe policies using offline RL or imitation learning (Levine et al. 2020 Kumar 2020). Nevertheless, most online and offline safe RL approaches (Xu et al. 2022 Ciftci et al., 2024 Stooke et al. 2020) treat safety as a soft constraint and regulate only the expected cumulative constraint violations. Such probabilistic constraints are insufficient for applications that demand strict state-wise safety, where even a single violation is unacceptable. Furthermore, jointly optimizing performance and safely from static datasets often leads to unstable training dynamics and overly conservative behavior, particularly when safety-critical transitions are sparsely (Lee et al. 2022
it al.
Control-theoretic tools provide an alternative and more rigorous foundation for safety. In particular, Control Barrier Functions (CBFs) (Ames et al.) 2014) offer a principled mechanism to enforce instantaneous safety constraints by guaranteeing the forward invariance of a prescribed safe set. When combined with learning- based controllers, CBFs serve as minimally invasive safety filters that adjust nominal actions only when necessary to prevent constraint violations. Their integration with Quadratic Program (QP) based controllers enables real-time implementation with modern optimization solvers. Consequently, CBF-based controllers have been successfully applied to a wide range of safety-criticn tasks, including adaptive cruise control (Ames) et al. 2014), aerial robotics (Wu & Sreenath, 2016 Tayal et al. 2024a), and legged locomotion (Nguyen & Sreenath 2015). In all of these applications, the performance and safety guarantecs fundamentally depend on the quality of the underlying CBF.
Constructing valid CBFs, however, is a challenging problem. Hand-crafting barrier functions requires deep system knowledge and does not scale well to high-dimensional or partially known dynamical systems. This has motivated significant. interest in Neural Control Barrier Functions (NCBPs), which leverage the ex- pressive power of neural networks to approximate complex safe sets. A variety of techniques have been proposed for learning NCBFs, including SMTT-aasd synthesis (Abate et al. 2021 2020), mixed-integer programming (Zhao et al. 2022) nonlinear optimization (Zhang et al. 20233; and loss- based training meth- ods (Dawson et al.] 2022 2023 Tayal et al. [202bb (025). Other recent approaches learn CBFs from value functions associated with nominal policies (So et al., 2024p. However, most of these methods rely on online interaction to collect informative samples or refine the barrier, which is often infeasible in safety-critical
settings.
Recent work has explored lcarning Control Barrier Functions (CBFs) from offline demonstrations (Robcy et al. 2020: Castañeda et al. 2023 Tabbara & Sibai 2025). Existing methods either fit CBFs only on expert trajectories or rely on data-likelihon likelihood measures to filter unsafe samples, which limits their ability to generalize beyond the demonstrated states. Uncertainty-aware approaches address distributional mismatch but often become overly conservative. Overall, current offline CBF learning methods are closely tied to the empirical data distribution and do not explicitly reason about. future system evolution, resulting in conservative safety.
This paper introduces Value-Guided Offline Control Barrier Functions (V-OCBF), a novel framework de- signed to overcome key limitations of existing offline RL and CBF-based approaches. We derive a model-free fmite-differrece barrier recursion and establish that, in the idealized setting without learning or approx- imation errors, enforcing this update provides a formal one-step forward-invariancee safety guarantee for control-affine systems. In addition, we propose an expectile-based learning objective that allows the syn- thesized safe policy to improve over the behavior poliey in the dataset while never querying the barrier on out-of-distribution actions, ensuring stable and reliable offline learning. To summarise, the main contribu- tions of this work are as follows:
2

---

Published in Transactions on Machine Learning Research (04/2026)
1. We propose V-OCBE, novel framework for learning safe controllers entirely from offline demon- strations.
We derive a model-free imite-difference barrier recursion and prove that adherence to this update guarantecs one-step forward invariance for any control-affine system.
3. We introduce the barrier outside the datasct action support.
sopectile-based objective that improves upon the behavior policy without evaluating
Across diverse systems, including high-dimensional Safety Gymnasium et al. 2023 tasks, V- OCBF consistently outperforms constrained offine RL and neural CBF baselines in both safety and reward.
2 Related Works
2.1 Safe Offline RL
Safe RL has traditionally been studied the online setting, where most methods rely on lagrangian-based constrained optimization (Chow et al. 2017 Tessler al. 2018 Stooke et al.) 2020)). These approaches regulate safety through cost penalties, treating constraints as soft and therefore allowing non-zero violation risk. CPO (Achiam et al., 2017) provides thcoretical guarantees during updates through trust-region mech- anism, but still cannot ensure strict safety throughout online learning. These limitations molivate shifting toward safe offline RL, where policies are synthesized from static data to avoid unsafe exploration. Among early offline-safe RL methods, CPQ (Xu et al.) 2022) assigns large costs to unsafe or out-of-distribution actions, but this can distort the value function and reduce generalization (Li et al. 2023). COptiDICE [Lee et al. )022), building on DICE objectives (Lee et al. 2021), inherits the difficulties of residual-gradien learning [Barrd] 1995). More recent work integrates safety considerations into Decision Transformer (Chen mooueiig-odellnng-- based safety (Liu et al. 2023 Zheng et al.) (024).. However, these architectures are computationally expensive and challenging to scale. Overall, whilc offline RL mitigates unsafe online interaction, existing methods primarily enforce soft constraints and often struggle when unsafe transitions are underrepresented or missing from the dataset.
et al.
2021 Diffuser models (Janner et al. 2022) enabling
2023
Lin et al.
2.2 Offline Neural CBFs
Complementary to RI-based strategies, several recent methods aim to learn Control Barrier Functions (CBF) directly from offline demonstrations. The approach in (Robey et al. 2020) enforces differentiable barrier conditions on expert trajectories but does not account for out-of-disttibutiio (OOD) states, limiting
generalization beyond demonstrated data. The iDBF framework (Castañeda et al. 2023 mitigates unsafe gencralization by filtering low-likelihood states and actions using a Behavior Cloned policy, although this reliance on BC likelihood restricts coverage when demonstrations do not span the full safe set. Conservative CBFs (Tabbara & Sibai 2025) incorporate uncertainty-awar penalties to avoid optimistic predictions OOD states, but this often yields overly cautious certificates that substantially underestimate the true safe region. While these approaches represent important progress, they fundamentally rely on the empirical demonstration distribution and do not explicitly model how system dynamics may evolve toward unsafe states. Consequently, they tend to produce conservative CBFs that under-approsee approxiape the true safe set.
2.3 Positioning of Our Work
V-OCBF positions itself primarily as a framework for lcarning safc control policies from offline datasets, utilizing Control Barrier Functions (CBFs) as a learned intermediate representation to enforce strictly safe behavior. Unlike safe offline RL methods (Lee et al. 2022 Liu et al. 2023 Lin et al. 2023) Zheng al. 2024), which rely on soft constraints, we dircctly construct a safety certificate and safe policy that guarantees one-step forward invariance, in the idealised setting. In contrast to model- based safe offline approaches that rely on learned dynamics both for training neural CBFs and during inference (Robey et al.
3

---

Published in Transactions on Machine Learning Research (04/2026)
V-OCBF employs model-free value recursion that models future safety evolution and uses dynamics only at the time of inference for local Lie derivative com- putation in the CBE-QP, rather than for policy optimization or imagined rollouts. This design prevents the policy from extrapolating into action regions unsupported by the dataset and preserves the OOD robust- ness of our expectile-basee based objective. Overall, our method combines the distributtonal robust of offline RL with the rigor of CBF-based control obtain an offline- leflnnelearned, actuation-aware neural CBF without requiring online rollouts.
2020
Castañeda al. 2023 Tabbara & Sibai 2025
3 Background and Problem Setup
We consider control-affine nonlinear dynamical system defined by the state z(t) E X S R", the control input u(t) E U C R", and governed by the following dynamics:
=(1) [(x(()) + + g(x(1))((().
(1)
R" + R and R" Ia are locally Lipschitz continuous functions. We are given set where f R"
C S X' that represents the safe states for the system and a failure set F S X that represents the set of unsafe states for the system (e.g., obstacles for an autonomous ground robot). Furthermore, the system is controlled by a Lipschitz continuous control policy т IR" + R Our focus lies in ensuring the safety of this dynamical system, which formally defined as follows:
Definition 1 (Safety). A dynamical system is considered safe if the set, C & CR, is positively inveriant under the control policy, n, i.e, x00) E C, u(t) = x<x(t)
x(t) E C, vtz 0.
Since, F X C, it can be trivially shown that xx(1) EC = al1) $ FVVIZ 0. Using this premise, we define the main objective of this paper:
Our objective to synthesize safe policy Taate 11, U such that the resulting closed-loop system satisfies the positive invariance property specified in Definition
Objective 1.
3.1 Control Barrier Functions
2017 are widely used to synthesize control policies with positive invariance guarantees, thereby ensuring system safety. The initial step in constructing Control Barrier Function (CBF) involves defining a continuously differentiable function B X' - I, where the super-level set of B corresponds to the safe region This leads to the following representation:
Control Barrier Functions (Ames al. 2014
C fx B(x) 0), x & B(x) 0}.
(2)
The interior and boundary of C are further specified as:
Int fxr E x: B(x) 0}, ac for E X: B(x) 0}.
(3)
The function à qualifies as a valid Control Barrier Function if it satisfies the following definition:
Definition 2 (Ame et al. 2017 Given control-affine system i f(x) + g(z)u, the set C defined by equation 2 with az (x) # for all x E aC, the function B is called the Control Barrier Function (CBF)
dB
defined on the set X, if there erists on exlended class-K, function &. such that for all x E X:
max W
B(x) B.) (B(x)) 20.
(4)
B(z,a)
where B()))
0B [() and C, B(x) '0%
AB g(x) are the Lie derivalives and is the dimension of the system.
8z
4

---

Published in Transactions on Machine Learning Research (04/2026)
As established in {Ames et al. 2017 any Lipschitz continuous control law x(x) that satisfies the condition B x(B) 2 0 guarantees the system's & safety when x<0) E C. Additionally, if the initial state xx0) lies outside C, this condition ensures asymptotic convergence to the safe set C.
While CBFs provide a principled framework to guarantee safety, their practical deployment is hindered by the lack of general methods for constructing valid barrier functions. As a result, practitioners typically resort handcrafted or domain-specifie CBFs, which can yield overly conservative safe sets. Furthermore, in the presence of control bounds, nominal CBF may conflict with feasibility requirements, causing the corresponding CBF-QP QP become infeasible.
3.2 Control Barrier Value Function (CBVF)
To overcome the limitations inherent in classical CBF formulations, (Choi et al. 2021) proposed the Control Barrier Value Function (CBVF), which integrates Control Barrier Functions with Hamilton-Jacobi (HJ) Reachability [Bansal et al. 2017). We begin by encoding the safety specification using a Lipschitz continuous function In + IR, where the failure set defined as F = {a E X ((x) < 0}. Under this construction, a CBVF - IR defined as the viscosily solution of the following Hamilton Jacobi-Bellman Variational Inequality (HJJB-I)::
m (B(z) (f)) + g(z)u)), (xx) -(x) }= =0. }
min
max NEU
(5)
with boundary condition B(=)|== = {(z))==0. The resulting value function induces forward -ivarrinn safe set C:= fx X B(x) 2 0} and ensures that admissible controls E U satisfy the following Lie-derivative condition:
max WEU
[, Lg (((())))++(BB)))))))))) 20. 0.
+
(6)
Quite often, we have a reference control policy, Fres(z), de- signed meet the performance requirements of the system. However, such controllers often lack safety guarantees. To ensure the system meets its safety requirements while preserving performance, the ref- erence controller must be minimally adjusted to incorporate safety constraints. This adjustment can be accomplished using the Control Barrier Value Function-based Quadratic Program (CBVF-QP), described follows:
Safe Controller Synthesis using CBVFs:
Teale (x) = min ||u - Tre 2|? wEUCR mm
(7)
s.t. B( + B( 0.
The CBVE-QP framework facilitates the synthesis of provably safe control policy, Tsate (æ), while staying close to the reference controller to preserve system performance.
Challenges in CBVF Synthesis: Traditional approaches compute Control Barrier Value Functions us- ing grid-based HJ reachability methods [Mitchell 2005). which are fundamentally limited by the curse of dimensionality. Recent efforts have attempted to overcome these issues by learning CBVFs through online reinforcement learning (So et al. 2024); however, as discussed Section such approaches require extensive online interaction, rendering them unsuitable for safety-critical systems. Furthermore, existing neural CBF methodologies (Abate et al. 2020 Zhang et al.) 2023 Tayal et al., 2024b) typically assume access to accu- rale system dynamics, an assutnplio that does not hold for many real-workd platforms. These limitations motivate a shift toward using offline demonstrations, either sourced from public datasets (Liu et al., 2024 Sun et al., 2020) or collected in controlled settings where safety can be guaranteed. Building on this premise, we refine Objective as follows:
Our objective is to synthesize a Control Barrier Value Function B : X' - RR directly from an offline dataset of demonstrations D, such that the resulting CBVF- QP controller Taate in equation 7 aims satisfy the positive invariance property specified in Definition
Objective 2.
5

---

Published in Transactions on Machine Learning Research (04/2026)
4 Methodology
Having introduced the CBVF formulation in the previous scction, ology for syuthesizing a valid Control Barrier Value Function and thereby the sale controller from offline demonstrations D. Our objective is to construct a data-driven approximation of the viscosity solution of the HJB-VI oquation 5 without relying on known system dynamics or online interaction. The key idea is to re-interpret it through a differeife barrier recursion compatible with demonstration data, and to use this recursion to learn a value-guided barrier function that inherits forward invariance.
describe practical method-
4.1 Finite Diffee-Dfffeeence Barrier Synthesis
To approximate the CBVF using trajectories in D, consider finite difference recursive version of equa- tion equation Given trajectory {2r, 24 l we define the finite-difference barrier update
B(x,) min {((x)), max B(x41)) ty
Vt e 20..,
(8)
where, =(t), un u(t) and Z441 x(t + At). This recursion has two significant advantages. First, it enables us to learn barrier directly from data without requiring and g. Second, under mild regularity assumptions, equation g preserves the forward invariance property of the CBVF. The derivation of the recursive equation is provided in Appendix A.1. Intuitively, the recursion encodes the principle that a state is safe and only if its immediate successor is safe or it lies outside the unsafe set as specified by E(x).
To represent this barrier function, we parameterize neural network (x) with eb utilizing the universal approximation property. However, directly solving for the recursion equation can load to degenerate solutions. For instance, setting (x) for a sulficiently small constant satislies equation S but clearly does not satisfy the CBVF conditions in equation 5 This pathology is analogous to the non-contractive behavior of undiscounted value iteration in MDPs. Following the approach in (Fisac et al. 2019). we incorporate a discounted finiteddiffrrncc value to avoid such trivial solutions:
Barget = 71 ((x) + min (((x), max (x()),
(9)
where x = x4, x¹ = Fx+1 and u u and - 1. This discounted recursion ensures contraction, promotes stable lcarning, and prevents the network from collapsing to uniformly unsafe or uniformly safe solutions. To avoid evaluating barrier targets al out-of-distribution actions, we remove the maximization over actions from the equation g and use only the demonstrated action in each transition. While this prevents unsupported queries, the resulting estimate reflects the safety profile of the behaviour policy that generated the data. Selaviour-induced barrier, which typically sub-optimal because it ignores
Consequently, this produces other admissible actions that could yield larger safe-set estimates.
4.2 Avoiding But-of-Distributie Actions in Offline Learning
The naïve regression objective in equation 9 with the maximization over actions dropped, fits B to the mean of the demonstrated next-state targets, but this corresponds to the behavior-inducet barrier and yields overly conservative safe sets. Tdeally, if we assume unlimited capacity and no sampling error, the optimal parameters should satisfy, B(x)= xs Ey] min {((x), max,, B(x')}] However, such unconstrained maximization can result in actions that are never observed in the dataset.
Since offline data only provides information about those actions selected by the behavior policy, evaluating values (or barrier targets) using unsupported actions can distort lcarning because the corresponding tran- sitions are not grounded in the dataset. Subsequently, motivated by the insights of Implicit Q-Learning (IQL) (Kostrikov et al. 2022), under idealized conditions, deterministic dynamics and disturbance free data, prioritizing the best observed next-state outcomes can yield strong empirical performance, since emphasizing high-value targets leads to safer barrier estimates. However, this naive maximization is brittle in practice: rare poor actions or stochastic disturbances can occasionally produce optimistic next states, and focusing

---

Published in Transactions on Machine Learning Research (04/2026)
Phase 1: Training
Phase 2: Inference
Learning Control Affine Dynamics
Reference Controller
Rasle) Bynrro [1r (1 Sulehoaal
fa(e)
Demonstrations
Offline
Et
21+1
(o(r)
Mref
D
Rst1 = Xs 4 (fo(as) + go(n))o...At
Unsafe Control Action Sequence
Guided Barrier Function
Safe Controller Synthesis
Reachability Bellman
Expectile Regression
CBF-QP
Equation
1 (2x) min(((2)) B (reil)
(T) B()))
min s. VBir) if(a) + +.))))) -ai
Meyll
Mar)
Exl (())
Framework Overview: (Left): We learn value guided barrier function with reachability based bellman equation for learning optimal safe region and apply expectile regression for OOD case handling, when learning from Offline Dataset. (Right): Inferencing with CBF-QP using learned barrier function as valid CBF to rollout safe step-wise actions, with any reference controller.
Figure 1:
only on the single best outcome can cause the learner to overfit such spurious transitions. Expectile re- gression provides a simple remedy by emphasizing high next-state barrier values that actually appear under dataset-supported actions, rather than committing to single extreme outcome. In this way, it preserves the benefit prioritizing good actions while remaining robust noise and avoiding evaluations on unseen (x, u) pairs. For details and sensitivity to r, see Appendix A.2 and Appendix C.3 respectively. This allows to perform principled value-style backup over the dataset control action without extrapolating to unsafe or unobserved actions. This IQL-inspired method shows that expectile regression produces value function that reflects the values induced by the behavior policy without requiring an explicit behavior model. This prevents the lcarning target from boing inffuenced by actions that lic outside the datasct support, while still capluring the highest feasible values supported by the demonstrations.
Following this principle, we estimate CBVF, Be, with 0 the Neural Network parameters, that reflects the safety values implied by demonstrated actions. Formally, minimize the expectile loss
CB(0) = -E(,, C ("
(BPaggec (
(10)
where (T)) =1 1 0)]y is the 7-expectile loss used in Kostrikov et al. 2022 and we replace the origi- nal objective term BT arget from equation 9 with the new objective Borget (117-)46 (xx) +y min Ba Ba(a)) which incorporates (a(x') in place of max,, B (x'). Intuitively, higher expectile level places greater weight underestination errors than overestimation errors, pushing B, toward the upper envelope of safety values supported by the dataset. Thus, controls how aggressively the learned barrier emphasizes high, data- supported safety values without extrapolating to unseen actions. In practice, this is implemented by com- puting the expectile loss over the empirical stale-action transitions within each sampled minibatch, thereby restricting the updates to the sutpset--upported action set without requiring explicit support construction. The barrier function thus obtained, Bg, is our proposed Value-guided Offline Control Barrier Function (V- OCBF). The complete learning procedure for B and the V-OCBF Bg is summarized in Algorithm and be visually seen in Fig.

---

Published in Transactions on Machine Learning Research (04/2026)
Algorithm Learning V-OCBF from Offline Demonstrations
Require: Offlin dataset D, expectile level batch size m, learning rates no na. discount factor 1: Initialize 4, 0 and optimizers
2: for epoch = to N do
3: for minibatch B C do
CB - [8] Eau L 5: 0 - AdamStep(0, VoLB)
1
L* (Br (x))
From Eq.
10
6: end for
7: end for
Return B
4.3 Controller Synthesis via Learned Dynamics
The learned barrier function B subsequently employed to fulfill the primary goal of synthesizing a safe policy ! using the CBVF-QP formulation in equation | Solving this QP necessitates the evaluation of the Lie derivatives B and C, both of which rely the underlying control-afiine system dynamics. In our offlin--only setting, the true dynamics are unavailable; therefore, construct neural network based surrogate model to approximate the underlying transition dynamics of the form:
F++1 = 2t + (fo(1) + (o())) At,
(11)
which enables computation of the required derivatives and supports safe policy synthesis. The model pa- rameters & are trained using one-step transitions from the offline dataset D by minimizing the prediction loss:
Layn = (( [| - (x (fo(z) + (o()) IP].
(12)
implemented as minibatch MSE objective. The full training procedure is summarized in Algorithm
2
Importantly, the learned dynamics model is not used when learning the barrier function. Incorporating it into the CBVF learning stage would require evaluating terms involving (Je, go) under actions outside the dataset- supported set Up, thereby violating the action constraints critical for preventing value underestimation in the offline regime. Hence, using learned dynamics during CBVF training would allow the network extrapolate into unsupported regions of the action space, defeating the purpose of the OOD-aware barrier learning objective described earlier.
In contrast, at inference time, the learned dynamics serve a different role: they enable the evaluation of Lie derivatives needed to solve the CBVF-QP (equation 7 Specifically, for any query state I, we compute
B fz()),
(,B(x)=V,,BA)) gz(x).
(13)
These quantities allow the QP in equation 7 to be solved for the safe control action usafes completing the pipeline for constructing a safe controller purely from offline demonstrations.
Note that one-step invarad-nvariaane holds under the idcalized assumptions (exact barrier and exact Lic- derivatives), and the approximation sources like function approximation, finite-data estimation, and model error can break the closed-loop guarantee in practice. In Section 5.2.2 provide analysis demonstrating the limitations of using learned dynamics inside the CBVF learning loop, further reinforcing the necessity of restricting their use to the inference stage only.
5 Experiments
The experiments are designed to evaluate: offline RL and neural CBF baselines on systems with unknown dynamics, (ii) the advantages of value-guided barriers over eehavior-policy-inneee
(7) the safety and performance of V-OCBF relative to constrained
barriors, (Mi) the robustness of the resulting QP controller under external disturbances, and (fv) the effectiveness of V-OCBF compared CBVF synthesized using learned dynamics.
8

---

Published in Transactions on Machine Learning Research (04/2026)
Algorithm 2 Controller Synthesis
Require: Offiine dataset D {(x, u, 2,)) batch size m, epochs N, learning rate, At, ure (t),((),
I: Initialize & and optimizer
2: for epoch = 1 to N do
3: for minibatch B C D do
4: (o) + (z, u, x', Ø)
From Eq.
Φ f AdamStep(d V Voldyn))
end for end for
8: Inference at state å
9: L + (V, (o(T) and B() + (V.Bo(o),
From Eq. b From Eq
13
10: Usafe + Qp-Solver (L,, B(t), Lg), (i), (o)
11: return safe controller ($)
Baselines: We compare V-OCBF against a diverse set of constrained offline learning and CBF-based meth- ods. For constrained offline learning, we include Behavior Cloning (BC), BEAR- (Lagrangian constraint version of (Kunar et al. 2019)); COptiDICE (Lee al. 2022 and FISOR (Zheng el. al., 2024) which enforce safety indirectly via soft constraints policy optimization or behavior imitation. For CBF-based approaches, evaluate Neural Control Barrier Function (NCBF) (Robey et al. 2020). Conservative Control Barrier Function (CCBF) (Tabbara & Sibail 2025), and In- Disstriibuuiin Barrier Func- tăon (iDBF) (Castañeda et al. conservative safe sets. In contrast, V-OCBF learns a value-guided barrier function from offline demonstra- tions that accounts for future unsale interactions, producing a state-wise safety filter that aims to reduce violations and increase safe-set coverage practice.
2023 which synthesize explicit safety filters from offline data but often yield
Evaluation Metrics: We evaluate all methods based (i) safety, measured as the total number of safely violations incurred before episode termination, and (ii) performance, measured via the cunulative episode rewards. These metrics allow us to assess the trade-off between strict safety enforcement and task performance across different offline RL and CBF-based approaches.
5.1 Experimental Case Studies
To perform a holistic performance analysis our proposed approach, we apply V-OCBF in conjunction with Behavior Cloning (BC) as the nominal (reference) controller for all the different environments which are supposed assess varying objectives. Below we list all the environments that we use:
Autonomous Ground Vehicle (AGV) Collision Avoidance: In our first experiment, we exam- ine 3-dimensional collision avoidance problem involving an autonomous ground vehicle governed by Dubins' car dynamics (Dubins) 1957). The objective to ensure safety by avoiding a static obstacle while navigating through bounded environment. Further details on the system dynamics, state space bounds, and experimental setup provided in Appendix B.1
MuJoCo Safety Gymnasium: We next evaluate our framework on Safety Gymnasium envi- romments (Ji et al. 2023). Specifically, we evaluate the V-OCBE-based QP (equation 7 on high- dimensional MuJoCo tasks like Hopper, Swimmer, Half Cheetah, Walker2D and Ant. The objective in each environment
maximize reward while keeping the agent velocity below the velocity thresholds. We keep the reward and safety-violation metrics identical definitions and use the standard DSRL dataset for safe offinne RL (Liu et al. 2024). To evaluate our method against baselines, we randomly sampled 500 initial states for each environment, respectively, the results for which can be referred to from Figure 2
the Safety-Gymnasiie

---

Published in Transactions on Machine Learning Research (04/2026)
Reward (Episode Final Reward)
Hopper
Swimme mer
HalfCheetah
1600
1000
1.400
900
1200
1000
105
*
100
600
90
85
20
Episode Safety Violations
Episode Safety Violations
Episode Safety Violations
Reward (Episode Final Reward)
Walker2D
Ant
BC
BEAR-Lag
1200
COptiDICE
500
450
1000
FISOR
BC+NCBF
800
BC+IDBF
BC+CCBF
300
600
BC+V-OCBF
250
20
10
Episode Safely Violations
Episode Safety Violations
Figure 2: Results plot for all the Mujoco based Safety Gymnasium (Ji et al. 2023) environments. Points
towards left. (+-) are more safe than those on right: (-+). Evaluated over 500 episodes and 5 seed values.
5.2 Results
5.2.1
Effectiveness in Co-optimizing Safety and Performance
We begin by ovaluating all methods the AGV Collision Avoidance task, which provides clear setting to study how different approaches balance safety and performance. The results in Table D highlight notable differences in how offline RL and based methods handle this trade-off.
Offline RL basclines such as BC, BEAR-Lag, and COptiDICE achieve relatively low safety rates. BC tends to reproduce unsafe behaviors from the dataset, while BEAR-Lag and COptiDICE try to account for safety but remain limited because they operate with soft constraint formulations. Their lower reward and safety scores indicate that they struggle to balance both safety and performance objectives.
In contrast, methods that incorporate CBF-QP layer, such BC+NCBF, BC+iDBE, and BC+CCBF, achieve much higher safety rates. The QP ensures that unsafe actions are filtered out, even if the nominal controller is imperfect. However, the performance of these approaches still depends heavily on the quality of the learned barrier. FISOR performs better than the other offline RL baselines because explicitly expands the feasible safe region before optimizing for performance. However, due to lack of explicit safety filtering, it Icads to lesser safety rates than our proposed method, due the impending learning errors the computation of feasible region. This also highlights the importance of QP based safety filtering scheme for achieving better safety. Overall, V-OCBF achieves the strongest results across all metrics.
To further analyze scalability, we extend the evaluation to MuJoCo Safety Gymnasium environments (Hop- per, Half-Cheetah, Ant, Swimmer, and Walker2D), with unknown dynamic models. The results in Figure 2 demonstrate that V-OCBF again achieves the lowest safety violation rates across all tasks. Notably, the method maintains ncar-zero violations on while prescrving satisfactory reward levels compared to BC and outperforming iDBF, NCBF, and CCBF. These neural CBF baselines degrade sharply in higher dimensions: NCBF suffers from optimization difficulties, while iDBF often enforces overly restrictive boundaries that
10

---

Published in Transactions on Machine Learning Research (04/2026)
Table 1: AGV Collision Avoidance Experiment: Percentage Safe Episodes, Mean Episode Reward and Safe
Set Volume (refer Appendix B.1.1
across different methods. Evaluated over 500 episodes and 5 seed values.
Method
Safe Episodes (%)
Episode Reward
Safe Set Volume (%)
BC
48.92 1 1.69
20.45 1.84
42.51
BEAR-Lag Kumar
2019
65.12 i 0.24
13.85 1 0.81
58.21
COPLDICCE
Lee et al.
2022
68.91 + 0.32
15.33 1 0.67
62.32
BC+NCBF (Robey
2020
92.48 0.60
44.61 2.58
81.92
BC+iDBF Castañeda et a..
2023
92.87 + 0.73
48.23 + 2.01
83.32
BC+CCBF TTbbara & Sibai
2025
93.56 0.56
49.66 2.34
90.94
FISOR (Zeng et al. 20244
95. 35.78 + 0.2
52.33 + 0.93
90.14
BC+V-OCBF (Ours)
98.28 + 0.54
54.93 0.46
92.57
suppress task performance. FISOR again remains competitive but does not match the safety consistency of V-OCBF.
Overall, the experiments provide strong empirical evidence that V-OCBF effectively co-optimizes safety and performance, scaling from low-dimensional AGV dynamics to complex MuJoCo systems. The method consistently outperforms existing offline RL and neural CBF basclines in terms of safcty while maintain- ing competitive reward, highlighting its suitability for offiine settings where both strict safety and reliable performance are required.
Without CBF
With V-OCBF
100
Percentage Rollouts > Threshold
Instantaneous Volocity
0.6
50
25
0.0
Timestop
Timestep
Velocity
Velocity
(25
Figure 3: Qualitative comparison of Hopper rollouts with and without V-OCBF. (Top): When using V-OCBF, the agent adapts its gait to keep its forward velocity within the allowable safety threshold, demonstrating how the learned barrier actively regulates unsafe accelerations. (Bottom): Instantaneous velocities across episodes for both methods. The left axis reports the velocity profile, while the right axis shows the fraction of episodes that violate the safety threshold each timestep. V-OCBF substantially roduces violation frequency, illustrating its ability to enforce safety constraints during deployment. Additional qualitative results for other environments provided in Appendix B.2.1
5.2.2 Effect of learned dynamics
We additionally examine how using the learned dynamics influences both CBVF learning and QP-based control on the AGV environment, where ground-truth dynamics available for comparison. Table reports
11

---

Published in Transactions on Machine Learning Research (04/2026)
safety and reward for three configurations: (i) V-OCBF with a QP evaluated using learned dynamics, (ii) V-OCBF with QP evaluated using true dynamics, and (iii) a neural CBVF trained using learned dynamics and QP-based controller evaluated on the same dynamics.
Across all metrics, using learned dynamics at inference time yields performance that is close to the known- dynamics case, indicating that, the controller synthesis using QP is robust to moderate model error. Tn contrast, using learned dynamics during CBVF training leads to noticeable drop in both safety and reward. This aligns with our carlier discussion in section [.. learning the barrier through model introduces distributional mismatch because the learned dynamics may generate actions outside the sapasee-supported set Up, yielding suboptimal targets for B. These findings reinforce our design choice, dynamics should be used only at inference to compute the lie derivatives in the QP, while barrier learning itself should avoid dependence on a learned model.
Table 2: Use of Learned Dynamics at the time of Training v/s Inference (Mean over 500 episode rollouts).
Method
Safe Episodes (%)
Episode Reward
BC+ V-OBBE (QP with Learned Dynamics)
98.28 = 0.54
54.93 + 0.46
BC+ V-OCBF (QP with Known Dynamics)
99.52 0.58
55.91 1 0.39
CC+C from Learned Dynamics
94.64 = 0.65
46.40 + 0.02
5.3 Ablation Studies and Additional Experiments
To further evaluate the robustness of V-OCBF, we conduct a series of studies that examine: (i) the sensitivity of the learned safe set to the size of the barrier network, (ii) the effect of disturbances on the safety of QP based controller, (iii) the dependence of the learned barrier on the expectile parameter т, and (iv) the resilience of the framework to approximation errors in the learned dynamics model. Additionally, we extend our evaluation to a boat navigation task with nonlinear drift to assess performance under complex environmental dynamics. Across these experiments, V-OCBF exhibits stable safety performance over a broad range of architectural and hyperparameter choices, remaining resilient even under perturbations to the control actions. These findings suggest that the framework generalises reliably beyond the specific tasks considered and is well-suited for settings where safety cortification must be carried out under imperfect system knowledge. The full results and visualisations for these studies are presented in Appendix C and
Appendix D
6 Conclusion, Limitations and Future works
This work presented V-OCBF, a value-guided framework for learning control barrier functions directly from offline demonstrations. By combining finitoddifference CBVF recursion with expectilo- objective, V-OCBF propagates safety information in principled manner while avoiding evaluations unsupported actions, enabling reliable CBF-QP control even when dynamics are unknown and action authority is limited. Experiments across AGV and high-dimensional Safety Gymmasium environments show that V-OCBF consis- tently reduces safety violations while maintaining strong task performance, outperforming both constrained offline RL. and neural CBF baselines.
While V-OCBF reliably learns useful control-barrier candidates from offline demonstrations and enables Qp-based safety filtering under unknown dynamics and limited action authority, the learned barrier is not formally verified certificate, function approximation and learned-dynamics errors mean guarantees remain empirical. Furthermore, the framework is not explicitly trained to handle adversarial disturbances or worst-case model errors. To close this gap, future work will explore integration of formal-certi io (e.g., Lipschitz-bar based verification (Tayal et al., 2024b) and conformal prediction (Lindemann et al. 2025 ). Morcover, we will explore robust extensions of V-OCBF that incorporate adversarial or uncertainty-awarg training objectives, enabling safety guarantees under stronger disturbance models and broader distribution shifts.
12

---

Published in Transactions on Machine Learning Research (04/2026)
References
Alessandro Abate, Daniele Ahmod, Mirco Giacobbe, and Andrea Peruffo. Formal synthesis of Iyapunov neural networks. IEEE Control Systems Letters, 5(3):773 778, 2020.
Alessandro Abate, Daniele Ahmed, Alec Edwards, Mirco Giacobbe, and Andrea Peruffo. Fossil: A software tool for the formal synthesis of Lyapunov functions and barrier certificates using neural networks. Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control, pp. -1, 2021.
Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17, pp. 22 22-31. JMLR.org, 2017.
Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu. Safe reinforcement, learning via shielding. Proceedings of the AAAI Conference on Arlificial Intelligence, 32 (1), Apr. 2018. doi: 10.1609/aai /tt./aaaa.v.....111.... v3211.11797. URL https::/ojs. aaai fong/index.. php/ AAAI/ (MAI/artie artiice// view/11797
Eitan Altman. Constrained Markov decision processes. Routledge, 2021.
Aaron D Ames, Jessy W Grizzle, and Paulo Tabuada. Control barrier function based quadratic programs with application to adaptive cruise roltrol. In 53rd IEEE Conference on Decision and Coutrol, pp. 6271 6278. IEEE, 2014.
Aaron D. Ames, Xiangru Xu, Jessy W. Grizzle, and Paulo Tabuada. Control barrier function based quadratic programs for safety critical systems. IEEE Transactions on Automatic Control, 62(8))3861 3876, 2017. doi: (0.1109/tac.20) 2e..222016..9690
Leemon Baird. Residual algorithms: Reinforcement learning with function approximation. In Machine Learning Proceedings 1995, pp. 30 37. Elsevier, 1995.
Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 2242 2253. IEFE, 2017.
Fernando Castañeda, Haruki Nishimura, Rowan Thomas McAllister, Koushil Sreenath, and Adrien Gaidon. In-distribution barrier functions: Sclf-supervised policy filters that avoid out-of-distribution states. In Nikolai Matni, Manfred Morari, and George J. Pappas (eds.), Proceedings of The 5th Annual Learning for Dynamics and Control Conference, volume 211 of Proceedings of Machine Learning Reseorch, pp. 286-290 PMLR, 15-16 Jun 2023. URI. https::/pprocedd
mlr www.mmllrr........ ppwss/v21s11eee1eeppppppppppppie v211 castaneda23a. htmml
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Ar- avind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 94:15084-15099 2021.
Jason J Choi, Donggun Lee, Koushil Sreenath, Claire Tomlin, and Sylvia L Herbert. Robust control barrier value functions for safety-critical control. In 2021 60th IEEE Conference on Decision and Control (CDC), pp. 6814 6821. IEEE, 2021.
Yinlam Chow, Mohammad Ghavanzadeh, Lucas Janson, and Marco Pavone. Risk-constrained reinforcement learning with percentile risk criteria. The Journal of Machine Learning Research, 18(1):6070-61) 2017.
Yusuf Umut Ciftci, Darren Chiu, Zeyuan Feng, Gaurav S Sukhatme, and Somil Bansal. Safe-gil: Safety guided imitation learning for robotic systems. arXiv preprint XXi::2404.05244 2024.
Charles Dawson, Zengyi Qin, Sieun Gao, and Chuchu Fan. Safe nonlinear control using robust neural Lyapunov-barrier functions. In Conference on Robot Learning, pp. 1724-1735 PMLR, 2022.
13

---

Published in Transactions on Machine Learning Research (04/2026)
Charles Dawson, Sicun Gao, and Chuchu Fan. Safe control with learned certificates: A survey of neural Lyapunov, barrier, and contraction methods for robotics and control. IEFF Transactions on Robolics, 2023.
Lester E Dubins. On curves of minimal length with a constraint on average curvature, and with prescribed initial and terminal positions and tangents. Americon Journal of Mathematics, 79(3):497-516, 1957.
Jaime F. Fisac, Neil F. Lugovoy, Vicenç Rubies-Royo, Shromona Ghosh, and Claire J. Tomlin. Bridging hamilton-jacobi safety analysis and reinforcement learning. In 2019 International Conference on Robolics and Automation (ICRA), pp. 8550 8556, 2019. doi: 10.1109/ICRA.200 2019.879411
Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning, pp. 9902-9913. PMLR, 2022.
Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuchai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Josef Dai, and Yaodong Yang. Safety gymnasium: A unilied safe reinforcement learning benchmark. In Thirty-seventh Conference on Newral Information Processing Systems Datasets and Benchmarks Track, 2023. URL
Ily Kostrikov, Ashvin Nair, and Sergey Levine. Offinee reinforcement learning with implicit q-learning. International Conference on Learning Representations, 2022.
Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off-policy q-learning via bootstrapping error reduction. Advances in neural information processing systems, 32, 2019.
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for of- fline reinforcement learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.). Advances in Neural Information Processing Systems, volume 33, pp. 1179-1191, Curran Associates, Tnc., 2020. URL https://proceedinggs www.inng....urs....... cctpcc/pppppp...eeeeecccceee.eee ptlles/paper/0220/tll. 2tt1:0618222da222211111188888882
pdf
Jongmin Lee, Wonseok Jeon, Byungiun Lee, Joelle Pineau, and Kee-Eung Kim. Optidice: Offline policy optimization via stationary distribution correction estimation. In International Conference on Machine Learning, pp. 6120 6130. PMLR, 2021.
Jongmin Lee, Cosmin Paduraru, Daniel J Mankowitz, Nicolas Heess, Doina Precup, Kee-Fung Kim, and Arthur Guez. COptiDICE: Offline constrained reinforcement learning via stationary distribution correction estimation. In International Conference on Learning Represenlations, 2022. URL httpp:://penreevier het//forummii=FFAA55Beeee
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint rxXi:2005.01643 2020.
Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, and Ya-Qin Zhang. When data geomelry meets deep function: Generalizing ofline reinforcement learning. In The Eleventh International Conference on Learning Representations, 2023.
Qian Lin, Bo Tang, Zifan Wu, Chao Yu, Shangqin Mao, Qianlong Xie, Xingxing Wang, and Dong Wang. Safe offline reinforcement lcarning with real-time budget constraints. In International Conference on Machine Learning, 2023.
Lars Lindemaun, Yiqi Zhao, Xinyi Yu, George J Pappas, and Jyotirmoy V Deshmukh. Formal verification and control with conformal prediction: Practical safety guarantees for autonomous systems. IEEE Control Systems, 45(6):72 122, 2025.
Zifan LIU, Xinran Li, and Jun Zhang- C2IQL: Constraint-onndtioned implicit leearning for safe offline reinforcement learning. Forty-second International Conference on Machine Learning, 2025. URL https: htwp:arrwweeee.eeeooooriiiiiieee
14

---

Published in Transactions on Machine Learning Research (04/2026)
Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, and Ding Zhao. Constrained decision transformer for offline safe reinforcement learning. In International Conference on Machine Learn- ing, 2023.
Zuxin Liu, Zijian Guo, Haohong Lin, Yihang Yao, Jiacheng Zhu, Zhepeng Cen, Hanjiang Hu, Wenhao Yu, Tingnan Zhang, Jie Tan, and Ding Zhao. Datasets and benchmarks for offline safe reinforcement learning. Journal of Dala-centric Machine Learning Research, 2024.
lan M. Mitchell. A toolbox of level set methods. In Toolboz of Level Set Methods, 2005. URL https:
7appi aaapaaattccccoola...ooo///aaaaaa
Quan Nguyen and Koushil Sreenath. Safety-critical control for dynamical bipedal walking with precise footstep placement. IFAC-PapersOnLinee 48(27) 2015.
Alexander Robey, Haimin Hu, Lars Lindemann, Hanwen Zhang, Dimos V. Dimarogonas, Stephen Tu, and Nikolai Matni. Learning control barrier functions from expert demonstrations. In 2020 59th IEEE Con- ference on Decision and Control (CDC), pp. 3717 3724, 2020. doi: D.1109/CDC42340 200.3400.220
Oswin So, Zachary Scrlin, Makai Mann, Jake Gonzales, Kwesi Rutledge, Nicholas Roy, and Chuchu Fan. How to train your neural control barrier function: Learning safety filters for complex input-constrained systems. In 2024 IEFF International Conference on Robolics and Aulomation (ICRA), pp. 11532-11539. IEEE, 2024.
Adam Stooke, Joshua Achiam, and Picter Abbeel. Responsive safety in reinforcement learning by PID lagrangian methods. In Hal Daumé III and Aarti Singh (cds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 9133 9143. PMLR, 13 18 Jul 2020. URL httpps://proceeddngg. mlr. mlr wwwps/veiiieeeeeeeee http//v11///sseee/e wwtp////t/oee...aatooo html
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Ihab Tabbara and Hussein Sibai. Learning neural control barrier functions from offline data with conser- vatism, 2025. URL
Manan Tayal, Rajpal Singh, Jishnu Keshavan, and Shishir Kolathaya, Control barrier functions in dynamic uavs for kinematic obstacle avoidance: A collision cone approach. In 2024 American Control Conference (ACC), pp. 3722 3727. IEEE, 2024a.
Manan Tayal, Hongchao Zhang, Pushpak Jagtap, Andrew Clark, and Shishir Kolathaya. Learning a formally verified control barrier function in stochastic environment. In 2024 IEEE 63rd Conference on Decision and Control (CDC), pp. 4098 4104, 2024b. doi: (B00//CCC52242.22...220
Manan Tayal, Aditya Singh, Shishir Kolathaya, and Somil Bansal. A physics-informed machinc learning framework for safe and optimal control of autonomous systems. In Forty-second International Conference on Machine Learning, 2025. URL htttps://openreview htt.:ppwwwrreeeeeeeeoeo/o/aaaaaa .et.//orruniieeeffelll....
Chen Tessler, Daniel J Mankowitz, and Shie Mannor. Reward constrained policy optimization. In Interna- tional Conference on Learning Representations, 2018
Honghao Wei, Xiyue Peng, Arnob Ghosh, and Xin Lin. Adversarially trained weighted actor-critic for safe offline reinforcement learning, In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024. URL https:://penrrviict hwt./oormmi.---22aaa/a/aaa httpps//aaaaoooovveeeee//oomooad
Guofan Wu and Koushil Sreenath. Safety-critical control of a planar quadrotor. In 2016 American Control Conference (ACC), pp. 2252-2258. 2016. doi: AA1100/ACC.2206.5555555
15

---

Published in Transactions on Machine Learning Research (04/2026)
Haoran Xu, Xianyuan Zhan, and Xiangyu Zhu. Constraints penalized q-learning for safe offinne reinforcement learning. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 36, pp. 8753-87660 80368,8600-00 2022.
Hongehao Zhang, Junlin Wu, Yevgeniy Vorobeychik, and Andrew Clark. Exact verification of relu neural control barrier functions. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (ods.), Advances in Neural Information Processing Systems, volume 36, pp. 5685-5705. Curran Associates, Inc., 2023.
Qingye Zhao, Xin Chen, Zhuoyu Zhao, Yifan Zhang, Enyi Tang, and Xuandong Li. Verifying neural network controlled systems using neural networks. In 25th ACM International Conference on Hybrid Systems: Computation and Control, pp. 1-11, 2022.
Weiye Zhao, Tairan He, Rui Chen, Tianhao Wei, and Changliu Liu. Safe reinforcement learning: A survey. arXiv preprint 090200200 2023.
Yinan Zheng, Jianxiong Li, Dongjie Yu, Yujie Yang, Shengbo Eben Li, Xianyuan Zhan, and Jingjing Liu. Safe oflline reinforcement learning with feasibilitt-guuided diffusson model. In International Conference on Learning Representations, 2024. URL
16

---

Published in Transactions on Machine Learning Research (04/2026)
A Theoretical Insights
A.1 Finite Difference Barrier Condition
Lemma 1 (Finite-Difterecee Approximation the Barrier Condition). Consider control-affine system
á f(x) + g(x))u with a continuously differentiable barrier function X - E. The continuous-time
barrier condition
min max VB(x)T + s(z)u), (xx) B(xr) } =
(14)
admits the finile-difference approccimation
minf max B(x+1)) ((x)) = B(x,),
(15)
up to positive scaling by At.
Proof. Starting from the continuous-lime condition in equation 14 note that (f(x) + )(x)u)= = VB(x) d B(x(t)).
(16)
dt
Using a forward Euler approximation, the time derivative satisfics
dt B(x(t)) ~ B(x+11) - B(x,)
(17)
At
where, x₁ =x((,, us = u(t) and 2441
x(t At). Substituting equation 17 into equation 14 yields
B(x++1) B(xx) {(xx)) B(xx) } 0.
min
max at
(18)
At
Let B(I++1)= B(x,) and b ((x+) B(xx). Since At 0, the function
m(a, b) = min a/At, b}
satisfies m(a, b) = and only if min(a, (an b} = 0. This follows from the fact that scaling one argument of the minimum by a positive constant does not change which argument is smaller, nor the value at which the minimum equals zero. Thus, equation 18 equivalent
min a, b) =0
(19)
which gives the finite-diferrenc barrier condition in equation 15 This establishes the claim.
A.2 Expectile Regression
Expectile regression is a classical tool in statistios and econometrics for est astimating asymmetric statistics of a random variable. For a random variable X, the r-expectile is defined as the minimizer of the asymmetric least-squares problem
my arg min Ex~x [L m)],
(20)
where L-(y) - 11y < 2)
When F > 0.5, this loss assigns more weight to samples above the estimate m, and less weight to samples below Conversely, T < 0.5 emphasizes lower values. Thus, expectiles interpolate smoothly between the mean (T 0.5) and a gigh---alu--ockking statistic as 7 +
Expectile regression also be extended to learning conditional expectiles:
my (x) arg min m(.)
(Da) [L" (L m())],
(21)
which can be optimized efficiently via stochastic gradient descent. This makes expectiles easy to implement in modern machine-learning pipelines, unlike alternative high-order statistics that require specialized solvers.
17

---

Published in Transactions on Machine Learning Research (04/2026)
A.2.1 Why Expectile Regression in V-OCBF?
In V-OCBF, the barrier function ideally requires computing max,'eu B(x',, but performing this maximiza- tion directly is problematic in the offline setting because the dataset supports only restricted action set Up {u (z, x') E D), and querying B(x') for unseen actions u' & Up introduces unsupported targets that can distort or destabilize learning. Moreover, even restricting the maximization to Up leads to brittle update since a strict maximum is extremely sensitive to noise or rare outlier transitions, often producing poor barrier estimates. Expoctile regression provides a smooth and data-supported alternative by empha- sizing higher (safer) barrier values when т > 0.5, effectively approximaling maximal barrier value over Up while never querying out-of-distribution actions, thereby yielding stable and principled mechanism for synthesising offline CBVF.
Expectile regression resolves these issues. For > 0.5, the expectile estimator concentrates on the largest barrier values consistent with the dataset. This yields a smooth approximation to max B(x'), which equivalent to performing a constrained maximization over only the suppset-supported actions, without ever querying B on out-of-distribution actions.
To illustrate the effect of the expectile parameter show barrier functions learned for the AGV case study using {0.5, 0.6, 0.7, 0.8, 0.9, 0.99} (Figure 4 As increases the volume of the sub-zero level set visibly shrinks: lower т values produce larger sub-zero regions because the learning target is more influenced by poorer next-state outcomes present in the data, which inflates the set of states that appear unsafe under the learned barrier. By contrast, larger v up-weights higher (safer) barrier targets within the dataset. and yields a smoother, upper-envelope like approximation to the maximization over suppootee actions; sampling actions from this higher expectile therefore produces smaller sub-zero level set (i.c., tighter unsafe region) and correspondingly larger estimated safe set.
Value maps for Tau=0.5
Value maps for 0.u=0.6
Value maps for Tau=0.7
1.00
1.00
1.00
0.25
0.25
0.00
0.00
0.00
0.4
0.25
0.25
-0.25
0.2
-0.50
0050
0.50
0.3
-0.75
-0.5
-0.75
-1.00
-1.00
-1.00
1.0
-0.5
0.0
0.5
1.0
0.55
0.0
0.5
-1.0
-0.5
0.5
1.0
Value maps for Tau=0.8
Value maps for Tau=0.9
Value maps for Tau=0,99
1.00
1,00
1.00
0.75
0.75
0.75
1.0
V40
0.6
0.00
0.00
0.4
0.25
0.25
0.25
0.2
0.50
0.50
0.50
-0.75
00.7
-0.7
1.00
1.00
1.00
-1.0
-05
0.0
0.5
-1.0
-0.5
0.0
0.5
-1.0
-0.5
0.0
0.5
1.0
Figure 4: 7- >-Expectiee: Figure shows depiction of the learned Barrier Function for different values with Expectile Regression. With increasing value of t (> 0.5), the feasible region increases, indicating a more accurate representation of the true barrier function.
18

---

Published in Transactions on Machine Learning Research (04/2026)
A.3 Feasibility of V-OCBF-QP
Formal safety guarantees of standard QP-based CBF formulations generally hold under the assumption of unbounded actuation. In practice, however, in the presence of control bounds, a nominal CBF may conflict with feasibility requirements, causing the corresponding CBF-QP to become infeasible. To address this within our framework, we adopt standard relaxation technique from the safe control literature by introducing a slack variable 8. This relaxes the hard safety constraint when strictly necessary while heavily penalizing any violations. The resulting slack-augmented V-OCBR-QP is formulated as:
saafe (z) arg min ||u [=)(2=)] + A8 uEU,EE s.t. C,B(z) + B()( (B(x)) + +820,
(22)
where À > 0 is a penalty coefficient.
Empirically, 8 remains close to zero throughout our experiments. Because V-OCBF explicitly accounts for control constraints during learning, the synthesized policy naturally respects admissible bounds. Conse- quently, the safety filter rarely becomes infeasible, and any triggered corrective adjustments require minimal slack usage.
B Description of the Experiments
Figure 5: Illustrations of the test environments: (Top-Left) environment illustrates Autonomous Ground Vehicle with an obstacle and a goal point. (MuJoCo Env) Red sphere denotes unsafe state.
B.1 Autonomous Ground Vehicle Collision Avoidance
We consider a collision-avoidance task for an autonomous ground vehicle modeled using the Dubins dynamics:
i
+ v
0 0
u sin &
U
(23)
0
where the state = [=1. 22, TH E X C R3 encodes the vehicle's position and heading. The forward velocity is fixed v 0.6 units/s, and the admissible state space given by X [-1, x x, T). The control input E [11,1] denotes the angular velocity
19

---

Published in Transactions on Machine Learning Research (04/2026)
The reward function r(x) is defined
C
r(x)
(24)
(I
xi 222
gg g2
+
where 0.1 is the scaling lactor and (iig "g2)" denotes the goal location. Rewards are accumulated only while the agent remains collision-free. The constant is included prevent numerical instability when the vehicle approaches the goal.
Safety is encoded Uhrough the function
((x)
x
Tol 3o2
R
(25)
22
where To2) (0, denotes the center of the obstacle and 0.2 units is the radius of the obstacle.
Offline Data Generation. Since this is custom environment, construct offiine dataset for training both the barrier function and the safe policy. We sample 1500 initial states uniformly from X' and simulate each trajectory for 500 discrete timesteps with step size At = 0.01s. During data collection, control inputs are drawn uniformly at random from the admissible range, ensuring diverse system trajectories for learning- based safety certification.
B.1.1 Safe Set Volume Analysis
Backward Reachable Tube is the set of initial states for which the agent acting optimally, will eventually reach the larget set T within the time horizon [0, 4]:
SSV fa Bu(+), 3t [0. T) (t) TT
(26)
where (t)) represents system state at time t [0, and u defined over time horizon [0, represents sequence of control actions over the time horizon.
Y position
B=0
0=0
0480
position
Figure 6: Safe Set Volume: Figure shows a depiction of feasible region (i.c., anything outside B-0 enclosure) for V-OCBF for 3 different heading angles. Note that calculation of the actual Safe Set volume is a cumulative of feasible region over countless such frames along the continuous heading angle (ø) dimension.
We performed Safe Set. Volume (SSV) analysis to inspect the effectiveness and conservalism of various approaches. Specifically, we aim to evaluate the volume of the state space from which the agent can safely initialize. Effectively, the larger this Safe Set volume is, bigger the feasible region gets, hence, making it easy navigate in the environment, and smaller the Safe Set volume becomes, lesser the feasible region gets, thus, overall making it less safe for the agent to navigate in the environment..
20

---

Published in Transactions on Machine Learning Research (04/2026)
To evaluate the safe initialization region that each of the method casts, uniformly sample different initial states across the state space and check whether they belong to the above-defined Safe Set Volume by rolling out trajectories in the online environment for over 100k times (as we started to observe convergence in the Safe Set Volume after the 100k mark). We investigate the results for this analysis the main paper (refer Table D where we show that V-OCBF enables the largest safe feasible region among all the various approaches.
B.2 Safety MuJoCo Environments:
To evaluate our framework on nigher-dimensional systems, we use MuJoCo Safety Gymnasium environments Ji al. (2023). where safe velocity tasks introduce velocity constraints for Gymnasium's 's MuJoCo-v4 v4 agents.
The agent receives scalar cost signal on the basis of its velocity at each step:
cost = bool Veurrent > Vihreshold)
(27)
This constraint can be framed as safety function
((x) = Vihreshold Veurrent
(28)
since {(x)) 0 corresponds to safe state. This safety value function provides us continuous signal instead of sparse values of 0 For agent specific threshold velocities and time-step At, we refer the official documentation, the official values for which have been compiled and included in Table 3
Table 3: Threshold velocity Vihreshold; time step size At and act ion space U for each MuJoCo based Safety
Gymnasium Ji et al. (2023 environment as per official documentation.
Environment Hopper
Half-Cheetal
Ant
Walker2D
Swimmer
Vthreshold
0.7402
3.2096
2.6222
2.3415
0.2282
At
0.008
0.05
0.05
0.008
0.04
1.0, 1.0]
[- 1.0, 1.0)
[ 1.0, 1.0]
1.0, 1.0
[- 1.0, 1.0]
B.2.1 Qualitative Analysis
In contimuation the Hopper plot in the main paper (Fig. 3
we include plots for all the remaining MuJoCo
safely gymnasium environments covered in this paper with the visualization of the episode rollouts, both with and without V-OCBF. Given the safety rates of V-OCBF compared to the other baselines (refer Fig. 2 it becomes important to visually realize the real-time effect on performance (Figures
Without CBF
With V-OCBF
Repset Trreshold
200
Terectop
Mean Velocity
Median Velocity
5 95
1QR (25 75 %%)
Rollouts Threshold
Threshold 2.3415
Figure 7: Walker-2D. Instantaneous velocity (left. Y-axis) across episodes. Right Y-axis depicts percentage of episodes when the agent violates the velocity threshold at particular timestep.
21

---

Published in Transactions on Machine Learning Research (04/2026)
Without CBE
With V-OCBF
Rolouds linashlld (50
2.4
2.
300
200
300
400
:20
200
300
400
Mean Velocity
Median Velocity
5 95 %
IQR (25 75 Rollouts Threshold
Threshold 3.2008
Figure 8: Half-Cheetah. Instantancous velocity (left Y-axis) across episodes. Right Y-axis depicts per- centage of episodes when the agent violates the velocity threshold at a particular timestep.
Without CBF
With V-OCBF
21
Timmttp
Timeebe
Velocity
Velocity
Figure Ant. Instantaneous velocity (left Y-axis) across episodes. Right Y-axis depicts percentage of episodes when the agent violates the velocity threshold at a particular timestep.
C Ablation Studies
We have performed set of ablation studies primary testing environment of Autonomous Ground Vehicle Collision Avoidance to further strengthen our claims of the paper. AGV acts an ideal testing bed for such ablations as it is easy to visualize and infer from the results in 2D space. Besides, the environ- ment properly encapsulates our requirements of providing a safety critical control environment. Following subsections cover the ablations we have performed.
C.1 Robustness size of the NN
In this section we show our framework's robustness against varying model sizes, thus, demonstrating the versatility different. hyperparameler choices. Table shows the safety rate for different model dimensions.
Table 4: Safety Rate (%) reported for varions model dimensions. Columns represent Width: neurons each hidden layer, and Rows represent number of layers.
Width
256 Neurons
96 Neurons
64 Neurons
48 Neurons
Layers
98.28 + 0.54
97.18 0.41
96.96 1.21
3
98.04 + 0.30
97.36 + 0.22
97.12 + 0.23
96.54 + 1.62
96.82 1.24
22

---

Published in Transactions on Machine Learning Research (04/2026)
As can be seen from the Table 4 the models don't affect the safety performance significantly, until drop the size by too much (as in the cases of (48,3), (48,4) and (64,2)). We consider the subtle drop is safety rate for the lower dimensional networks to be within considerable limits.
C.2 Robustness to errors in Learned Dynamics
To evaluate the robustness of our pipeline to errors in the learned dynamics, we measure how additive per- turbations affect the safety performance of the CBF-based controllers. Concretely, we perturb the computed safe action w by adding zero-mean Gaussian noise NN(0, with a range of standard deviations a (refer Table 5 For every noise level we run 500 independent trials (with 5 different random seeds each) and report the safety rate defined as the fraction of trials that remain within the safe set over the evaluation horizon. Table summarizes the results across several CBF variants, for each method we report mean safety rate + standard error. Across noise levels spanning small to large perturbations, the safety rates degrade only mildly, indicating that the learned CBFs tolerate moderate inaccuracies in the dynamics control sig- nal. These results suggest that, while precise dynamiis models improve performance, our approach provides substantially safely-aware control even when the dynamics model is imperfect.
Table 5: Safety Rate (%) reported for Robustness Testing of learned Dynamics for Autonomous Ground Vehicle by adding upto 5%, 10% and 20% of the possible control action as noise.
+ Small Noise (5%)
+ Medium Noise (10%)
Large Noise (20%)
Method
No Noise
BCBN Robey al. 2020
92.48 + 0.60
92.48 + 0.62
92.52 0.41
92.32 1 0.56
BC+iDBF Castaneda el al. 2023
92.87 + 0.73
92.84 + 0.64
92.40 0.84
92.76 +: 0.28
BC+C Tabbara Sibai (2025
93.56 0.56
93.72 + 0.58
93.68
0.80
93.76 + 0.38
BC+V-OCBF (Ours)
98.28 + 0.54
98.08 + 0.61
97.84 + 0.68
97.94 + 0.48
This empirical test shows that even we haven't learned the most accurate dynamics model, we can expect a considerably precise safety-aware control for the system with our approach.
C.3 Sensitivity to Expectile parameter (T)
we use a r-expectile loss to obtain the final CBVF. When T = 0.5, the objective reduces to a symmetric MSE fit, causing the learned barrier to reflect the average safety value induced by the behavior policy. This reproduces the aehavioorinnducee barrier and typically yields overly conservative safe sets that do not generalize well beyond the demonstrated trajectories.
As discussed in Section 4.2
To investigate this effect, conduct a study by comparing т 0.5 against higher expectile levels that emphasize the upper tail of the safety value distribution. Table summarises the safety rates achieved on the AGV environment for different т values.
Table 6: Safety Rate (%) reported for Autonomous Ground Vehicle by varying r in the expectile regression.
0.5 (Behavior Policy)
0.7
0.8
0.9
0.99
Safety Rate (%)
96.53 + 0.23
96.87
0.54
97.16 + 1.01 97.72 +: 0.88
98.28
0.54
The results reveal two key patterns. First, the behavior-indueed barrier (r = 0.5) yiclds the lowest safety rale, consistent with its tendency to underrepresent safety-crilical regions that are not frequently explored by the behavior policy. Second, as т increases, the barrier becomes progressively more conservative in safety-critical areas and better aligned with the upper end of the demonstrated safety values, resulting consistently higher safety rates. Figure depicts this nature of the learned barrier functions on varying the value of т.
23

---

Published in Transactions on Machine Learning Research (04/2026)
Under idealized conditions (deterministic dynamics and no dataset disturbances ,ie.., when the best actions reliably lead to the best next states); pushing the expectile + 1 concentrates the estimator the upper tail of the target distribution and, under favorable (nearly deterministic) conditions, can yield the strong empirical performance, prioritizing high-value next-state targets. Howover, this extreme sctting is brittle to rare poor actions or stochastic disturbances that produce optimistic next stales; such cases an estimalor that strictly targets the extreme upper tail may overfit. those spurious outcomes.
To empirically validate this reasoning, conducted an additional set of experiments evaluating the impact of across multiple Safety MuJoCo tasks using standard, public datasets from DSRL, which are not strictly deterministic. We evaluated both safety (measured via Cost) and task performance (measured via Reward) for E (0.5, 0.7,0.7,..0 0.9, 0.99), as shown in Table 7
Table 7: Reward and Cost across different 7 expectile values for selected MuJoCo environments
0.5
T 0.7
0.9
0.99
Environment
Reward
Cost.
Reward
Cost
Reward
Cost
Reward
Cost
Ant
565.36
0.0
639.61
0.0
791.77
0.0
734.66
0.02
Half Cheetah
100.39
0.0
180.87
0.0
679.73
0.0
582.58
0.05
Hopper 96.94
0.0
126.54
0.0
769.69
0.0
526.72
0.27
As observed in Table 7 increasing toward 1 (specifically т 0.99) results in degraded safety performance, reflected by increased Cost, and in some cases reduced Reward. This trend is consistent across the reported tasks and aligns with our theoretical intuition regarding sensitivity to stochastic transitions. In contrast, I = 0.9 achieves more favorable balance between reward maximization and safety prescrvation. Similar palterns were observed across other environments. Based on these findings, we selected = 0.9 for the experiments reported in the main paper.
C.4 Sensitivity to Dynamics Model Accuracy
To empirically evaluate the resilience of our framework against approximation errors the learned dynamics, conducted analysis using the Autonomous Ground Vehicle environment, to quantify how variations in model fidelity influence safety performance. We evaluated four distinet configurations representing varying levels of accuracy: the known system dynamics, the standard learned model used in our main experiments, and two degraded models trained for only 50% and 25% of the standard steps, respectively. We quantified the model quality by computing the mean squared prediction error over a set of randomly sampled state-action pairs, which confirmed a monotonic reduction in predictive accuracy across the models.
Table 8: Effect of Dynamics Model Accuracy on Safety Rates at the time of Inference (Mean over 500 episode rollouts).
Method
Safe Episodes (%)
Episode Reward
Mean L2 Error
BCOV- (Known Dynamics)
99.52 0.58
55.91 + 0.39
BC+V-OCBF (Learned Dynamics)
98.28 + 0.54
54.93 + 0.46
1.48 10-2
BC+V-OCBF (Half-Learned Dynamics)
97.03 + 0.45
52.44 = 0.18
2.93 x 10
BC+V-OCBF (Quarter-Leanne Dynamies)
96.32 0.73
51.91 + 0.32
4.46 10
Subsequenly, we utilized each model to senthesize the controller and evaluated the resulting empirical safely rates. As detailed in Table | the results indicate that the safety rate experiences only modest reduction despite the increase in model prediction error. These findings suggest that V-OCBF maintains its effective- ness in mitigating safety violations even when the underlying dynamics model is imperfect, supporting the practical applicability of the method in settings where exact system identification is challenging.
24

---

Published in Transactions on Machine Learning Research (04/2026)
D Additional Experiments on Boat Navigation
To further assess the generalization capabilities of V-OCBF, we evaluated the method on a boat navigation task characterized by nonlinear river drift and static obstacles. This environment, while low-dimensional, presents a control challenge due to the drift dynamics which can actively push the agent into unsafe regions if not properly anticipated. Such drift dynamics are particularly difficult for offline safe RL and CBF-basod methods to handle when relying on learned models or indirect constraint formulations.
The system dynamics given by:
(21)141= = (21); : + (a1 + - 0.5zz) At
(x2)++1 (22), + 0g At
where At represents discrete time step, 0,, 42 represent the velocity control action in and directions respectively, subject to the actuation limit aj + aз s 1. The term 2 0.5z3 introduces a state-dependent drift along the 01 -axis.. The safety constraints are encoded as:
((x) min (0mm|ll-0005,05)))) (-0.5,0.5) 0.4, | (-1.0, 1.2)
0.4)
(29)
where {(x) < indicates that the boat inside a obstacle, thereby ensuring that the sub-level set of ((x)
defines the failure region.
Table 9: Boat Navigation Experiment: Percentage Safe Episodes against additional baseline methods. Eval-
uated over 500 episodes and seed values.
Method
Safe Episodes (%)
BC+NCBF (Robey et
2020
88.43 + 0.59
CPQ Xu et al. 2022
54.14 + 0.26
WSAC Wei et al 2024
79.34 + 0.87
CDT Liu al. 2023
69.94 + 0.42
C2TQL (LIU et al. 2025
70.77 + 0.76
BEAR-Lag KKumar
2019
71.69 + 0.16
COptiDICE Lee et al.
2022
87.88 + 0.65
FISOR (Zheng et 2024
85.78 0.22
BC+ V-OBB (Ours)
97.56 1 0.13
We compared V-OCBE against a broader set baselines, including CPQ (Xu et al. 2022), WSAC Wei et al.] 2024), CDT [Luu et al., 2023), and C2IQL (L.UU et al. 2025). addition to the methods evaluated the main text. The results, sunmarized in Table demonstrate that V-OCBF consistently achieves higher empirical safety rate compared to the baseline methods. Notably, the strongest baseline exhibits safety rate approximately 9% lower than approach. These results, combined with the experiments presented in Scction 5 indicate that V-OCBF provides a robust mechanism for minimizing safety violations across diverse dynamical systems, effectively handling the challenges posed by nonlinear drift and learned dynamics.
E
Experimental Details
E.1
Experimental Hardware
To keep the evaluation fair and avoid any discriminatory added advantage to any specific experiment, all experiments were conductod on a single system equipped with an 14th Gen Intel Core 19-14900KS CPU, 128GB RAM, and an NVIDIA GeForce RTX 5090 GPU for training and experiment evaluations.
E.2 Hyperparameters for the Proposed Algorithm
We have compiled and listed down all the hyperparameters that we used to perform our experiments and report the results. These training settings for all the environments detailed in the Table TO For the
25

---

Published in Transactions on Machine Learning Research (04/2026)
Table 10: Hyperparameters for the Proposed (V-OCBF) Network, and Learned Dynamics Model.
Hyperparameter
Value
Network Architecture
Multi-Layer Perceptron (MLP)
Activation Function
ReLU Adam optimizer
Optimizer
Learning Rate
x 10 0.99
Discount Factor (7)
Autonomous Ground Vehicle Collision Avoidance
Number of Hidden Layers
2
Hidden Layer Size
256 neurons per layer 75K
Dataset Size
Dynamics Model Hidden Layers
3
Dynamics Model Hidden Layer Size
64
Safe Velocity Hopper
Number of Hidden Layers
Hidden Layer Size
256 neurons per layer
Dataset Size
1.32M
Dynamics Model Hidden Layers
4
Dynamics Model Hidden Layer Size
64
Safe Velocity Chef-Cheetah
Number of Hidden Layers
3
Hidden Layer Size
128 neurons per layer
Dalaset Size
249K
Dynamics Model Hidden Layers
4
Dynamics Model Hidden Layer Size
64
Safe Velocity Ant
Number of Hidden Layers
Hidden Layer Size
256 neurons per layer
Dalaset Size
2.09M
Dynamics Model Hidden Layers
4
Dynamics Model Hidden Layer Size
64
Safe Velocity Walker2D
Number of Hidden Layers
3
Hidden Layer Size
256 neurons per layer
Dataset Size
2.12M
Dynamics Model Hidden Layers
Dynamics Model Hidden Layer Size
64
Safe Velocity Swimmer
Number of Hidden Layers
Hidden Layer Size
256 neurons per layer
Dataset Size
1.68M
Dynamics Model Hidden Layers
4
Dynamics Model Hidden Layer Size
64
MuJoCo environments, we use the widely accepted DSRL, Liu et al. (2024) dataset. We also use the Class-K function k x B(-) where a 1.
E.3 Hyperparameters for the Baselines
For all the CBE-based baselines (NCBF, iDBF, CCBF), we use the same network architectures and learned dynamics models described above to ensure a fair comparison. Hyperparameters for the safe offline RL basc- lines (COptiDICE, BEAR-Lag, BC, FISOR) are provided in Table 11 We use the official implementations of CPQ, CDT, BEAR-Lag and COptiDICE from (OSRL) Liu et al. (2024) WSAC from Wei et al. 2024
26

---

Published in Transactions on Machine Learning Research (04/2026)
C2IQL from LIU al. (0255) and FISOR from Zheng et al. (0024) and train all methods the same DSRL datasets.
27

---

Published in Transactions on Machine Learning Research (04/2026)
Table 11: Hyperparameters for all the Baselines network. For COptiDICE and BEAR-Lag refer OSRL, Liu ct al.
2024). for the official implementations. For FISOR refer Zheng al. (2024) for official implementation.
Autonomous Ground Vehicle (AGV)
BC Actor
MLPActor, (128,2)
BEAR-Lag Actor
SquasleeCCaausssaMMMPPPPCCC (128,2)
BEAR-Lag- Critic
FnsemblleDonblleCChttt, (256,)
BEAR-Lag Cost critic
MT.P, (256,2)
COptiDICE Actor (extraction)
MLPActor, (128,2)
COptiDICE - Dual / network
DualNet (value-like) like), (256,2)
FISOR Actor (diffusion denoiser backbone)
DiffusionDDenoiserMLL. (128,2)
FISOR Feasibility classifier
TeasibiliyClassifierr (256,2)
Safe Velocity Hopper
BC Actor
MI.PActor, (256,2)
BEAR-Tag Actor
SquasheiCCannssianMMM.PPP. (256,2)
BEAR-Lag Critic
EnsembbleDouubleQCCittt,,, (256,22
BEAR- Cost critic
MLP, (256,2)
COptiDICE Actor (extraction)
MLPActor, (256.2)
COptiDICE - Dual / network
DualNet (value-like), (256,2)
FISOR Actor (diffusion denoiser backbone)
DiffusionDenoiserMMIPPP (256,2)
FISOR Feasibility classifier
FeasibilityClassifeer, (256,2)
Safe Velocity Half- Cheetah
BC Actor
MLPActor, (256,2)
BEAR-Lag Actor
MunnsheeCCauussiamMMPAAAAe (256,2)
BEAR-Lag Critic
EnusembleeDoubbeeCCCittt,, (256,2)
BEAR-Lag Cost critic
MLP, (256,2)
COptiDICE Actor (extraction)
MLPActor, (256,2)
COptiDICE Dual / network
DualNet (value-like), (256,2)
FISOR Actor (diffusion denoiser backbone)
DiffusionDennosseMMLPPP (256.2)
FISOR Feasibility classifier
CussibilitClassiierr, (256.2)
Safe Velocity Ant
BC Actor
MLPAAtor, (256,2)
BEAR-Lag Actor
Squashed(GaussianMLLAAAAe (256,2)
BEAR-Lag Critic
EnsembleDoubbeeQChitti,, (256,2)
BEAR-Lag Cost critic
MLP, (256.2)
COptiDICE Actor (extraction)
MLPActor, (256,2)
COptiDICE Dual network
DualNet (value-like), (256.2)
FISOR- Actor (diffusion denoiser backbone)
DiffusionneeniserrMMM ML.P, (256,2)
FISOR Feasibility classifier
PeasiblittClhsssfferr, (256,2)
Safe Velocity Walker2D
BC Actor
MLPActor, (256,2)
BEAR-Lag Actor
CuasseeddCausssiauMMMMP..... (256,2)
BEAR-Lag Critic
EuseubblaDoubbleCCCiiicc (256,2)
BEAR-Lag Cost critic
ML., (256,2)
COptiDICF: Actor (extraction)
MI.PActor, (256,2)
COptiDICE- Dual / network
DualNet (value- like), (256,2)
FISOR Actor (diffusion denoiser backbone)
DiffusionnDenoiserM.L... (256,2)
FISOR Feasibility classifier
FeassbilityClussifeerr (256,2)
Safe Velocity Swimmer
BC Actor
MLPActor, (256,2)
BEAR-Lag Actor
SqusshedCCaussianM....PP (256,2)
BEAR-Lag Critic
EnsembbleDoubbleeCCitii,,, (256,22 (256,2)
BEAR-Lag Cost critic
MLP, (256,2)
COptiDICE - Actor (extraction)
MLPActor, (256.2)
COptiDICE Dual network
DualNet (value-like), (256,2)
FISOR Actor (diffusion denoiser backbone)
DiffusionDenoiirrM.... (256,2)
FISOR Feasibility classifier
PeasibilityClassif (256,2)
28


Reviews:
[{"review_id": "hQ89ic6bwQ", "reviewer": "Reviewer_P4mu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes V-OCBF, a framework to learn a neural Control Barrier Function (CBF) / Control-Barrier Value Function (CBVF) from offline demonstrations without assuming access to true system dynamics. The method derives a model-free finite-difference recursion for the barrier (a reachability-style Bellman backup) and trains a neural barrier using a discounted regression objective to avoid trivial fixed-point solutions. To mitigate the offline “out-of-distribution action” issue, the paper adopts an expectile regression objective (inspired by IQL) to bias the learned barrier toward high (safe) values supported by the dataset’s action support, without querying barrier targets on unseen actions. At deployment, the learned barrier is used in a CBF-QP safety filter; since Lie derivatives are needed, a separate learned control-affine dynamics model is trained from the dataset and used only at inference to compute the QP terms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- Clear problem motivation: the paper targets strict, state-wise safety (forward invariance) rather than expected-cost constraints, and focuses on offline settings where online exploration is infeasible.\n- Methodological clarity: the finite-difference barrier recursion and the two-stage learning procedure (behavior-induced backup then value-guided/expectile shaping) are explained cleanly, with algorithms and a workflow diagram.\n- Addresses a key offline pitfall: the expectile objective is a reasonable, practical mechanism to avoid maximizing over actions not supported by the dataset, while still pushing beyond a purely behavior-induced barrier.\n- Empirical results are fairly extensive for the offline-safety setting: evaluations include a low-dimensional collision-avoidance task (AGV Dubins car) and multiple high-dimensional Safety Gymnasium MuJoCo tasks. Reported metrics include safety violations, reward, and a “safe set volume” proxy.\n- The paper includes ablations: effects of using learned dynamics in training vs inference, robustness to barrier network size, robustness to perturbations/noise, and sensitivity to expectile level tau.", "audience_yn": "Yes", "audience_explanation": "Learned CBF is an important topic in safe control.", "requested_changes": "1) Detailed questions for the authors\n\nQ1. Guarantee statement: What precise property is claimed when the learned B_theta only approximately satisfies the finite-difference recursion (due to function approximation and sampling)? Do you measure recursion residuals, and do they correlate with violations?\n\nQ2. Action-support construction: How do you implement U_D(x) in continuous spaces? Is it kNN over states, or do you use all actions in a minibatch? Please provide exact details and complexity.\n\nQ3. QP feasibility: When the QP is infeasible (due to control limits or model error), what is the fallback? Do you soften constraints with slack variables, and if so, how is safety assessed?\n\nQ4. Learned dynamics: How large are the dynamics model errors (one-step prediction error) in each environment, and is there a relationship to safety violations? Do you use ensembles or uncertainty estimation?\n\nQ5. Expectile tau: The ablation suggests tau near 1 gives best safety in AGV. Does very large tau ever lead to overly conservative behavior or degraded reward on MuJoCo tasks? A per-task tau sensitivity plot would be useful.\n\n2) Suggestions for improvement\n- Clarify and narrow the formal claim: explicitly state whether the guarantee is discrete-time one-step invariance, and under what assumptions it extends to multi-step invariance.\n- Add a short section on deployment-time robustness to model error (even if only empirical): e.g., dynamics ensembles, uncertainty-aware margins in the QP constraint, or a discussion of robust CBF-QP variants.\n- Provide concrete implementation details for approximating dataset action support U_D(x) and a small sensitivity check (k in kNN, bandwidth, etc.).\n- Extend evaluation with at least one additional constraint type beyond velocity thresholds, or a clear discussion of how the approach scales to multi-constraint / geometry-based safety.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hAIcxJv6iV", "reviewer": "Reviewer_4BQn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**: This paper considers learning a safety filter from offline data for a value-guided control barrier function. Traditional methods use the known dynamics, or grid search, to find the safety filter; in this paper, they use offline data and the value function to learn the action. In particular, they use the offline data to learn the V-OCBF, and also learns a dynamic model, so that during the inference time, a safe action can be taken. The paper then provides empirical validation. \n\n**Strength**: \n\n1. The problem is timely, as using offline data would be of great importance, and the approach is practical as it is grounded in previous works. \n2. The approach has shown improvement over the others. \n\n**Weakness**: \n\n1. The main weakness is the lack of technical novelty. The paper itself has pointed out that learning the value-based control barrier function is inspired by the Fisac et al.  There are plenty of approaches for learning the control barrier function [A1--A4]. The only contribution is to use the offline data. However, the techniques proposed in this paper are not new, they have already been proposed. The adaptation is new, though for learning the control barrier function. However, it is not clear whether that would be a valid technical contribution. \n\n2. The paper uses a safety filter-based approach; however, there are plenty of papers on using a safe offline RL-based approach. The paper has not compared with those works. The reviewer is aware of the fact that they are two different tools, but they are solving the same problem. For example, one can put the cost in the unsafe region to be 1, and in the safe region to be 0. One can pose this problem as a constrained MDP problem. There are existing offline approaches [A5--A9] to solve this problem effectively. \n\n3. Even though the empirical results show improvement, the improvement is mild over the existing ones. Can the authors identify one experiment where the existing approaches' results are poor, and the approach proposed by the authors show a significant improvement?\n\n[A1]. Robey, Alexander, Haimin Hu, Lars Lindemann, Hanwen Zhang, Dimos V. Dimarogonas, Stephen Tu, and Nikolai Matni. \"Learning control barrier functions from expert demonstrations.\" In 2020 59th IEEE Conference on Decision and Control (CDC), pp. 3717-3724. Ieee, 2020.\n\n[A2]. Taylor, Andrew, Andrew Singletary, Yisong Yue, and Aaron Ames. \"Learning for safety-critical control with control barrier functions.\" In Learning for dynamics and control, pp. 708-717. PMLR, 2020.\n\n[A3]. Emam, Yousef, Gennaro Notomista, Paul Glotfelter, Zsolt Kira, and Magnus Egerstedt. \"Safe reinforcement learning using robust control barrier functions.\" IEEE Robotics and Automation Letters (2022).\n\n[A4]. Wabersich, Kim P., and Melanie N. Zeilinger. \"Predictive control barrier functions: Enhanced safety mechanisms for learning-based control.\" IEEE Transactions on Automatic Control 68, no. 5 (2022): 2638-2651.\n\n[A5]. Xu, H., Zhan, X., and Zhu, X. Constraints penalized q-learning for safe offline reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8):8753–8760, Jun. 2022. doi: 10.1609/aaai.v36i8.20855.\n\n[A6]. Wei, H., Peng, X., Ghosh, A., and Liu, X. Adversarially trained weighted actor-critic for safe offline reinforcement\nlearning. Advances in Neural Information Processing Systems, 37:52806–52835, 2024.\n\n[A7]. Liu, Z., Guo, Z., Yao, Y., Cen, Z., Yu, W., Zhang, T., and Zhao, D. Constrained decision transformer for offline safe reinforcement learning. In International Conference on Machine Learning, pp. 21611–21630.\n\n[A8]. Wang, Y., Zhan, S.S., Jiao, R., Wang, Z., Jin, W., Yang, Z., Wang, Z., Huang, C. and Zhu, Q., 2023, July. Enforcing hard constraints with soft barriers: Safe reinforcement learning in unknown stochastic environments. In International Conference on Machine Learning (pp. 36593-36604). PMLR.\n\n[A9]. Zifan, L. I. U., Xinran Li, and Jun Zhang. \"C2IQL: Constraint-Conditioned Implicit Q-learning for Safe Offline Reinforcement Learning.\" In Forty-second International Conference on Machine Learning.", "claims_and_evidence_yn": "No", "claims_explanation": "Even though using an offline RL technique to learn the control barrier function might be new, the tools are not new, which limits the contributions. Further, the approach has not compared with the safe offline RL approaches. Hence, the paper needs a significant revision.", "audience_yn": "Yes", "audience_explanation": "The paper is useful for the robotics, and the control community.", "requested_changes": "1. Can you please identify the main technical novelties and difficulties in extending the existing offline approaches to learn the value-based control barrier function? \n\n2. Can the authors compare with the existing safe offline RL approaches, as the reviewer pointed out?\n\n3. Can the authors identify one experiment where the existing approaches' results are poor, and the approach proposed by the authors show a significant improvement?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qEtOm8rv7c", "reviewer": "Reviewer_FH4j", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Value-Guided Offline Control Barrier Functions (V-OCBF), a framework designed to synthesize formally safe controllers for autonomous systems using only static and offline datasets. Its primary contributions include model-free barrier synthesis, involving the derivation of a recursive finite-difference barrier update that allows learning a neural Control Barrier Function (CBF) without requiring prior knowledge of the system's analytical dynamics. Additionally, the framework implements an expectile-based learning objective using an expectile regression loss to approximate optimal safe-set expansion by focusing on the \"upper envelope\" of safe actions found in the data. This objective prevents the barrier from being distorted by out-of-distribution (OOD) actions while still improving upon the safety of the original behavior policy. The authors also propose decoupled dynamics learning, a training pipeline where a surrogate dynamics model is learned separately from the barrier function. These dynamics are utilized only during inference to compute Lie derivatives for a Quadratic Program (QP) safety filter to synthesize real-time safe control. Finally, the paper provides empirical validation demonstrating superior performance over existing safe offline RL methods and neural CBF baselines.\n\n\n### Strengths\n- Hard Safety Guarantees: Unlike many offline RL methods that treat safety as a \"soft\" penalty (minimizing expected cost), V-OCBF enforces hard, state-wise constraints through a QP formulation, providing formal one-step forward invariance.\n- Practical Implementation: It functions as a \"minimally invasive\" safety filter, meaning it can be wrapped around any reference controller (like Behavior Cloning) to provide safety without requiring the reference controller itself to be safe.\n\n### Weaknesses\n- Inference-Time Model Reliance: While the barrier is learned model-free, the real-time safety filter (the QP) depends on a learned surrogate dynamics model. Significant inaccuracies in this learned model at inference time could lead to safety violations.\n- Point-Wise Safety: The paper specifically proves one-step forward invariance, but managing safety over long horizons in highly stochastic environments may require further extensions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical claim that the model-free finite-difference recursion preserves forward invariance is supported by a formal derivation and proof in the appendix. Convincing empirical evidence is provided through high-dimensional experiments in the MuJoCo Safety Gymnasium, where V-OCBF consistently achieves the lowest safety violation rates while maintaining competitive rewards compared to several baseline methods.", "audience_yn": "Yes", "audience_explanation": "TMLR's audience of researchers in safe reinforcement learning and control theory would find these findings highly relevant. The paper addresses a critical gap by providing hard, state-wise safety guarantees in offline settings where dynamics are unknown, which is a major challenge for autonomous systems like robotics and vehicles.", "requested_changes": "- Clarify Dependence on Learned Dynamics: The authors state that V-OCBF is \"model-free\" during barrier learning , yet a learned surrogate dynamics model is strictly required at inference to solve the Quadratic Program (QP). The authors must explicitly discuss the safety implications if the learned dynamics is not accurate enough.\n - V-OCBF vs. Model-Based Offline RL: Could you identify the difference between model-based offline RL and your proposed method? Also what is the pros and cons comparing between them?\n- Figure 3 Analysis and Theory-Experiment Gap: Could you elaborate more on figure 3 about why there are more episode safety violations in Walker 2D compared with other environments in your proposed method? Also, since you claim that your method is hard safety constraints, then what is the gap between theory and experiment that there are still violations in practice?", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wuhFflr79O", "reviewer": "Reviewer_RsGZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Value-Guided Offline Control Barrier Functions (V-OCBF), a framework for synthesizing safety-certified controllers purely from offline demonstrations. The key idea is to learn a neural Control Barrier Value Function (CBVF) without access to system dynamics by introducing a finite-difference barrier recursion that propagates safety information across time using offline transitions. To avoid querying unsupported actions in the offline setting, the method incorporates an expectile-based objective that emphasizes high, data-supported safety values while remaining within the dataset action support. The learned barrier is then used in a standard CBF-QP formulation, where a separately learned dynamics model is employed only at inference time to compute Lie derivatives. Experiments on an autonomous ground vehicle task and high-dimensional Safety Gymnasium environments show that V-OCBF achieves fewer safety violations than constrained offline RL and prior offline neural CBF methods, while maintaining competitive reward.\n\n**Strengths**\n\n__1. Clear separation of learning and control__\nThe method clearly separates barrier learning from controller synthesis. The barrier is learned purely from offline data without using system dynamics, while learned dynamics are used only at inference time for QP-based safety filtering.\n\n__2. Dynamics-free offline barrier learning__\nThe finite-difference barrier recursion enables learning a safety certificate without known dynamics or online interaction, making the approach suitable for offline and safety-critical settings.\n\n__3. Strong empirical results across environments__\nExperiments on both AGV and high-dimensional Safety Gymnasium tasks show consistently fewer safety violations than constrained offline RL and offline neural CBF baselines, with competitive reward.\n\n__Weaknesses__\n\n__1. Limited theoretical guarantees__\nThe formal guarantee is restricted to one-step forward invariance under the idealized barrier recursion, and long-horizon safety under function approximation is not theoretically established.\n\n__2. Dependence on learned dynamics at inference__\nAlthough dynamics are not used during barrier learning, the final safety filter relies on a learned dynamics model, and safety under model error is supported empirically rather than theoretically.\n\n__3. Heuristic nature of expectile maximization__\nExpectile regression is well-motivated but does not provide a formal guarantee of approximating the true maximization over admissible actions, and introduces an additional hyperparameter.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims of the paper are supported by a combination of theoretical analysis and empirical evaluation. The finite-difference barrier recursion is formally derived from the CBVF formulation, and the paper proves that satisfying this recursion guarantees one-step forward invariance for control-affine systems. The motivation for expectile regression is clearly explained and aligns with prior work in offline reinforcement learning for avoiding out-of-distribution action evaluation. The experimental results span both low-dimensional and high-dimensional environments, include comparisons with strong constrained offline RL and offline neural CBF baselines, and are supported by ablation studies that validate key design choices. While the theoretical guarantees are limited in scope, they are consistent with what is claimed in the paper.", "audience_yn": "Yes", "audience_explanation": "The paper addresses an important problem in safe decision making from offline data, which is relevant to researchers in offline reinforcement learning, control barrier functions, and safety-critical control. The focus on hard, state-wise safety guarantees and the separation between offline certificate learning and online control execution make the work particularly relevant to audiences interested in combining learning-based methods with control-theoretic safety mechanisms.", "requested_changes": "1. Clarify more explicitly the scope of the theoretical guarantees in the main text. In particular, emphasize that the formal result applies to one-step forward invariance under the idealized barrier recursion, and that closed-loop safety under function approximation and learned dynamics is not theoretically guaranteed.\n\n2. Strengthen the positioning with respect to model-based offline RL. Since a dynamics model is learned and used at inference time, it would be helpful to clearly state that the learned dynamics are not used for policy optimization or imagined rollouts, but only for local Lie derivative computation in the CBF-QP.\n\n3. Add a brief discussion in the related work section contrasting V-OCBF with model-based offline RL approaches, highlighting differences in objectives and the role of dynamics.\n\n4. Provide a short intuitive explanation of how the expectile parameter affects conservatism in the learned barrier, to improve accessibility for readers less familiar with expectile regression.", "broader_impact_concerns": "The work aims to improve safety in learning-based control systems, which is generally a positive contribution. No major ethical concerns are identified.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)