Safely Learning Controlled Stochastic Dynamics
Authors: Luc Brogat-Motte
Alessandro Rudi
Riccardo Bonalli
Venue: NeurIPS 2025 poster ()
Keywords: Kernel Methods
Dynamical Systems
Exploration
Density Estimation
Learning Theory
Primary area: theory
TL;DR: We propose a method for safely learning controlled stochastic dynamics from trajectories by incrementally expanding an initial safe control set using kernel-based confidence bounds, with theoretical guarantees on both safety and estimation accuracy.

Abstract: We address the problem of safely learning controlled stochastic dynamics from discrete-time trajectory observations, ensuring system trajectories remain within predefined safe regions during both training and deployment. Safety-critical constraints of this kind are crucial in applications such as autonomous robotics, finance, and biomedicine. We introduce a method that ensures safe exploration and efficient estimation of system dynamics by iteratively expanding an initial known safe control set using kernel-based confidence bounds. After training, the learned model enables predictions of the system's dynamics and permits safety verification of any given control. Our approach requires only mild smoothness assumptions and access to an initial safe control set, enabling broad applicability to complex real-world systems. We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics. Experimental evaluations demonstrate the practical effectiveness of our method in terms of safety, estimation accuracy, and computational efficiency.

Safely Learning Controlled Stochastic Dynamics
Lue Brogat-Motte
Laboratoire des Signaux et Systèmes, CNRS, CentraleSupélec Université Paris-Saclay, Gif-sur-Yvetie France Istituto Italiano di Tecnologia, Genoa, Italy luc. brogatmotte@iit.it
Alessandro Rudi
SDA Bocconi, Bocconi University, Milan, Italy alesssndro..o www.sandro...diissddeeeeee.. www.oooccodccoooi....
Riccardo Bonalli
Laboratoire des Signaux et Systèmes, CNRS, CentraleSupélec Université Paris-Saclay, Gif-sur-Yvette France
riccardo.
Abstract
We address the problem of safely learning controlled stochastic dynamics from discrete-time trajectory observations, ensuring system trajectories remain within predefined safe regions during both training and deployment. Safety-critical con- straints of this kind are crucial in applications such as autonomous robotics, finance, and biomedicine. We introduce method that ensures safe exploration and efficient estimation of system dynamics by iteratively expanding an initial known safe con- trol set using kernel-based confidence bounds. After training, the learned model enables predictions of the system's 's dynamics and permits safety verification of any given control. Our approach requires only mild smoothness assumptions and access to an initial safe control set, enabling broad applicability to complex real- world systems. We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics. Experimental evaluations demonstrate the practical effectiveness of our method terms of safety, estimation accuracy, and computational efficiency.
Introduction
We consider the problem of safely learning the dynamics of controlled continuous-time stochastic systems from discrete-time observations of trajectory data. This setting is common in applications such robotics, finance, and healthcare, where system dynamics
only partially known and must be estimated from data. A key challenge in these applications is ensuring safety during both the learning phase and subsequent deployment [ (2). As an example, consider an autonomous robot navigating a partially known and turbulent environment, as illustrated in Figure I While the deterministic part of the dynamics may be approximately modeled using prior knowledge, the stochastic disturbances (represented by the brown region in Figure [ due to wind or sensor noise are often unknown and must be learned. Collecting data through naive exploration can result in unsafe trajectories, potentially causing damage to the system or its environment. A second example arises in financial portfolio management, where the drift component of asset prices may be known from historical data, but market volatility remains uncertain. Safety here may correspond to the requirement
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

5000
LEDONICC Hightn
High region
4.44
Ranatroges
3.80
local
3,30
133
277
177
1/4T
1100
Figure 1: Illustration of complex, smooth dynamical system under deterministic conditions (left) and stochastic conditions with unknown disturbances (right). Shown are 100 simulated trajectories under three different controls. Ignoring stochastic disturbances (e.g., wind turbulence) can lead to unsafe trajectories (right), emphasizing the necessity of safe estimation methods that explicitly account for uncertainty.
that the portfolio value stays above a critical threshold with high probability, simulating portfolio loss aversion in risk-sensitive financial decision-making. These examples highlight common need: learning stochastic dynamics of system from data, while ensuring safety throughout the process. This requires ensuring that all executed trajectories remain within predefined safe region with high probability. In addition, at deployment time, the learned model should enable prediction of whether proposed control input satisfies the safety requirements, including those not encountered during training 4.
Outline of contributions.
The contributions this work are follows.
Safe learning method. We derive a method that safely learns controlled stochastic dy- namics, where safety is defined as the requirement that system trajectories remain within a designated set of safe states with high probability. Our approach incrementally expands the known safe control set by selecting novel controls to evaluate, collecting corresponding trajectory data, and refining three models: dynamics model for predicting state evolution, safety model that estimates the probability of remaining within the safe region, and a reset model that captures the probability of returning the system to its initial state distribution, enabling repeated safe exploration under uncertainty. Alongside these models, refine uncertainty estimates using kernel-basse based confidence bounds. After training, these models enable prediction of dynamics, safety, and reset feasibility for any given control, including those not seen during training.
Provably safe exploration and adaptive estimation rates. prove that the proposed method guarantees safety and derive learning rates for system estimation, with rates that are adaptive the Sobolev regularity the underlying dynamics. Crucially, our approach requires only smooth dynamics (with respect to time, state, and control variables) and an initial non-empty set of known safe controls. These mild assumptions make the method applicable to broad range of complex real-world systems subject to stochastic disturbances, found in diverse areas including robotics, fluid flow control, and chemical reaction control 10]
[5| 6, 7 8
Experimental validation. We empirically demonstrate the performance of the ap- proach terms of safety, estimation accuracy, and computational efficiency. Specif- ically, we evaluate it on benchmark two-dimensional stochastic dynamical sys- tem evolving in bounded, safety-critical environment under stochastie perturbations (see Figure D. An open-source Python implementation is provided (available at gi github. www/wmottteeeeeeeeeeeeeeeoeeoeee wow//motte//yyamiiioomcommm -aafee--aamae..ea

---

2 Background
We formalize the problem of safely learning controlled stochastic dynamical systems as follows.
Controlled SDE.
Let X be dynamical system governed by non-linear, n--dimensional controlled
SDE I 12,
d((()) ((X()),
u(t, ((X))))))
a E H. X(0) Po- (1)
Here, Tiax > 0 is the fixed time horizon, (6, a) are functions mapping IR x IR" to R² x RR*,, (tt) is an n-dimenniioal standard Brownian motion, po an initial probability density over R" and H S [max] R", (") ) is mntee-iimensional set of admissible controls, where F(00, maax] x R", R4) denotes the space of measurable functions from [0, max] x R to Rd.
Example 1 (Second-orde) dynamical system). The system
dx = ((t) dt, (V = utt, x(t)) dt + a(((t) dW()),
captures second-order dynamics under control and state- sete-dependent diffusion a (t)).). woal-world examples include drones navigating turbulent environments (where u represents thrust or steering, and al-) models wind turbulence), fluid-dynamical systems (13), and molecular dynamics (where captures external forces such as optical traps, and al-) reflects spatially varying thermal fluctuations 14 15 (61).
Safety-critica environments. Let R + RR be a function that partitions the state space into safe and unsafe regions. The safe region is given by {x R" g(x) 2 0}, while the unsafe region is given by {x E R g(x) 0}.
Safe control. Let X, denote the solution to Eq. ① under the control E (well-defined by existence and uniqueness; see Sec.3 in 17).. Let ug [0, Tmax] x RR - Rd denote a family of controls parameterized by dimensional parameterization is a mild assumption that is widely prevalent in many real-world
C R where D) is a compact subset of TR. Such finite-
applications, including roboties [18), process control [19], and financial engineering [20]
We define the safety level of the control up at time & E [0, T max] as
s00,t) P (g(X,,, (f)) 2 0).
We define the safety level up time T [0, Tmax] of the control us
8 0 (0,T)
inf rE|O,T]
s00
Learning problem. We formulate the problem of safely learning controlled stochastic dynamical systems as the estimation of the probability density map
p 10,t, x) D x [0, Tmax] x R" + po(t,
where (t, x>) is the density of the state X (t) under control ug, well-defined by standard existence and uniqueness results (see 3 in (17). To estimate this density, we collect a dataset of trajectories
(0k, Xun, (w) ())))E( (E(I....Q). 1e).t(( (I).....M))
where each w denotes an independent Brownian motion sample path driving the stochastic trajectory. and Mr denotes the number of time steps in trajectory k. All controls ua, are required to safe. Specifically, the minimal probability of remaining within the safe region up to the time horizons (Te) K = ( constrained by
00 (O,, 11)21-5 for each k [1,
This problem poses a fundamental challenge due to the coupling between learning and safety: accurately estimating the density Pe requires data, but collecting data must respect safety constraints defined by 80 which themselves depend on the very dynamics encoded in pa.
3

---

2.1 Related work
Safe learning in control systems under uncertainty is a central topic reinforcement learning, with methods built on assumptions such as known dynamics [21] controllability (22 23 24 or recovery
policies (2 26].
Much of the literature focuses on discrete-time or discrete-state systems modeled as Markov Decision Processes (MDPs), where risk-sensitive and safe exploration techniques have been developed (27]
28 26 29]. Among these, Safe Bayesian Optimization (BO) methods stand out for providing some
of the strongest high-confidence safety guarantees during exploration [30) 31 32 33], particularly
in MDP settings with known dynamics or access to safety level evaluations 22 34 In continuous
domains, early work focused on designing control policies that avoid unsafe regions B5 36), while
more recent approaches incorporate offline learning and online adaptation for nonlinear systems B7. Stability- guarantees via Lyapunov theory offer formal certification but often require full dynamics knowledge [18 38) Safe BO has also been adapted to continuous settings 39 40].
under assumptions such as access to dynamics, safety oracles, or specific control-theoretic properties. Joint estimation of both dynamics and safety remains comparatively underexplored, particularly in
continuous-timg settings 41
45 46
48 49
In contrast to much of the literature, we assume no prior model of the system dynamics or safety function. Instead, we jointly explore and learn both the stochastic dynamics and the safety probabili- ties from trajectory data. Our method applies to broad classes of continuous- time, continuous-state stochastic systems, and provides provable guarantees on both safety and estimation accuracy. To the best of our knowledge, no prior work provides joint safe exploration and density estimation guarantees in this setting.
Assumptions
As formalized by the No-Free-Lunch Theorem (50, learning is only possible under prior assumptions. We now state and discuss the key assumptions used in this work.
Assumption (A1) (Initial safe controls). For e E [0, 1], a non-empty set So C D x [0, Tmax] is provided such that
s(0,1) (0,1)21-2 for all (0, E Sp.
This assumption ensures that at least one control is known to be safe at the outset, allowing safe exploration begin. In fact, So may be small singleton; only known safe control is required. Without such a point, safe learning cannot be initiated. We express the assumption in set form to allow for larger safe sets, which can accelerate exploration while preserving guarantees. This is a standard assumption in the literature of safe UCB methods [30] 22 18), and is realistic in many applications including robotics (39) and safety-critical process control 51 where systems naturally start in safe conditions, e.g., So { 0) 0 E D).
Resetting control. Let h R" + E. define a region in the state space from which resets are feasible. Specifically, if (() 2 0, then it is feasible to reset the system to the initial distribution Pa- Formally, this means there exists a mechanism (or control) that reinitializes the system from the current state X(t to a new state independently sampled from Po. We define the reset level at time t [0, Tuax] for given control ug as
r(0,t) A P un (1)) 0).
(8)
The function h delimits a region of the state space from which resets are feasible. Larger reset regions correspond to greater operational flexibility. In simulated environments, where resets effectively cost-free, the reset region can cover the entire state space IR." In contrast, real-world systems typically require substantial resources or manual intervention, making resets feasible only in restricted regions (e.g., near the original distribution po).
Assumption (A2) (Initial resetting controls). For & E [0, 1). a non-empty set Ry C D X [0, Tmax] is provided such that
r(0,t) 2 21-6 for all (0,t) e Ro.
(9)

---

This assumption guarantees the existence of at least one control capable of returning the system to the reset region with high probability, Assumption [A2]enables enables the generation of independent sample paths starting from the same initial conditions, which is crucial for evaluating variance and managing uncertainty during safe exploration. As with Assumption [AT) one known reset point is sufficient, though larger reset sets accelerate learning. A simple case is Ro {(0, 0) 0 E D). corresponding to systems that can always be reset from the initial condition. In practice, reset feasibility depends on system constraints: for instance. batch chemical reactors can often be reset only in early phases, before irreversible reactions occur [9]. In contrast, many autonomous systems like drones or driving robots usually be reset, at least during training.
Assumption (A3) (Smoothness of system dynamics). The map p lies in the Sobolev space H' (R with trol parameter dimensions, respectively. Moreover, supaeR= ( < +00,
1 max(n, m + 1), where n. and m denote the state and con-
(0 |lp(0, 4, (BH)))))) 80.
This smoothness assumption ensures that the system dynamics are sufficiently regular for our purposes. It is standard in statistical learning theory and underpins our convergence guarantees (52). Soboley regularity of the drift and diffusion terms in the underlying stochastic differential equation is expected to imply Sobolev regularity of the resulting state densities under standard conditions. This follows from classical results in parabolic PDE theory, where solutions typically gain regularity relative to the coefficients, roughly two derivatives in space and one in time. A formal analysis of this connection beyond the scope of the present work and is left for future investigation (see Bonalli and Rudi [17 for related results).
Proposed method
We propose method for safely exploring and learning system dynamics over parameterized control space H = [ug E D c J. Following the safe UCB framework [30 31 32 our goal is to select controls that reduce model uncertainty while ensuring, with high probability, that trajectories (i) remain within the safe region and (ii) end in the reset region. We jointly learn three models: dynamics model (state densities), safety model (safety probabilities), and reset model (reset probabilities), each equipped with confidence bounds from a shared kernel. This enables active exploration under high-probability constraints. After training, the learned models support inference on unseen inputs and yield a certified control set that can be deployed with safety guarantees.
The known safe-resettable set is expanded iteratively by alternating between system estimation (Section 4.2) and safe sampling (Section 4.3), leveraging prior knowledge of the initial safe and reset sets well as the regularity of the dynamics (0, (,, + pt((,
step-by-stey breakdown of the overall method is provided in Appendix with algorithm tables for each module and their computational complexities.
4.1 Initialization
Let N E NN denote the current iteration. We initialize N 0 using the known safe set Su C C D x [0, Tmax] We define the initial rase-reset set
D x [o, Tmax] and reset set Ro
To { (,, D
0, T max 1 t s T. (0,t') E So for all [0, (0, E Ro).
X
We select (Oo, la, To) To. ensuring that the control is known to be safe over [0, To] and ends in the reset region.
4.2 System estimation
In this step, we update the dynamics, safety, and reset models based on the observed trajectories, and compute predictive uncertainty for each.
Estimation at (0x, !x). At iteration N. the control uay is evaluated using Q stochastic trajectories. Here, N' indexes the iteration, and i indexes the i-th trajectory simulated under that control, each corresponding to an independent sample w/"' of the Brownian motion driving the system. We collect
5

---

the samples (Nuo (w. tN)) and estimate the state density at (0N,ty) tv) using kernel density estimator:
1 pR(x Xue (urf (N))s
PoN (x)
(10)
where pn(x)= - || Bn/2 (2 R1] R > O. and Bay2 is the Bessel J function of order
n/2 (See Bonalli and Rudi [77).
We then compute estimates of the safety and reset probabilities
S Pen (tN, dr,
ñuNNN JVERRMAOD200) pex (tN(tN dx.
(11)
Let the collection of values all observed points (10,, (,)) be
P(-) & (.)= $ t (S0,.)) R a (Po...))
Model update. We fit kernel ridge regressors for the density, safety, and reset functions using Matérn kernel lo (with Sobolev smoothness v) and regularization
(o x) & (())(K + NNN) (00,
(System dynamics)
(12)
(0 t) & Š(K + (ke)))
(Safety function)
(13)
PN (0, 1) A Ř(K + (NAI)='((0,0))
(Reset function)
(14)
where k(0, t) A (k((0, 1), (0;, ())) K (k((O;, (4), (0,, and is regularization
term. Although training data consist of discrete-time observations, learned regression models defined over continuous time, a distinction seldom addressed in the literature.
The predictive uncertainty at (0, t) is given by
(0, A k((0, 1), (0,1)) - k(0, N(,))
(15)
4.3
Safe sampling
We now select new point to sample by maximizing uncertainty over the safe-resettable region.
Feasibility criteria. A point (0, 6, T is feasible if (i) S T, (ii) the system remains safe up to time T. i.e., 80 (0,T) 2 1 e, and (iii) the trajectory ends in reset region with high probability, i.e., r(0,T) f.
We implement these constraints via lower confidence bounds (LCBs)
(C,)(O,T)
inf LE[0,]]
(S (6, t) (0.
(16)
(O, 7) (0, (0,
(17)
where Bi, By > O are confidence parameters set from known upper bounds on the RKHS norms of the safety and reset functions (see Remark[]]
We then define the sesetrase feasible set
I'y To {(0,t,T) D [0, Tmax] 1t ( (0,T) 2 e, (C 2
}
Sampling rule. We choose the next (ON+I,N=I1,Te: by maximizing uncertainty over the feasible set
(0N+1, TN+1)
arg max O N (0, (9,1T))TTx
(18)
Optimization performed using discretization or gradient-based methods. Several computational techniques for efficient optimization are presented in the Appendix (see Algorithm

---

Stopping rule. We stop the exploration once the maximum uncertainty over the feasible set falls below a threshold 17 > 0
max (N, t) <)
(19)
ensuring that exploration concludes once the models reaches the desired level of accuracy.
The derivation in Appendix A.7 shows that the confidence parameters depend on upper bounds of the RKHS norms of the safery and reset functions. These bounds may be available from prior knowledge of the system's 's regularity. We view this prior knowledge as a reasonable minimal assumption for guaranteeing safe learning under unknown dynamics. When unavailable, the bounds can be conservatively overestimated, ensuring safety but potentially leading to slower exploration. Developing adaptive strategies estimate these quantities without prior knowledge is a promising direction for future work, for instance through online adaptation via the doubling trick 153).
Remark 1.
Safety and estimation guarantees for Sobolev dynamics
Safety and exploration guarantees for safe kernelized UCB methods have been developed in prior work [B0 B1 32]. grounded in kernelized bandit theory [54] [55] [56,, which in turn builds on linear
bandit results [57 58. Building on this foundation, we establish novel theoretical guarantees for
safe exploration and dynamics estimation under Sobolev regularity. Complete proofs are deferred to Appendix A
Theorem 5.1 (Safely learning controlled Sobolev dynamics). Let 1j > 0, and assume Assump-
tions [A] (A3 hold. Set R Q1/(n+ze) and N Then there exist constants . C6 independent of N, Q 5, such that if
ei 1og(4N/6)) 2 < 1/2
then the stopping condition max ON (0,t) < i satisfied after at most S C2nl 22/110/1-0 iterations for any a (m+ 1)/ + 2v). Moreover:
(Safety): All selected triples (0,, l4, T3) satisfy s (0,,T)2211 and r(0,, 1-- providing safety guarantees during training. Moreover, the final set I'N includes only controls meeting these thresholds and can thus serve as a certified safe set for deployment.
(Estimation guarantees): For all (0,1,) TTN.
(po((t.) po(t, l)l S c.n. (0, s(0, $ c17, (0, - r(0,t)) cs?).
This result ensures that our method both respects safety constraints and achieves convergence rates adaptive to the system's 's Sobolev regularity. The condition on Q provides lower bound on the number of trajectory samples Q required per control to guarantee the prescribed confidence level. Up to logarithmic factors, it requires
Q N
where n is the state dimension and v the Sobolev regularity. For instance, when the system is sufficiently regular with 2 + + 1, the algorithm terminates in at most N O(1 - iterations, assuming Q Z N3, Although we do not analyze the size of N', its structure can be inferred from the available uncertainty estimates; formal characterization of T is left to future work.
Numerical experiments
We evaluate our method on a representative smooth nonlinear stochastic system. Specifically, the experiments aim assess the following: (i) satisfaction of safety and reset constraints, (ii) efficiency of exploration under different safety thresholds, (iii) prediction accuracy for dynamics, safety, and reset maps, (iv) computational cost and scalability.
All exponents be read + E, with arbitrarily small.
7

---

E= 0.1
E= 0.3
E 0.5
Figure 2: One trajectory per selected control after 1000 iterations for various thresholds.
System and environment. We consider 21) second-order dynamical system whose acceleration directly controlled by the input. The system evolves according to the controlled SDE
(1) (()dtt, (v u(t, ((),, V(t))dt +((
where (t) E IR? (t) E R2 denote position and velocity, u is the control function, and W, is Brownian motion. The noise amplitude a(X) is spatially dependent:
a(X)= Axp 1|X-xx11' XxI12
20
with x= (5.5), 2, and A 5. The initial state follows (0, Tpa)) with do 0.1, and the maximal time horizon is T max = 20. The system evolves within the bounded safe region (-10, 10) and each trajectory must end in the reset region defined disk of radius 2.5 centered at the origin. Such models arise in robotics and autonomous navigation, involving trajectory control with localized disturbances (e.g., slippery or uneven terrain). Figure Saaureeilllsse Iillstrraess the effect of such state-dependent noise through 100 trajectories generated under different controls.
Control space. Controls parameterized sequences of fixed accelerations of magnitude v, applied over intervals of equal length, yielding
applied in directions (fI....,d) ,,,).
During the exploration phase (0 < t S Texplo), each direction 0
u(t, X, V) (in) - V,,
with damping term -V ensuring velocity convergence. For (t ( Texplo), ) feedback controller steers the system toward No:
V, = K
Po - X (T
XI| -v).
with damping factor > 0. Controls are clipped to keep the system within the safe region. We set 2.0, k 0.5, 2, Texplo 6, and Misteps 500.
Method's hyperparameters. Our method depends on several hyperparameters that govern safety thresholds (e, E). confidence levels (,), kernel smoothness (A, and bandwidth R, with distinct values for estimating dynamics and constraints. We test () E (0.1, 0.3, 0.5, 20), with 1000 iterations and initial safe control (-=/3, 3/3). Candidate selection for uncertainty maximization is restricted to a local subset for efficiency (Appendix B Algorithm 4. A detailed discussion of each hyperparameter's role, tuning procedure, and practical heuristics is provided Appendix B.3
Safe exploration. Figure Tigure2 displays one trajectory per selected control after 1000 iterations, under various threshold settings
E (0.1, 0.5, =o0)). In Figure 3 the top row displays the learned safety level maps while the bottom row shows the corresponding reset probability maps for various threshold pairs E, with values increasing from left right in {0.1, 0.3, 0.5, +oc). Our approach only accepts candidate controls whose predicted safety and reset probabilities (estimated vía 200-path Monte Carlo simulations) exceed the predefined thresholds. By filtering only controls meeting the safety criteria, exploration is confined to a safe region with chosen probability of staying safe. Overall, these visualizations highlight how increasing the threshold values influences control selection, providing insights into the trade-off between exploration and safety.

---

186
6.0
Ieroton
Brotion
Bouysoon
Rersiton
COT
040
40
Brasion
188
(.0
Bouyton
Rerstio
0.1
0.3
0.5
Figure 3: Safety (top row) and reset (bottom row) probabilities over iterations for various thresholds.
Exploration rate and coverage. Figure 4 we plot the selected controls after 1000 iterations for various threshold pairs = with values increasing from left to right in {0.1, 0.3, 0.5, + Figures 2 and cleeary illustrate the exploration-safety trade-off. Relaxing thresholds leads to broader control coverage and faster information gain, but with decreased safety guarantees. Conversely, strict thresholds restrict exploration, particularly around regions with safety or reset probabilities close to the specified thresholds. This aligns with the intuition supported by our theoretical analysis: sample complexity tends to increase in regions where smaller uncertainty is required to proceed safely. As result, these regions act as bottlenecks, slowing down the process and potentially stopping exploration within the connected component that satisfies the constraints and includes the initial safe control. Additional results information gain across iterations are provided in Appendix
6 0.1
{ 0.3
= 0.5
8 E
Figure 4: Control coverage for various thresholds.
Safety and reset level prediction. In Table quantify the accuracy of the learned model for various threshold pairs = E in {0.1, 0.3, 0.5, +oo} by evaluating the prediction quality of the safety and reset levels over 1000 predictions. We report the mean squared error (MSE) and the standard deviation, with the ground truth provided by Monte Carlo estimates based on 100 samples (displayed in Figure 5p. In Figure plot the learned safety and reset maps, whose accuracies can be qualitatively assessed by comparing their values with those in Figure 5 As expected, prediction accuracy improves safety constraints relaxed, due to the broader coverage of the control space.
Table 1: Safety and reset level prediction error statistics (MSE + Std. Dev.)
Model (c, {
Safety MSE
Reset MSE
(0.1, 0.1)
0.7010 + 0.3847
0.6919 + 0.3764
(0.3, 0.3)
0.5217 0.4254
0.5197 0.4161
(0.5, 0.5)
0.3736 + 0.4299
0.3701 + 0.4258
(+00, +00)
0.0023 0.0065
0.0024 0.0062
Dynamics prediction. To verify that our method captures the underlying system dynamics, we compare predicted trajectory densities with ground-truth trajectories under known-safe controls. Qualitative results show close agreement in both mean and variance. Full visualizations and evaluation details are provided in Appendix C.1

---

Figure 5: Ground-trute safety (left) and reset (right) probabilities estimated via 100 Monte Carlo samples for 1000 randomly selected controls.
Computational considerations. Our method rums end-to-end in under 32 minutes standard hardware, covering candidate selection, simulation, evaluation, and model updates. Appendix [B.4 provides runtimes, hardware spees, and potential optimizations (e.g., sketching, parallelization), confirming the method's practicality on standard hardware.
0.1
6 E 0.3
6 = 0.5
E 00
Figure Learned safety (top row) and reset (bottom row) level maps for various thresholds.
7 Conclusion
We introduced a provably safe and efficient method for learning controlled stochastic dynamics from trajectory data. By leveraging kernel-based confidence bounds and smoothness assumptions, our method incrementally expands an initial safe control set, ensuring that all trajectories remain within predefined safety regions throughout the learning process. Theoretical guarantees were established for both safety and estimation accuracy, with learning rates that adapt to the Sobolev regularity of the true dynamics. Numerical experiments corroborate our theoretical findings regarding safety and estimation accuracy. By tuning the safety (s) and reset ({) thresholds, users can explicitly control the trade-off between conservative safety satisfaction and exploratory behavior. While our experimental validation focuses on a low-dimensions setting, the theoretical results scale with dimension: the convergence rates for the proposed estimators decrease polynomially with dimension and can mitigate the curse of dimensionality under sufficient smoothness. This makes the method applicable to higher-dimensional systems, which we plan to investigate in future work. Further research will include validation on physical systems (e.g., autonomous robots), improved scalability through fast kernel methods (e.g., sketching or incremental updates), comparisons with safe RL baselines, systematic analyses of kernel and threshold selection, and extensions handle abrupt dynamics and non-diffusive disturbances such as jump processes arising in pedestrian- hybrid systems. These developments will further support applications in safety-cun -eritccl control and decision-making under uncertainty.
interactions and
Acknowledgements
The Agence Nationale de la Recherche (grant ANR-22-CF448-0006 Pl: R.B.) provided funds assist authors with their research. A.R. acknowledges support from the European Research Council (grant REAL 947908).
10

---

References
[1] R. Bonalli, T. Lew. and M. Pavone. Sequential Convex Programming for Non-linear Stochastic Optimal Control, ESAIM: Control, Optimisation and Calculus of Variations, 28:64, 2022.
[2] T. Lew, R. Bonalli, and M. Pavone. Sample Average Approximation for Stochastic Programming with Equality Constraints. SIAM Journal on Optimization, 34:3506-3533, 2024.
[3] Kim Peter Wabersich and Melamie N Zeilinger. predictive safety filter for learning-base control of constrained nonlinear dynamical systems. Automatica, 129: 109597, 2021.
[4] Lars Lindemann, Matthew Cleaveland, Gihyun Shim, and George Pappas. Safe planning in dynamic environments using conformal prediction. IEEE Robotics and Automation Letters, 8(8):5116 14-513,,,,,,, 2023.
[5] Philippe Martin and Erwan Salaün. true role of accelerometer feedback in quadrotor control. 2010 IEEE international conference on robotics and automation, pages 1623-1629. IEEE, 2010.
[6] Matthias Plappert, Marcin Andrychowiez, Alex Ray, Bob McGrew, Bowen Baker, Glenn Powell, Jonas Schneider, Josh Tobin, Maciek Chociej, Peter Welinder, et al. Multi-goal reinforcement learning: Chal- lenging rohotics environments and request for research. arXiv preprint arXiv: 1802.09900 2018.
[7] Jeremy Morton, Antony Jameson, Mykel Kochenderfer, and Freddie Witherden. Deep dynamical modeling and control of unsteady fluid flows. Advances in Neural Information Processing Systems, 31, 2018.
[8] Jonas Peters, Stefan Bauer, and Niklas Pfister. Causal Models for Dynamical Systems, page 67 671-690. Association for Computing Machinery, New York, NY, USA, edition, 2022. ISBN 9781450395861. URL https://doi. 101 1155/35011444 3551175
[9] Zipeng Fu, Tony Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation using low-cost body teleoperation. In 8th Annual Conference on Robot Learning, 2024.
[10] Long Wei, Peiyan Hu, Ruiqi Feng. Haodong Feng, Yixuan Du, Tao Zhang, Rui Wang. Yue Wang. Zhi-Ming Ma, and Tailin Wu. Diffphycon: A generative approach to control complex physical systems. arXiv preprint ar arx:::
2024.
[11] R. Bonalli and B. Bonnet First-order Pontryagin Maximum Principle for Risk-averse Stochastic Optimal Control Problems. SIAM Journal Control and Optimization, 61:188 11:1881-1909 2023.
[12] T. Lew, R. Bonalli, and M. Pavone. Risk-averse Trajectory Optimization via Sample Average Approxima- tion. IEEE Robotics and Automation Letters, 9:1500 1507, 2023.
[13] George Keith Batchelor. An introduction to fluid dynamics. Cambridge university press, 2000.
[14] Giorgio Volpe and Giovanni Volpe. Simulation of a brownian particle in an optical trap. American Journal of Physics, 81()):224 20002222--00.000 2013.
[15] Onofrio M Marago, Philip I Jones, Pietro G Gucciardi, Giovanni Volpe, and Andrea C Ferrari. Optical trapping and manipulation of nanostructures. Nature nanotechnology, 8(11)807-819, 2013.
[16] Giuseppe Pesce, Philip H Jones, Onofrio M Maragå, and Giovanni Volpe. Optical tweezers: theory and practice. The European Physical Journal Plus, 135:1-38, 2020.
[17] Riccardo Bonalli and Alessandro Rudi. Non-parametric learning of stochastic differential equations with non-asymptotic fast rates of convergence. Foundations of Computational Mathematics, pages 1-56, 2025.
[18] Felix Berkenkamp, Matteo Turchetta, Angela Schoellig, and Andreas Krause. Safe model-based rein- forcement learning with stability guarantees. Advances in neural information processing systems, 30, 2017.
[19] Dale E Seborg, Thomas Edgar, Duncan A Mellichamp, and Francis Doyle III. Process dynamics and control. John Wiley & Sons, 2016.
[20] Robert Merton. Optimum consumption and portfolio rules in a continuous-time model. Stochastic optimization models in finance, pages 621-661. Elsevier, 1975.
[21] Lukas Hewing, Kim P Wabersich, Marcel Menner, and Melanie N Zeilinger. Learning based model predictive control: Toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems, 3(1):269-294) 2020.
11

---

[22] Matteo Turchetta, Felix Berkenkamp, and Andreas Krause. Safe exploration in finite markov decision processes with gaussian processes. Advances in neural information processing systems, 29, 2016.
[23] Horia Mania, Michael I Jordan, and Benjamin Recht. Active learning for nonlinear system identification with guarantees. Journal of Machine Learning Research, 23(32):1-30, 2022.
[24] Mahmoud Selim, Amr Alanwar, Shreyas Kousik, Grace Gao, Marco Pavone, and Karl H Johansson. Safe reinforcement learning using black-bor reachability analysis. IEEE Robotics and Automation Letters, 7(4); 10665 10672, 2022.
[25] Alexander Hans, Daniel Schneegaß, Anton Maximilian Schäfer, and Steffen Udluft. Safe exploration for reinforcement learning. In ESANN, pages 143-148, 2008.
[26] Teodor Mihai Moldovan and Pieter Abbeel. Safe exploration in markov decision processes. arXiv preprint arXiv: 1200,4810 2012.
[27] Stefano P Coraluppi and Steven Marcus. Risk-sensitive and minimax control of discrete-time, finite-state markov decision processes. Automatica, 35(2):301-309 1999.
[28] Peter Geibel and Fritz Wysotzki. Risk-sensitive reinforcement learning applied to control under constraints. Journal of Artificial Intelligence Research, 24:81-108, 2005.
[29] Javier Garcia and Fernando Fernández. Safe exploration of state and action spaces in reinforcement learning. Journal of Artificial Intelligence Research, 45:515-564, 2012.
[30] Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with gaussian processes. In International conference on machine learning, pages 997-1005. PMIR, 2015.
[31] Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Stagewise safe bayesian optimization with gaussian processes. In International conference machine learning, pages
PMLR, 2018
[32] Alessandro Bottero, Carlos Luis, Julia Vinogradska, Felix Berkenkamp, and Jan R Peters. Information- theoretic safe exploration with gaussian processes. Advances in Neural Information Processing Systems, 35:30707-30719, 2022.
[33] Jialin Li, Marta Zagorowska, Giulia De Pasquale, Alisa Rupenyan, and John Lygeros. Safe time-varying optimization based gaussian processes with spatio-temporal kereel, Advances in Neural Information Processing Systems, 37:95326-95338 2024.
[34] Matteo Turchetta, Felix Berkenkamp, and Andreas Krause. Safe exploration for interactive machine learning. Advances in Neural Information Processing Systems, 32, 2019.
[35] Anayo K Akametalu, Jaime F Fisac, Jeremy H Gillula, Shahab Kaynama, Melanie N Zeilinger, and Claire J Tomlin. Reachability-based safe learning with gaussian processes. In 53rd IEEE conference decision and control, pages 1424-1431. IEEE, 2014.
[36] Torsten Koller, Felix Berkenkamp, Matteo Turchetta, and Andreas Krause. Learningbbaee based model predictive control for safe exploration. In 2018 IEEE conference on decision and control (CDC), pages 6059-6066. IEEE, 2018
[37] Thomas Lew, Apoorva Sharma, James Harrison, Andrew Bylard, and Marco Pavone. Safe active dynamics learning and control: A sequential exploration-exploitatoo framework. IEEE Transactions Robotics, (8(5))2888-2990) 2022.
[38] Spencer M Richards, Felix Berkenkamp, and Andreas Krause. The Yyаршnov neural network: Adaptive stability certification for safe learning of dynamical systems. In Conference on Robot Learning, pages 466. PMLR, 2018
[39] Bhavya Sukhija, Matteo Turchetta, David Lindner, Andreas Krause, Sebastian Trimpe. and Dominik Baumann. Gosafeopt: Scalable safe exploration for global optimization of dynamical systems. Artificial Intelligence, 100:20 2023.
[40] Manish Prajapat, Johannes Köhler, Matteo Turchetta, Andreas Krause, and Melanie N Zeilinger. Safe guaranteed exploration for non-lincar systems. arXiv preprint (xXXi:2402.06566 2024.
[41] Amir Ali Ahmadi, Abraar Chaudhry, Vikas Sindhwani, and Stephen Tu. Safely learning dynamical systems from short trajectories. In Learning for Dynamics and Control, pages 498-509. PMLR. 2021.
[42] Gal Dalal, Krishnamurthy Dvijotham, Matej Vecerik, Todd Hester, Cosmin Paduraru, and Yuval Tassa. Safe exploration in continuous action spaces. arXiv preprint 10X.v:1801.08857,00 2018.
12

---

[43] Kim P Wabersich and Melanie N Zeilinger. Safe exploration of nonlinear dynamical systems: A predictive safety filter for reinforcement learning. arXiv preprint arXiv:1812.05506, 2018.
[44] Jaime FF Fisac, Anayo K Akametalu, Melanie N Zeilinger, Shahab Kaynama, Jeremy Gillula, and Claire J Tomlin. A general safety framework for learning-basse based control in uncertain robotic systems. IEEE Transactions on Automatic Control, 64(7):2737-2755 2018.
[45] Kim Wabersich and Melanie N Zeilinger. Linear model predictive safety certification for learning-based control. In 2018 IEEE Conference on Decision and Control (CDC), pages 7130-7133 IEEE, 2018.
[46] Mohammad Javad Khojasteh, Vikas Dhiman, Massimo Franceschetti. and Nikolay Atanasov. Probabilistic safety constraints for learned high relative degree system dynamies. Learning for Dynamics and Control, pages 781-792. PMLR, 2020.
[47] Nolan Wagener. Byron Boots, and Ching-Au Cheng. Safe reinforcement learning using advantage-bant intervention. In International Conference on Machine Learning, pages 10630-10640. PMLR, 2021.
[48] Akifumi Wachi Yanan Sui. Safe reinforcement learning constrained markov decision processes, In International Conference on Machine Learning, pages 9797-9806. PMLR, 2020.
[49] Shangding Gu, Long Yang. Yali Du, Guang Chen, Florian Walter, Jun Wang. and Alois Knoll. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint M..iv:2205.10331 2022. 2022.
[50] Luc Devroye, László Györfi, and Gábor Lugosi. A probabilistic theory of pattern recognition, volume 31.
Springer Science & Business Media, 2013.
[51] Joe Qin and Thomas A Badgwell. A survey of industrial model predictive control technology. Control engineering practice, 11(7):733-764, 2003.
[52] Loucas Pillaud- Fillau-Vivven, Alessandro Rudi, and Francis Bach. Statistical optimality stochastic gradient descent on hard learning problems through multiple passes. Advances in Neural Information Processing Systems, 31. 2018.
[53] Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and Trends@ in Machine Learning, 4(2):077-194 2012.
[54] Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.
[55] Michal Valko, Nathaniel Korda, Rémi Munos, Ilias Flaounas, and Nelo Cristianini. Finite-time analysis of kernelised contextual bandits. arXiv preprint arXiv: 1309.660098699 2013.
[56] David Janz, David Burt, and Javier González. Bandit optimisation of functions in the matérn kemel rkhs. In International Conference Artificial Intelligence and Statistics, pages 2486-2495. PMLR, 2020.
[57] Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedhack. In COLT, volume 2, page 3, 2008.
[58] Peter Auer. Using confidence bounds for cipoiiaaioonexxllraaiionn trade-offs. Journal of Machine Learning Research, 39N4,2:397-42,, 2002.
[59] Matthias W Seeger, Sham M Kakade, and Dean P Foster. Information consistency of nonparametric gaussian process methods. IEEE Transactions Information Theory, 54(5):2376-255) 2008
[60] Sattar Vakili, Kia Khezeli, and Victor Picheny. On information gain and regret bounds in gaussian process bandits. In International Conference on Artificial Intelligence and Statistics, pages 82-90. PMI.R, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and the introduction clearly state the contributions: a method for safe learning of controlled dynamics, theoretical guarantees under Sobolev regularity, and numerical evaluation of performance. These claims are consistently supported throughout the methodological, theoretical and experimental sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses limitations including model assumptions (Sobolev reg- ularity, known initial safe/reset sets), computational cost (see Assumptions, Theory and Experiments sections).
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All theoretical results are stated with assumptions and formally proved in Appendix A The main theorem clearly specifies conditions on smoothness and sampling.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The main paper and appendix detail the full experimental setup: system dynamics, control parametrization. thresholds, kernel parameters, and iteration budget. All elements needed to reproduce the results are included.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
15

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code and data will be made publicly available upon publication, and are shared during the anonymous review phase. Instructions for reproduction are included in the code's documentation.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/ public//uides/Codest ooaple/s//oooeeeeemiisssoooooooe
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
wwtpps.ccppblli//uuiieee guwde// CodeeSumiia ssi onyolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The system definition, exploration policy, control parameters, thresholds, hy- perparameters, and evaluation metrics are all clearly described in Section 6 and Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars (mean + std) are reported for prediction error in Table
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section and Appendix B..4 runtimes and hardware details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httppp/s/eeerrrsssccpppbbc/heeie
Answer: [Yes]
Justification: The research adheres to NeurlPS Ethics Guidelines. does not involve human subjects, personal data, or high-risk deployments.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
17

---

The paper addresses safe learning for stochastic control systems, with ap- plications in robotics and autonomous systems. It offers methods ensure safety during training, which is positive contribution. We see no direct negative societal impacts from the proposed methodology.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The method does not involve pretrained models scraped datasets and poses minimal risk misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external tools used are standard academic libraries. No external datasets are used.
Guidelines:
18

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new datasets or models are released. The experiments are synthetic.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects are involved in this research.
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: No LLMs were used in the design, implementation, analysis of the research method.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standardd components.
Please refer to our LLM policy for what should or should not be described.
20

---

A Proofs
A.1 Notations
We denote by ST. the set of positive-definte matrices in R The space of all measurable functions mapping a set A to a set is denoted by F(A, B3.. For any vectors u, U, their tensor product is denoted by u @ v. For any conformable operator A, we define A _ A + AI, where is the identity operator. The minimum and maximum between two scalars a, b are denoted as a - min(a, b) and a V - max(a, b). To quantify function smoothness, we use Sobolev spaces. For a domain I C R the Sobolev space (S2) consists of functions whose weak derivatives up order exist and qquare-integrain
The Sobolev embedding theorem states that if v > d/2 + k, then functions in In our work, we consider domains such as R" for spatial variables and D x [0, Maxx] for control-time timee spaces, where D C. R" is compact set of control parameters. Finally, for the RKHS on D x [0, max]] with kernel k, and any p D x [0, Tmxx] x R" + R, we define the mixed uup-nomm || r. (R=:9)) & suP.ER= (0, + p(0,t. t, x>
(22 are at least C* (1))
A.2 Proofs organization
The proofs are structured as follows.
1. Validity of confidence intervals (Section A.3 confidence intervals.
First, we establish the validity of the
2. Safety and reset guarantees (SectionA. A.4): maintain safety and reset properties.
We prove that controls chosen by the algorithm
3. Sample complexity bounds (Section A.5 Next, we analyze the sample complexity
required to achieve desired accuracy.
4. Learning rates for safety, reset, and density estimates at fixed (0, t) (Section A.6 We derive learning rates, under Sobolev regularity, for estimating the safety and reset levels, as well as the state density, evaluated at fixed (0, t) pairs.
5. Safe learning of controlled Sobolev dynamics (Section A.7): Finally, we establish com- plete guarantees for safely learning controlled dynamics with Sobolev regularity.
A.3 Proof of the validity of confidence intervals
Assumption (A4) (Attainability). There exists bounded and continuous reproducing kernel k defined on x [0, Tmax], with associated RKHS G, such that the safety and reset level functions satisfy
8, E.
(20)
This assumption ensures that the safety and reset level functions can be represented a suitable function space for estimation. In particular, it encodes prior knowledge; for example, if the functions are known to lie in a Sobolev (Hilbert) space Hm one may choose a Sobolev kernel of order m. This is a standard assumption in the literature on kernel methods. In many practical applications, such as industrial process control and robotics (39 5T], SDEs with smooth coefficients produce smooth probability densities [17]. ensuring that s(, t) smooth and can be accurately represented by Gaussian or Sobolev kernels. Therefore, this assumption is mild in practice.
This lemma establishes the relevance of the defined confidence intervals.
Lemma A.1 (Validity of confidence intervals). Under Assumptions
Al
A22)
and (A4)
for any
(0,t) E D x [0, Tmax] and 0,
|s 1) =(0, 1)1 S Bg
(21)
(0, r(0,t) t) S (f,
(22)
where B%
X- N maxie[1,N) 180. - ((0 + ||s||g. By
A
- 1112 max fe,,,, r(0,, +.] |||||
21

---

Proof. Without loss of generality, we provide the detailed proof only for sN, since the proof for TN entirely analogous.
We start by recalling the definitions
(0, A S* + NNT)
(23)
(0, A ((, k(0, (K + ((),).
(24)
where k()) A (k((O,t), (0,, (.)) K & (k((O), (0,,, and S = (S0,.:) )
We define the feature map @: : (0, 1) E x x [0, T + (0,0)),) g, and the operators
_ (0(0), 11 60N (2) &
(25)
Ô 1 N ((0,,t))0(00
2
(26)
such that ²= NV "00 K = 4' k((0,t), (0', t). 6(0', l'))g. and k(0,t) x- t0,,t).
Then.
(0) - 5 + NNI BEN ) t),,
(27)
= - N M4 - (0000((0
(28)
$""''000"" + N N GV ((,,tt)
(29)
= ~N-15"00 + Mlg *(00t).
(30)
using the push-through equality (I+ + AB) 1 A + BA) for any conformal operators A, B.B. Moreover,
(0) k((0,t), (0,t)) (( + NA. RM Sx S (k(00 t)
(31)
((0,t)" (1owg "(0" N NRR - 00000) t) o(0,t)" - -C((NC+NNlgsss (00,
(32)
+ =o0( ((ge
(33)
(34)
AIC,"12000 t)
(35)
again using the push-through equality.
To bound the error (0, t) s(0, t)), we decompose it as
(0, s(0, t)1 S (0, - (0, t)l + ()) - s(e,t))
(36)
((A)
2(B)
where S A (s(0,, t.) and t0) é N (c + (00,).
For the first term (A), we have
(A) N 1 2 - 60,
(37)
N 11$ 511R~ || || ((0,t)ll
(38)
S NN 112 1/2 max
| - s(0,, (0,, 12, I)llc
(39)
[c[|,N]
$ N 1/2
max 1S0.1, 1z) ox t0.
(00)
iE[L,N]
From Assumption (A4) s(0, 1) = (s, $(0, 0)), such that S = 4 s, and then (0, = s*x0"(("" A) o(0,t) >)=1d(0,t)
s* CCC + -) o(0,t) t).
Therefore, similarly, for the second term
(B), we have
(B) 1 (ÔO=1 t))
(41)
(42)
S 11121001G1 1/2 (0,00ll
(43)
= (1),))
(44)
22

---

Combining the bounds for (A4) and (B), we obtain the bound for s. Similar proof yields the bound for r.
In the following lemma, we derive confidence intervals for the proposed estimate of the system's 's dynamics 0, 1, + z)),
Lemma A.2. Under Assumptions (AI) (A2) and (A3) for any (f, D x [0, (0,Tm) we have
(t, po(t, (220 S (0,
(45)
by defining FN A X-1N-1/22 max l1io,,; (-) pa, (4: [ 11
Proof. Given any (0,t,z) E D x [0, (0,xm x R", we have
N
po((, -
Σ
az (0, (1,0),,,,,,0)) (2),
(46)
i=1
with (00 (K+ NNT) 0), K (b:((0,, 4,), (0,, (M0,) 4))
=
We detine the feature map ō = (0,t) E D 0, T max
G, and the operators
- 16(01, 41),
tx)] E
(47)
Ò A 1
(0,, 4) 8 (0,,44)
(48)
such that Ō= N 400 K = 4 4. k( t), (0', (0), = (d(0,t). t). 6(0', t'))g. and k(0,t) = x (00, t).
With same derivations than in the proof of Lemma A.1 for any (0, (, D [0, Toxx) x R", have
[t, - po(t, x)) s (A) + (B)
(49)
with
(A) S ----~~-
N 1/2
max 10o. (x) - pe, (ti, 2);, (N [E[L.N]
(50)
(B) 1 [pp xr) 1gg N (0,
(51)
Therefore, for any (0,t,)) D x [0, maax] x RR, we have
(t. :) (t. (3 S (0,
(52)
by defining Sy A - 1/2 max (t) Po, (tz, (R)) + (p
A.4 Proof of safety and reset guarantees
A direct consequence of Lemma A.I is the safety and reset guarantees for the method.
Lemma A.3 (Safety guarantees). Under Assumptions (AI)
(A2) and (A4) the algorithm selects
only safe trajectories. Namely, for any i E N², we have s'( T:)
Proof. For any E [1, we have by construction infre(o,r:1 (Sx(0,,t) (N,n(Or,t)) 2 -E
Moreover, from Lemma A.1 we have s(B,t) 2 $(0, t) (N, for any (0, E D 10, Tmax]. such that
(0,, T:)
inf s0i, 1) <E(),T;
(53)
(s(O,, t) (Nox((..tt) eE().T;)
inf
(54)
2 E.
(55)
23

---

Lemma A.4 (Reset guarantees), Under Assumptions (A1) resetting trajectories. Namely, for any i NN, we have r 0,
(A2)
and (A4
the algorithm selects only
Proof. Similar proof as for Lemma A.3
A.5 Proof of sample complexity of confidence intervals
Assumption (A5) (Sublinear information growth). Let the maximum information gain up to N observations be defined as
(0,0
NN
log 1+
(56)
( Toaee 1
AN
We assume there exist constants a E [0, 1] and > such that, for all E N.
YN S cNa
(57)
This assumption always holds for when l: is bounded, since 7N is sum of terms bounded by Dx k((0,t) t), (0, t)). As a decreases within [0, 1]. the assumption becomes stricter. It quantifies the maximal total information that can be acquired about the unknown function after N observations. Specifically, when < the sublinear growth of information gain with N reflects diminishing returns as more data points are observed. This growth rate measures the effective dimension of the learning problem, influenced by the regularity of the RKHS and the dimensionality of x [0. max higher regularity of RKHS functions and lower dimensionality of x [0, Tmax) lead to faster decay in information gain.
Example 2 (Sublinear growth for common kernels). For many commonly used kernels, IIN exhibits sublinear growth in N, which is crucial for obtaining sublinear regret bounds in bandit problems. For example, assuming compact domain D) Tn and setting X N-1:
RBF kernel: IN Olog"*+1 N). This logarithmic growth arises from the high smoothness of the RBF kernel, which causes rapid reduction in uncertainty about the manknown,functionn more observations are collected.
Matérn kernel with v 1: YN -O(mm/(m220) log 2mg2m/((m222)) N). This sublinear growth rate is faster than that of the RBF kernel, reflecting the lower smoothness of the Matérn kernel.
We refer the reader to Seeger et al. 159); Srinivas et al. 1549; Vakili et al. [60) for additional examples and their associated proofs, which show that the growth rate of YN depends on the eigenvalue decay in the Mercer expansion of the kernel, when available.
This lemma establishes the sample complexity of the proposed confidence intervals.
Lemma A.S (Sample complexity of confidence intervals). Under Assumptions [AI] (A2) and (A5)
for any O, considering the stopping condition max(0.t,Tee oN (0, 1,, then the proposed
method stops in N en) -2. steps where >> 0 is constant that does not depend on N and n.
Proof. Hence, under Assumption (A5) when the algorithm stops we have
.N
2T? & Σ (f 44) S
Σ log(1 + of(0,,t))) S IN S oNo
(58)
using x & 2ogg(1 + x) for r E [0, 1]. and overloading the constant c 0. Hence, N S c 1n 1-0.
A.6 Proof of learning rates for system maps estimation fixed (0, t),
This lemma provides a bound on the density estimation error at the selected points (0,, t,).

---

Lemma A.6 (Density estimation learning rates). For each i E []. define _ (ts :), and
Q
P: _ Pol.c. -
pR(x - (w,, (4),
(59)
Q
1=1
R O, and Buj2 is the Bessel J function of order where pR(x) 1=//1|==||- - n/2 Bn (2m 111) n/2.
Under Assumptions (AI) (A2) assume that (
llpo(t, )1l < for n/2. Set
R Qutar Then, with probability least - 6, the following holds
1p. pell ~ (Re) S clog <log(4N//)) 1x
(60)
iE[L
for some constant e independent of N. 2.
Proof. Fix any s > 0. The Sobolev embedding H²/2+t((R)) + L (RR) gives
| Pall (R=) I C. ll. Pi pillam//+=((00)) i = I,...,.. N,
where > 0 depends only on n,
Following steps and 4 of Theorem 5.3 in Bonalli and Rudi (17), for > n/2 establish
life 10-Pp P:l ma/244 S max E LE
max
*E[L,]] [|p:|
(61)
2 (+ log(4N/6)122-112
where Vn the volume of the n-dimensional unit ball.
Under the assumption that Dx (t, )l < co, we have in particular
maxie[I,N] H- < co. Hence, with probability at least - 6,
max Iips - Pell S ce (R + Rate (2 [E[L,N]
(62)
for some C₂ independent of N, Q, R, 8 (but possibly depending on n, v,
Finally, setting R = Q m 27 balances the two terms in (62) and yields
max vE[
1 - Pill I cllo(44N/8)
2nt4r
20422
(63)
for some constant e 0 that does not depend on 6,
This lemma bounds the pointwise estimation errors of the safety and reset levels at the selected points (0,, f4) in terms of the nnrm error of the corresponding density estimations (t,(t,)) ))-
Lemma A.7. Under Assumptions
(A7 (A2) we have
max 1s0,.t - s(O,, t+)l S V, max llp: - PillE= [E[L.N]
(64)
[e[1.N]
max =E[[,]]
Poo.e - -((,, Vr max 11.s - Dell [E[],N]
(65)
where ps e Po, (t., V. A fg.. 1 (1(x)20) dx, and V, e fg. 1 (M(=)20)
Proof. For any E [1, NI. using Hölder's inequality, we have 180..: - s0,,, 44) S Valld: - p:ll p where p; e "o, (t..)) and V, A J 1 (g(x))2fdzz.
(66)
Similarly, (fe.., r(O,, t4) s V, pill== where V, & fgn (x(x))20)z.
Throughout, exponents involving should be interpreted for arbitrarily small 0.
25

---

A.7 Proof of safe learning of controlled Sobolev dynamics
We conclude the following theorem from all previously established lemmas.
Theorem A.8 (Safely learning controlled Sobolev dynamics). Let 1 O, and assume Assump- lions (AI) (A3) hold. Set R = Q1/(n+20), Then there exist constants C1,..., > 0, independent of
N, Q, 6, T, such that if
2x Ci looglo////// 1 2T SN-1/2
then the stopping condition max (0, <nis satisfied after at most N S c2n 22((1-a iterations for any > (m + 1)/ 1+ 22). Moreover:
(Safety): All selected triples (0;, 1i, T3) satisfy 800 (0) and r(0,, T:) 2 g. The final set Tx n includes only controls meeting these thresholds and can thus serve certified safe set for deployment.
"Estimation accuracy); For all (0,t,T) E EIN.
()o((t,)-Pooll
po(t, -
S czn, (0, s(0,t) t)] S ca7) (0x - r(0,t)) s esn.
Proof. From Lemma A.1 and Lemma A.2 for any (0, E D x [0. Tmax] and > O.
[& [0, t) s(0, ()) S x t), [0, r(0,1) 1) N l): (t,) poll, )) (x0) S NN
(67)
(68)
(69)
where 8%
A-'N-11 1/2 max 18,.11 s(0,, + || BN
--12 max fe,, r(0,, t4)l + ||lle, and 8% A x max Ii00, (-)
Po, ((1;)))l===<e)) + [pp] (Rr:g))
Then, from Lemma A.7 we have
max (E[I.N]
180,0: - s(Or,tt)) S V. max
Il pillo
(70)
[E[I.N]
max |o,.4. - r((0,, V. max Il.s pille [E
(71)
However, from Lemma
A.6 with probability at least 8, we have
max E
I - Pollo=(R>)) S clog(4N/6)122
(72)
such that there exists c₁ > 0 independent of N.Q. 8, n such that if
c 1og(4N/6))
2= < (V. VV.)
(73)
then
1 - Pill (ell S 1/12 [E[L,]]
max
(74)
Therefore, from Lemma A.7 we have, if A N-1, for any E [1, NJ.
B& A N1/2
max |80 s0,,,, + |ااا| JE[L.V]
(75)
< 1 + ||ا||ء.
(76)
Similarly, we have 8y - + ||r||s and By 1 + ||||| (1
Assumption (A3)
simultaneously guarantees || (R= A sup || (0,t) + p(0,t, 1gg
+00, (up() (x mav] [lpo(t, (t,-)))))
so with v n/2. Furthermore, considering kernel that induces Sobolev RKHS 9 of order v, Assumption (A4) holds true, and Assumption (A5) holds for any a > (m + 1)/(m+ + 1 + 2v) as mentioned in the examples of Assumption (A5)
Therefore, from Lemma A.3 and Lemma A.4 we have that:
26

---

All selected controls are safe, i.e., (b,, 2 1 for any i E N'.
All selected controls ensure reset, i.e., r(0, T;) 2 & for any E N
Moreover, Lemma A.5 ensures that for any a > (m + 1)/ (m + 1 1+ 2n), with probability at least 1 8, the stopping condition
max 8,
(N(0,)) ?/3
(77)
reached for N & c2n 1-a where 02 does not depend N, Q,
B
Implementation details
This section details the implementation of the method described in Section available on GitHub dimotteddynamics-saff-llee
open-source Python library. We detail all computational steps, in- cluding vectorized implementations using Python libraries such as NumPy for efficiency. Additionally, outline the computational complexity of each step to provide insights into scalability.
B.1
System estimation
Density estimation.
For each data point (0 4, Ti), the density is estimated with
Q Xnn, (wf,, tz)),
po (2)
Σ
pn(s
Q
j=l
where Q is the number of samples generated for each control 0g and time 44 pre(x)) is kernel density function, typically defined as p=(x)) Rm/ ||=l|=///__,/ (2 R[(=l)) with R >> and Jn/2 the Bessel function of order n/2. Xuo, (wt, t4) are the system trajectories generated under control ug.
Computational complexity. Evaluating e (x) requires ((Q) operations per data point. This step trivially parallelizable across data points, since all kernel evaluations independent.
Algorithm 1: DensityEstimation((z)))
Input: Q trajectory samples { at (0i, fi), kernel pR
Output: density estimator Pa... R2a + Ryn
x is queried then return Por.e. (x)
Pn(x - 2z)
Q
j=1
Probability Computation (s, f). The vectors P, S, and R are constructed from the observed data points (00 4G T.) as follows:
Do (41.:) Do (12)
80
for d1
802.
f02.22
P(.) =
$
Dex (tN:))
SōN ANd
fex AN-
The vectors Š and R are computed by integrating the densities over the safe and reset regions, respectively:
$8, I - Po..e, (z) dx,
27

---

f0 (neR=:h(=)20)
Pa,,, (x) dx.
These integrals are approximated using Monte Carlo integration. we can sample Ty ~ to then
80 2J
(g(xx) 2 0},
f0, ~
(
Q'
Alternativlly, using trajectory samples {Xuo, (wg,,t))33
Q
80,0 Ho((u,, (w), 1:)) 2 0},
1 1(((Xun um (w), 0}.
Q
==1
Here {() denotes the indicator function.
Computational complexity. Constructing P involves N density evaluations, each costing ((), for total of ((NQ). For $ and R, Monte Carlo based samples from costs O(NQ'), while using trajectory samples costs (NNQ). Overall, constructing (P, S, R) costs ((NQ) (trajectory MC) or (Q + Q")) (using both), All computations are trivially parallelizable across evaluation points.
Algorithm 2: Computerrobabiltiies(Ct... 9, h, Q', mode)
Input: density estimator Po.... constraint functions g, h, number of MC samples Q', mode E {density, trajectory]
Output: probabilities ($p,... f0...)
1 if mode = dens ity then
Sample fax 12 from Po
8
Hg((z) 2 00
Q'
k=1
Q'
1
f
1(((vx) 2 0}
Q'
k=1
5
else
// trajectory
Reuse trajectory samples {zr)) at (0,,4)
Q
7
8
+
1g(x;)) 2 20
j=1
Q
8
f
Σ
1 1(((z,) 20
j=1
return f)
Kernel-based estimation of system maps. Gram matrix K constructed as
Given N observed data points {(0,, 4, T;) the
K.J = A((O,, t4), (0,,
where & the kernel function. For new input (0, t), the estimates for the system dynamics, safety, and reset functions
2o(1,2) (K + ((0,)
(x, S(K + NAI) -
tx(),t) = ÅCC + NAI) k(0,t), -((,t))
28

---

where k(0,t) = ())( (0,, (0,,t))));1 À is a regularization parameter, P(-) (po..: S (S0,,;))=1, R (fo,,t))i=1-
The predictive uncertainty for (0,t) is computed as
t)) = k( f), (0, () k(0,0))(K (NA)))KKKNNN)NNA NAT)
Computational complexity. Gram matrix construction requires O(N2) operations, matrix inver- sion involves (NV)) operations. Evaluating (0, t), (0 t), (0, t), or o (0, t), costs ONN2) per prediction.
Algorithm 3: (itKernelMaps((((04444)
P(), S, k,
Input:
inputs ((0,, targets P(.). S. R. kernel k, regularization X
Output:
query operators for (t, -), (0,, t), PN (0. t), (0,1))
1 Build K with Kij- <- k((0,, (4), (0g, 1;)); set Kx
NXT
2 Compute and store K
3 Precompute: ag S on k R
Query at (0, t) (and optionally at x):
1. ke +- [k((0, t), (0,, 1)))
2. < k
// used for variance and density map
3. (0, + kaas
4. (0,t))) + TTaR
5. (0, k((0, (), (0. - T
6. If density at x is requested: build P(x) + Penm (x), PaN (x)] and return ................................
xt(t, + T
B.2 Safe sampling
Feasibility criteria.
The feasibility criteria for safe sampling are defined as:
(0, inf (&x (0, By (),
=E(0,7]
(0,1) A fN (0, T) - ON (0,
where 8x, B > 0 are confidence parameters. The safe-resettable feasible set is:
IN (0,1,7) E D [0, Tiaa]2 1 IS LCB3 2 - (0 21)))111 2-
with
To {0,44 T miaxx E D 70,xm
(0.1) (0, E So n Ro}
Sampling rule.
The next control, time, and trajectory are selected by solving:
arg max (8( t): (0t,T)eT'
where
(0, = k(0,t), ((. -NN)
Stopping rule.
Exploration terminates when:
max
N N (0,t) n
(T
where 7) >> 0 is the user-defined uncertainty threshold.
29

---

Computational complexity.
Set construction xTn):: Evaluating LCBN and LCBN involves O(N2) operations for each candidate point. For M candidate points, constructing I'N costs (MMN-).
Uncertainty evaluation: Evaluating involves matrix-vector multiplications and inver- sions, where Gram matrix construction costs O(N2), matrix inversion costs ((N)), and per-query uncertainty evaluation costs O(N2).
Optimization (arg max):
For discretization over M candidates: ((M)2),
For gradient-based optimization: O(kN2), where k is the number of optimization iterations, using nonlinear constrained solver (e.g. SQP or interior-point). Each iteration involves computing the gradient N t), costing ((V3).
Efficient sampling algorithm. In practice, the sampling process can be improved to reduce computational costs by focusing promising regions and avoiding unnecessary evaluation of low- uncertainty points: (1) threshold-baae filtering: select any candidate where ON (0, t) > y to avoid costly global optimization while maintaining guarantees; (2) exclude evaluated points: skip candidates where N (0, t) < 17, assuming the uncertainty is non-increasing (true for À = 1/N); (3) localized sampling: restrict N points near the initial safe set and previously selected points. Algorithm 4 implements a region- regionngrowing strategy that encourages local exploration around previously selected safe points.
Algorithm 4: Efficient sampling algorithm
Input:
Initial safe-resettable set So n Ro. threshold y,, feasible set I' n
Output:
Updated sets Pk. Ag. and the selected candidate (if found)
Initialization:
2 Set Po = So n Rg and Ao
3 Define the feasible set using localized search region:
I = I'N ((),t,T) d((0, t), Pk) rk \ As.
where (((,,),PP, denotes the minimum Euclidean distance from (0, any point in Pk. Iterations:
for k = 0, ..
foreach candidate (0, T) E k do if (0, 7 then
Select the candidate (0, t, Update the safe-resettable set:
Ps+1 = PA ((0,t,T))
return (Pa+1, (0,t,T)
end else
II
Update the excluded set:
A = A ((0,t,)))
13
end
14
end
Expand the radius: Fk - Tk+1
Recompute I
r* then
17
18
return (Pt. Ac, None)
19
end
end
30

---

j. t
44
Figure Predicted probability density of the trained model E 0.1) along with 10 true trajectories, for three test controls ((-0.81 1.20), (-1.07 0.92), (-1.13 1. 1.18)) from the known set of safe controls for this model. Predictions were computed on spatial grid of 2500 points and times t E [0,16]] 16].
B.3 Role and tuning of hyperparameters
Hyperparameters critically influence the balance between exploration, safety, and computational efficiency in method. The safety and reset thresholds (e, {) directly control this balance: lower thresholds enforce stricter constraints, restricting exploration to safer regions at the expense of slower coverage, whereas higher thresholds allow broader exploration but increase risk. The confidence parameters (3,, (,) modulate how conservatively the safe-resettable set expands, reflecting tolerance uncertainty in safety and reset predictions. The parameters À and y define the smoothness of the estimated functions, thus capturing prior knowledge about the system dynamics. Different hyperpa- rameters may be chosen for each map: 'kde and A for density estimation, and Toolleet, Acollects Scollect for safety and reset predictions, as these functions may have different smoothness characteristics. Typ- ically, hyperparameters are tuned using validation data. Parameters for density estimation (7kdes Akde) can be optimized after data collection by maximizing log-likelin likelihooooo.... In contrast, the parameters governing safety and reset exploration (Yeolleet, Acollect Seotleer) must be set beforehand, as they directly impact safe exploration. When prior knowledge is limited, we recommend initially conservative settings hhghiggg-hhhhhee y, low high 8, and large kernel bandwidth R-and gradually relaxing them based on data-driven insights. In our experiments, "ade and Aude were visually tuned using validation controls (2tr/3, - /3)), 2 -x /3), and (0, -3/3.. Meanwhile, Tcollect, Acollect, Pcallent were set heuristi- cally, assuming reasonable prior smoothness estimates to minimize computational overhead. Without such prior knowledge, comprehensive hyperparameter tuning would likely demand significantly higher computational resources, as extensive parameter searches become necessary.
B.4 Computational considerations
To provide practical insight into computational requirements, we report measured execution times from experiments on a standard machine (Apple M3 Pro. 18GB RAM). Each training iteration required approximately 1.92s + 0.002 totaling around 31.77 0.38 minutes for 1000 iterations, based on 20 repeated runs. This includes candidate selection, trajectory simulation, computing safety and reset probabilities, and model updates at each iteration. Density predictions took approximately 10 seconds in average for computing p(0, t, x) over grid of 2500 x for each considered (6, t). Although the computational times are non-trivial, they remain manageable on standard hardware for the problem sizes considered. It should be noted that several approximation methods-such as sketching for matrix inversion and online matrix inversion-as well as parallelization techniques (e.g., parallelizing the simulations) can be leveraged to alleviate the computational burden. However, exploring these techniques beyond the scope of this paper.
Additional experimental results C
C.1 Dynamics prediction accuracy
we present the predicted probability density from the model trained with & = = 0.1, alongside TO true trajectories for three test controls ((-081 1.20). (-1.07 0.92), (-1.13 1.18)) chosen from the known set of safe controls with uncertainty below 0.1. The density evaluated over a spatial grid of 2500 points and time steps t [0, 16]. This visualization provides qualitative assessment of
In Figure 7
31

---

the model's dynamic prediction accuracy. The predicted probability distributions closely match the true dynamics, exhibiting similar means and variances over time.
C.2 Information gain iterations
To complement the analysis of exploration behavior, we report the cumulative information gain over the course of training, for different safety and reset thresholds E {0.1, 0.3, 0.5, +00 Figure shows how the information gain evolves as new trajectory data is collected.
We observe that larger thresholds, which allow mation acquisition. In contrast, stricter thresholds slow down exploration and yield more gradual information growth. This reflects the fundamental trade-off between exploration and safety: ensuring high-probabiln safety requires restricting the sampling space, particularly in regions with high model uncertainty.
aggressive exploration, result in faster infor-
1.0
1500
==(=0.1
#=(=03
Information gain
0.8
1000
0.6 mm
-0.2
0
200
400
600
800
1000
Iteration
Figure 8:
Cumulative information gain iterations for various thresholds
{0.1, 0.3, 0.5, +00}.
32


Reviews:
[{"review_id": "0692kqa1Xg", "reviewer": "Reviewer_oPjs", "summary": "This paper proposes a method for learning the dynamics of controlled stochastic systems from trajectory data in the presense of safety constraints, and provides safety guarantees during training and deployment. The method is based on expanding a safe control set using kernel-based confidence bounds. Also, adaptive learning rates that improve with the smoothness are derived. The method is validated on a 2D second order dynamical system.", "questions": "If I understand it correctly, the method requires the upper bounds ||s||_G, ||r||_G, which are not obviously available (as also bounds of RKHS norms).\nTo increase the evaluation, it would be good to see how can these bounds be non conservatively estimated for real systems or system models? Does that impact the practical use of the method?  How sensitive are the safety guarantees to misspecification of  ||s||_G, ||r||G, and ||p||? Can you demonstrate similar performance with large underestimation of the bounds?\nHow are the hyparparameters β^s, β^r, λ, γ, R tuned?\nAnother suggestion for more convincing demonstration is to increase the complexity and dimensionality of the proposed numerical example and show how the method works in a scenario closer to reality.\nAlso, it would help to understand how the method performs with a comparison to safe learning control methods like model-based safe RL and control barrier functions, or more baseline method comparisons.", "limitations": "- the method relies on known bounds, and it is not specified how to estimate them or how sensitive the guarantees are to misspecification\n- the numerical simulation example is limited", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\nThere is a solid theoretical analysis which provides safety guarantees and adaptive convergence rates under Sobolev regularity assumptions. \nThe paper is generally well-written with clear problem motivation and clear description of the approach. \nThe addressed problem in control of safety-critical systems is important with applications in robotics, autonomous systems, and other domains.\nThe paper extends safe UCB methods to continuous-time stochastic differential equations and provides joint (probabilistic) guarantees for dynamics estimation and safety.\n\nWeaknesses\n\nThe method relies on assumptions as Sobolev regularity, RKHS membership, eventually known safe/reset sets, which might be limiting.\nThe validation of the method is very limited (a 2D toy example) and does not show or address the scalability of the method to more complex systems or higher dimensionality, especially in terms of computational complexity.\nThe comparison with other safe learning approaches is limited. It is unclear what is the impact of the reset in more realistic applications.", "was_revised": true, "final_justification": "With the additional clarifications provided in the rebuttal, I find the authors are proposing a promising method. \nHowever, the validation of the method remains limited. I therefore maintain the initial score on the paper.", "rebuttal": "Thank you for your thoughtful review. Below are our responses to the points you raised.\n\n**(General comment).** This work takes a first step toward provably safe learning under unknown stochastic dynamics, a setting largely unexplored compared to prior work assuming known dynamics. Our focus is theoretical and methodological; code will be released for reproducibility. Experiments on more complex problems are part of ongoing work. The Sobolev and RKHS assumptions are standard in learning theory and are commonly used to derive generalization guarantees. While the rates depend on dimensionality, this is inherent to the problem. Sobolev regularity helps reduce this effect and leads to near-optimal rates.\n\n*\"The method relies on assumptions as Sobolev regularity, RKHS membership, eventually known safe/reset sets, which might be limiting.\"*\n\nWe acknowledge that Assumptions (A1) and (A2) may appear strong, but they are minimal in what they require: only one known safe and one known reset point are sufficient to initiate safe exploration. The sets $S_0$ and $R_0$ can be as small as singletons. Without at least one such point, safe learning cannot begin. In many systems, states of the form $S_0 = \\{(0, \\theta) : \\theta \\in D\\}$ and $R_0 = \\{(0, \\theta) : \\theta \\in D\\}$ are valid and realistic, as systems are often initialized in known, safe and resettable conditions.\n\nWe express the assumptions in general (set-based) form to support more practical and informative choices: our approach leverages non-singleton safe and reset sets to accelerate learning while preserving formal safety guarantees.\n\n*\"Validation is limited to a 2D toy example, and scalability is not demonstrated.\"*\n\nWe agree that evaluating the method on higher-dimensional systems would be valuable. In this paper, we focused on a low-dimensional setup to clearly illustrate the core theoretical guarantees. Higher-dimensional experiments are planned for future work. Nonetheless, the theoretical results scale with dimension, with convergence rates for the three estimators of order $\\mathcal{O}(N^{-\\nu/(m+1+2\\nu)})$, provided the number of samples per trajectory satisfies $Q \\geq \\mathcal{O}(N^{(2\\nu + n)/(2\\nu - n)})$ (up to log factors). This means the curse of dimensionality can be mitigated under sufficient smoothness (i.e., large $\\nu$), making the method applicable in higher-dimensional settings.\n\nThe main computational cost arises from Gram matrix inversion, which can be reduced using standard approximation techniques such as Nyström methods, without sacrificing theoretical guarantees (see Rudi et al., 2015). Appendix B provides algorithmic details and complexity analysis. In short, the per-iteration cost is $\\mathcal{O}(i^3 + 3Qi)$ (vs. $\\mathcal{O}(i^3 + Qi)$ for standard regression), leading to overall complexity $\\mathcal{O}(N^4 + QN^2)$. Under sufficient regularity, $Q < N^2$ is enough, making evaluation costs negligible and overall complexity comparable to batch kernel ridge regression (unsafe).\n\n*\"Comparison with other safe learning approaches is limited. Impact of reset unclear.\"*\n\nWe agree and thank you for raising this. Our setting differs from most safe RL approaches, which typically assume known dynamics or do not offer formal safety guarantees. In contrast, our method ensures certified safety during exploration under fully unknown stochastic dynamics.\n\nThe reset mechanism is optional but beneficial: it prevents the learner from restarting exploration far from the initial safe region, improving sample efficiency in practice. Resets are also common in prior safe learning works, e.g., \"Turchetta, M., Berkenkamp, F., and Krause, A. (2016), which rely on resets to known safe states to enable safe expansion. Safe exploration in finite Markov decision processes with Gaussian processes. Advances in neural information processing systems, 29.\", which rely on resets to known safe states to enable safe expansion.\n\nImportantly, the impact of resets is reflected in our theoretical analysis. Our convergence rates depend on the Sobolev regularity of both the safety function and the reset mechanism. Thus, even if the safety constraint is highly regular, the reset function can become the bottleneck if it has lower regularity. In that sense, the reset constraint may limit the overall convergence rate.\n\n*\"The method requires upper bounds on $\\|s\\|$, $\\|r\\|$, which are not always available. How sensitive are the safety guarantees to misspecification?\" \"How are hyperparameters $\\beta^s, \\beta^r, \\lambda, \\gamma, R$ tuned?\"*\n\nWe agree this is a key practical limitation.  The bounds on  $\\|s\\|$, $\\|r\\|$,  are required for our theoretical guarantees. When not known a priori, they can be conservatively overestimated, which ensures safety but may result in more cautious exploration. In some settings, such as Sobolev RKHS, these norms can be estimated from prior knowledge, for example using bounds on $|p|_{H^\\nu}$.\n\nAlthough a quantitative analysis of how safety degrades under misspecified bounds is still missing, we believe this requirement is a reasonable trade-off for enabling provable safety under fully unknown dynamics.\n\nIn our experiments, hyperparameters ($\\beta^s, \\beta^r, \\lambda, \\gamma, R$) were selected heuristically based on smoothness assumptions to reduce computational overhead (see Appendix B.3). Developing more adaptive tuning strategies to estimate these quantities more accurately is a promising direction for future work. Following Shalev-Shwartz (2012), one could explore approaches based on the doubling trick over a finite decreasing sequence of candidate values, which may help preserve theoretical guarantees without requiring prior knowledge of the quantities $\\|s\\|$ and $\\|r\\|$.\n\nShalev-Shwartz, S. (2012). Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2), 107–194.\n\nMore broadly, this reflects a central challenge in safe learning: guaranteeing safety when both the system and its regularity are unknown. Our framework allows for variable smoothness through flexible kernel assumptions, but understanding safety under uncertainty in regularity remains an open and important problem.\n\n*\"Suggestion: increase complexity and dimensionality of the experiments; compare to safe RL baselines.\"*\n\nWe agree that evaluating on more complex, higher-dimensional environments is an important next step. This work aims to establish a principled foundation for safe learning under unknown dynamics, and scaling to more realistic domains (e.g., robotics) is part of our ongoing research.\n\nRegarding comparisons to safe RL and CBF methods: we acknowledge their relevance and value. However, to our knowledge, most existing approaches either rely on known or structured system dynamics or do not offer provable safety guarantees in the fully unknown setting we consider. We would be grateful for any specific references we may have overlooked and will gladly include relevant baselines in future versions or extended work.\n\nSeveral notable examples that reflect growing interest in safety-aware learning, while operating under model knowledge, include:\n\nPrajapat, M., Köhler, J., Turchetta, M., Krause, A., and Zeilinger, M. N. (2025). Safe guaranteed exploration for non-linear systems. IEEE Transactions on Automatic Control.\n\nBerkenkamp, F., Turchetta, M., Schoellig, A., and Krause, A. (2017). Safe model-based reinforcement learning with stability guarantees. Advances in neural information processing systems, 30.\n\nTurchetta, M., Berkenkamp, F., and Krause, A. (2019). Safe exploration for interactive machine learning. Advances in Neural Information Processing Systems, 32.\n\nBaumann, D., Marco, A., Turchetta, M., and Trimpe, S. (2021, May). Gosafe: Globally optimal safe robot learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4452-4458). IEEE.\n\nSukhija, B., Turchetta, M., Lindner, D., Krause, A., Trimpe, S., and Baumann, D. (2023). Gosafeopt: Scalable safe exploration for global optimization of dynamical systems. Artificial Intelligence, 320, 103922."}, {"review_id": "PiLeimVqAf", "reviewer": "Reviewer_o5R2", "summary": "To enable safe exploration and learning in control systems subject to noise disturbances, this paper proposes a novel safe learning framework that incorporates three key components: a dynamics model, a safety model, and a reset model. The resulting safe region is theoretically guaranteed by a provable bound.", "questions": "1.It would be helpful if the authors could include a practical case study or provide a clearer explanation of the paper's contributions-particularly how the proposed method differs from existing safe exploration approaches.\n\n2.Does the computation of the safety model introduce significant overhead during data collection? A discussion on the computational cost and efficiency would be beneficial.", "limitations": "Yes, the author discuss limitations, but it will be helpful to include the practical discussion of this safe exploration method in real-world method.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**：\n\n1.This paper is well-written and proposes a novel safe exploration framework to address the challenges posed by the dynamic nature of controlled continuous-time stochastic systems.\n\n2.The theoretical analysis and derivations effectively support the authors' claims.\n\n3.The numerical experiments are solid, and the visualizations clearly illustrate the process of safe region coverage.\n\n**Weaknesses**：\n\n1.Some notations and writing could be improved for clarity. For example, w_i in Equation (5) and subsequent equations appears to represent noise but is not explicitly defined in the paper. Additionally, M_k in Equations (5) and (6) may correspond to T_{\\text{max}} as defined earlier (Line 83), but this should be clarified.\n\n2.The paper would benefit from the inclusion of an algorithm table or pseudocode to enhance clarity and reproducibility. Overall, the presentation could be improved.\n\n3.While the experimental results demonstrate the effectiveness of the proposed method, it would be valuable to include comparisons with existing safe exploration approaches as baseline models to better prove the performance and contribution.", "was_revised": true, "final_justification": "1.This paper is well-written and proposes a novel safe exploration framework to address the challenges posed by the dynamic nature of controlled continuous-time stochastic systems.\n\n2.The theoretical analysis and derivations effectively support the authors' claims.\n\n3.The numerical experiments are solid, and the visualizations clearly illustrate the process of safe region coverage.\n\nSince the authors effectively address my concerns, i will maintain my positive evaluation.", "rebuttal": "Thank you for your detailed and practical comments. We address your points below.\n\n**(General comment).** This work takes a first step toward provably safe learning under unknown stochastic dynamics, a setting largely unexplored compared to prior work assuming known dynamics. Our focus is theoretical and methodological; code will be released for reproducibility. Experiments on more complex problems are part of ongoing work. The Sobolev and RKHS assumptions are standard in learning theory and are commonly used to derive generalization guarantees. While the rates depend on dimensionality, this is inherent to the problem. Sobolev regularity helps reduce this effect and leads to near-optimal rates.\n\n*\"Some notations and writing could be improved...\"*\n\nThank you. We agree and will clarify that $w_i$ indexes the Brownian motion samples and that $M_k$ denotes the number of time steps in trajectory $k$. Allowing $T_k \\leq T_{\\max}$ enables learning controllers that are safe over shorter time horizons, improving flexibility.\n\n*\"The paper would benefit from the inclusion of an algorithm table...\"*\n\nWe will include an algorithm table in the revised version. A well-documented Python library has already been developed and will be released as open source. Appendix B provides a detailed breakdown of each algorithmic step and its computational complexity.\n\n*\"Include comparisons with safe exploration baselines.\"*\n\nWe agree this would be valuable. Nevertheless, most existing approaches assume known dynamics and typically lack formal safety guarantees during exploration. We plan to include such comparisons in future work and welcome any relevant references we may have missed.\n\n*\"Include a practical case study.\"*\n\nThank you for the suggestion. We agree that practical case studies would help illustrate the relevance of our approach. Our work addresses a challenging and, to our knowledge, previously unaddressed setting: learning general unknown dynamics under safety constraints with formal guarantees. While our focus here is on establishing these theoretical foundations, the method is directly applicable to robotics domains where safety and uncertainty are central. For example, flying humanoid robots like iRonCub3 involve highly nonlinear, safety-critical dynamics, and tasks such as cloth folding require learning deformable dynamics safely (e.g., Amadio, F., Delgado-Guerrero, J. A., Colomé, A., and Torras, C. (2023). Controlled Gaussian process dynamical models with application to robotic cloth manipulation. International Journal of Dynamics and Control, 11(6), 3209–3219). We view these as promising directions for future applications.\n\n*\"Does the computation of the safety model introduce overhead?\"*\n\nThe overhead is moderate. The per-iteration cost is $\\mathcal{O}(i^3 + 3Qi)$ (compared to $\\mathcal{O}(i^3 + Qi)$ for standard regression), leading to total complexity $\\mathcal{O}(N^4 + QM^2)$. Under sufficient Sobolev regularity, $Q < N^2$ is enough, making the evaluation cost negligible and the overall complexity comparable to batch kernel ridge regression (unsafe)."}, {"review_id": "sE1fOuQjcN", "reviewer": "Reviewer_gbjw", "summary": "The paper tackles safe learning in controlled stochastic systems that begins with an initial safety-certified state–control set and models the unknown stochastic disturbance in an otherwise deterministic system under smoothness assumptions. \nTo do this, they make the following assumptions:\n- there is an initial known safe set of controls\n- there is an initial \"reset set\" from which the system can be reset to its initial state distribution. For example, a local region around the launchpad for a UAV from which I suppose someone could pick up the UAV and reset it.\n- smoothness of the dynamics wrt time, state, and control\n\nFrom these assumptions, they provide a method for updating a) the dynamics model, b) the safety model (e.g. safe set), and c) a reset model to estimate the probability of returning to the initial state distribution for repeated exploration.  This is all done iteratively based on discrete-time trajectory observations paired with kernel-based confidence bounds. Controls are selected such that the trajectories will reduce uncertainty in the dynamics while, with high probability, maintain safety while ending in the reset region.  This seems to be largely building on Felix Berkenkamp's work on safe exploration, with the addition of stochasticity.", "questions": "1. Could the authors comment on how sensitive exploration is to mis-specification of the sets from Assumptions?\n2. Can this algorithm be applied to higher-dimensional and longer-horizon problems?", "limitations": "No limitations section, but assumptions are made clear.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper tackles an important problem (data-efficient, safety-critical learning) relevant to different applications.\n2. The paper is well-written with a clear presentation.\n3. The assumptions are explicitly articulated and immediately followed by a discussion.\n4. The theorem certifies that every trajectory sampled during training satisfies safety and reset constraints, while also obtaining convergence rates.\n\nWeaknesses:\n1. Evaluation is restricted to a single low-dimensional synthetic environment, drawing questions about the practicality of the approach. The paper does not test higher-dimensional states or longer horizons, nor discuss scalability.\n2. The algorithm requires both an initial set of controls that are guaranteed safe and a separate set that reliably reset the system. These assumptions (especially the latter) may be quite limiting in certain contexts\n3. Ablation study of kernel choice or safety/reset thresholds should be included.", "was_revised": true, "final_justification": "1. This paper tackles an important problem (data-efficient, safety-critical learning) relevant to different applications.\n2. The paper is well-written with a clear presentation.\n3. The assumptions are explicitly articulated and immediately followed by a discussion.\n4. The theorem certifies that every trajectory sampled during training satisfies safety and reset constraints, while also obtaining convergence rates.", "rebuttal": "We appreciate your constructive feedback. We address your points below.\n\n**(General comment).** This work takes a first step toward provably safe learning under unknown stochastic dynamics, a setting largely unexplored compared to prior work assuming known dynamics. Our focus is theoretical and methodological; code will be released for reproducibility. Experiments on more complex problems are part of ongoing work. The Sobolev and RKHS assumptions are standard in learning theory and are commonly used to derive generalization guarantees. While the rates depend on dimensionality, this is inherent to the problem. Sobolev regularity helps reduce this effect and leads to near-optimal rates.\n\n*\"Evaluation is restricted to low-dimensional synthetic environment...\"*\n\nWe agree. Our goal is to provide a statistical analysis of the interaction between learning and safety. Theoretical guarantees in our framework explicitly scale with dimension: the convergence rates for the three estimators are of order $\\mathcal{O}(N^{-\\nu/(m+1+2\\nu)})$, provided the number of samples per trajectory satisfies $Q \\geq \\mathcal{O}(N^{(2\\nu + n)/(2\\nu - n)})$ (up to logarithmic factors). This implies that, under sufficiently smooth dynamics (i.e., large $\\nu$), the curse of dimensionality can be mitigated.\n\nDetails on computational complexity are provided in Appendix B. Briefly, the per-iteration cost is $\\mathcal{O}(i^3 + 3Qi)$ (versus $\\mathcal{O}(i^3 + Qi)$ for standard regression), resulting in total complexity $\\mathcal{O}(N^4 + QN^2)$. Under sufficient Sobolev regularity, $Q < N^2$ is enough, making evaluation cost negligible and overall complexity similar to batch kernel ridge regression (unsafe).\n\n*\"Initial safe/reset sets may be limiting.\"*\n\nWe acknowledge that Assumptions (A1) and (A2) may appear strong, but they are minimal in what they require: only one known safe and one known reset point are sufficient to initiate safe exploration. The sets $S_0$ and $R_0$ can be as small as singletons. Without at least one such point, safe learning cannot begin. In many systems, states of the form $S_0 = \\{(0, \\theta) : \\theta \\in D\\}$ and $R_0 = \\{(0, \\theta) : \\theta \\in D\\}$ are valid and realistic, as systems are often initialized in known, safe and resettable conditions.\n\nWe express the assumptions in general (set-based) form to support more practical and informative choices: our approach leverages non-singleton safe and reset sets to accelerate learning while preserving formal safety guarantees.\n\n*\"Ablation study on kernel and thresholds.\"*\n\nThank you for the suggestion. While we did not include an ablation study in this version, we agree it would offer valuable insights. Our framework supports a broad class of kernels, and the safety/reset thresholds directly govern the trade-off between conservativeness and exploration. We plan to investigate these aspects more systematically in future work.\n\n*\"Sensitivity to mis-specification of the sets in assumptions?\"*\n\nThis is an important point. The algorithm requires at least one correctly specified safe and resettable point to initiate exploration. If this point is misspecified (e.g., not actually safe or resettable), the algorithm may halt immediately, as it cannot proceed without safety guarantees. In this sense, the method is sensitive to gross mis-specification of the initial sets. However, this is standard in safe learning and is typically satisfied in practice by choosing conservative initial conditions, e.g., using $S_0 = \\{(0, \\theta) : \\theta \\in D\\}$ and $R_0 = \\{(0, \\theta) : \\theta \\in D\\}$, which are valid in many settings.\n\n*\"Can this method be applied to higher-dimensional and longer-horizon problems?\"*\n\nYes. While our experiments focus on a 2D setting, the method's statistical and computational performance is theoretically characterized in arbitrary dimensions (see above). In particular, convergence rates depend on both the smoothness of the target functions and the dimensionality, allowing scalability under sufficient regularity. The main computational cost lies in kernel regression, which can be mitigated using standard techniques such as Nyström approximations or random features, without sacrificing theoretical guarantees (See, e.g. Rudi, A., Camoriano, R., and Rosasco, L. (2015). Less is more: Nyström computational regularization. Advances in neural information processing systems, 28)."}, {"review_id": "LyTse2xD1K", "reviewer": "Reviewer_1MU6", "summary": "This paper presents a method for safely learning the dynamics of a nonlinear continuous time system, where safety is defined as high probability constraint satisfaction at each instant and learning is in terms of the state probability density map. Under assumptions on known control inputs for ensuring safety and resets, and on the smoothness of the (unknown) dynamics, the authors provide a method which guarantees safety and learning. The method uses kernel density estimation and ridge regression to estimate safety/reset/state density functions and their uncertainties, and safety constrained uncertainty maximization for sample new controllers.", "questions": "- Does this method suffer from a curse of dimensionality? Would it be feasible to present numerical experiments on systems with higher state dimensions or input parameters?\n- In assumptions A1 and A2, is the requirement for all theta, or there exists a theta? The former is very restrictive but appears at odds with the written text. \n- Can you expand more on how smoothness of dynamics relates to drift and diffusion terms?\n- It is hard to parse the definition of initial safe resettable set, can you explain the role of t and t'?\n- At the beginning of section 4.2, should it be index i instead of N? Or can you explain the role of these two indices?\n- Can you further explain the assumption on N in theorem 5.1? It seems unintuitive to upper bound N by assumption. But maybe its actually better understood as a bound on Q, the number of samples per trajectory?\n- Can you compare the size of the learned safe region to that of the known dynamics? If not theoretically, then at least empirically?\n- Why is the definition of safety meaningful? Shouldn't we care about absolute safety integrated over time, rather than just at each instant probabilistically?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper considers an important and difficult problem, and tackles challenges related to handling safety in continuous time stochastic dynamics.\n2. The method comes with nice guarantees about about safety, accuracy, and sample complexity.\n3. The method's performance is demonstrated in numerical experiments which illustrate the effect of different parameters.\n\nWeaknesses:\n1. There are some important technical details that are not clear, especially in the assumptions (see questions)\n2. It seems that this method would suffer from a curse of dimensionality in the state or control parameter.\n3. The guarantees do not address the size of the feasible set and the definition of safety is somewhat weak.\n4. The text in the figures is too small. Additionally, it would be better to label the plots directly with the value of epsilon rather than in the caption.", "was_revised": true, "final_justification": "Author response addressed several of my concerns.", "rebuttal": "Thank you for engaging with the technical details. Your questions raised important clarifications that helped us improve the overall exposition.\n\n**(General comment).** This work takes a first step toward provably safe learning under unknown stochastic dynamics, a setting largely unexplored compared to prior work assuming known dynamics. Our focus is theoretical and methodological; code will be released for reproducibility. Experiments on more complex problems are part of ongoing work. The Sobolev and RKHS assumptions are standard in learning theory and are commonly used to derive generalization guarantees. While the rates depend on dimensionality, this is inherent to the problem. Sobolev regularity helps reduce this effect and leads to near-optimal rates.\n\n*\"Does this method suffer from a curse of dimensionality? Would it be feasible to present numerical experiments on systems with higher state dimensions or input parameters?\"*\n\nWhile our experiments focus on a 2D setting, the method is theoretically designed to scale to higher-dimensional problems. The curse of dimensionality is fundamental in nonparametric learning, but our approach scales gracefully with smoothness. The convergence rates scale as $\\mathcal{O}(N^{-\\nu/(m+1+2\\nu)})$, where $\\nu$ denotes smoothness and $m$ the input dimension. Thus, the curse of dimensionality can be mitigated under sufficient regularity.\n\nThe main computational cost arises from Gram matrix inversions, which can be reduced using standard approximations such as Nyström or random feature methods, without sacrificing theoretical guarantees. Appendix B provides full complexity analysis. In particular, the computational cost of our safe method is of the same order of magnitude as that of the (unsafe) standard batch kernel ridge regression.\n\nExtending experiments to higher-dimensional systems is a promising and feasible direction for future work.\n\n*\"In assumptions A1 and A2, is the requirement for all theta, or there exists a theta? The former is very restrictive but appears at odds with the written text.\"*\n\nWe acknowledge that Assumptions (A1) and (A2) may appear strong, but they are minimal in what they require: only one known safe and one known reset point are sufficient to initiate safe exploration. The sets $S_0$ and $R_0$ can be as small as singletons. Without at least one such point, safe learning cannot begin. In many systems, states of the form $S_0 = \\{(0, \\theta) : \\theta \\in D\\}$ and $R_0 = \\{(0, \\theta) : \\theta \\in D\\}$ are valid and realistic, as systems are often initialized in known, safe and resettable conditions.\n\nWe express the assumptions in general (set-based) form to support more practical and informative choices: our approach leverages non-singleton safe and reset sets to accelerate learning while preserving formal safety guarantees.\n\n*\"Can you expand more on how smoothness of dynamics relates to drift and diffusion terms?\"*\n\nWhile we do not study this explicitly, Sobolev regularity of the drift and diffusion terms is expected to imply Sobolev regularity of the resulting state densities under standard conditions. This follows from classical results in parabolic PDE theory, where under standard assumptions, solutions gain regularity relative to the coefficients, approximately two derivatives in space and one in time. We will add a comment on this in the revised version of the paper. See Bonalli, R., and Rudi, A. (2025). Non-parametric learning of stochastic differential equations with non-asymptotic fast rates of convergence. Foundations of Computational Mathematics, 1–56, for related results.\n\n*\"It is hard to parse the definition of initial safe resettable set, can you explain the role of t and t'?\"*\n\nIn the definition of $\\Gamma_0$, $t$ denotes the planned sampling time and $T$ the reset time. The condition $(\\theta, t') \\in S_0$ for all $t' \\in [0, T]$ ensures that the system is known to be safe at every intermediate time between sampling and reset. Thus, $t$ indicates where the observation is collected, and the condition on $t'$ ensures safety over the whole trajectory. This guarantees that a sample at $(\\theta, t)$ lies within a trajectory that is entirely safe and ends in a known resettable state.\n\nAs mentioned earlier, $\\Gamma_0$ can be as simple as a singleton, e.g., $\\Gamma_0 = {(\\theta, 0, 0) \\mid \\theta \\in D}$, corresponding to a trajectory of length zero-trivially satisfying the requirement. The general formulation allows the use of larger sets when more prior knowledge is available, enabling faster exploration.\n\n*\"At the beginning of section 4.2, should it be index i instead of N? Or can you explain the role of these two indices?\"*\n\nIn this section, $N$ refers to the iteration index, i.e., the control $(\\theta_N, t_N)$ being evaluated. The superscript in $w_i^N$ indicates that the $i$-th sample trajectory is drawn at iteration $N$ using control $u_{\\theta_N}$. So $i$ indexes the sample trajectories for a fixed control, and $N$ identifies which control they correspond to. We agree this could be clarified in the notation and will improve the explanation in the revision.\n\n*\"Can you further explain the assumption on N in theorem 5.1? It seems unintuitive to upper bound N by assumption. But maybe its actually better understood as a bound on Q, the number of samples per trajectory?\"*\n\nWe agree: the condition is better understood as a lower bound on $Q$ as a function of $N$. It ensures that the number of sample trajectories per control is large enough to guarantee accuracy. The logic is: fix a target accuracy $\\eta$, then determine the required \\$N\\$, and then determine the required \\$Q = Q(N)\\$ to achieve it. We will clarify this in the revised version.\nMore precisely, the assumption requires: $Q \\gtrsim N^{(2\\nu + n)/(2\\nu - n)},$ up to log factors, where $n$ is the state dimension and $\\nu$ is the regularity. This reflects the classical trade-off in kernel estimation and helps quantify how regularity mitigates the curse of dimensionality.\n\n*Can you compare the size of the learned safe region to that of the known dynamics? If not theoretically, then at least empirically?\nWhy is the definition of safety meaningful? Shouldn't we care about absolute safety integrated over time, rather than just at each instant probabilistically?\"*\n\nWhile a theoretical comparison is not provided, Figures 4, 5, and 6 show that the learned safe set expands empirically toward the reachable true region. Looser safety thresholds allow the algorithm to recover a larger portion of the reachable set-i.e., the region accessible without crossing unsafe states or requiring discontinuous jumps.\n\nRegarding the safety definition, we use the quantity\n\n$$\ns^\\infty(\\theta, T) = \\inf_{t \\in [0, T]} \\mathbb{P}(g(X(t)) \\geq 0),\n$$\n\nwhich ensures high-probability safety at each individual time point. While enforcing trajectory-level safety (i.e., $\\mathbb{P}(g(X(t)) \\geq 0 \\ \\forall t \\in [0, T])$) is indeed a desirable goal, it remains fundamentally harder to verify under unknown dynamics. Our pointwise formulation provides a tractable and meaningful notion of safety, especially in the absence of model knowledge.\n\nTo the best of our knowledge, no existing methods for exploration in unknown stochastic systems offer stronger or comparable theoretical guarantees. That said, we would be very interested in identifying and comparing to such approaches and would be glad to include any relevant references.\n\n*\"Figures too small; label plots directly.\"*\n\nWe will revise the figures to improve clarity and annotate plots directly with threshold values."}]

Author rebuttal:

Meta-review: This paper considered the task of learning an unknown controlled SDE subject to a safety constraint involving the state applied pointwise in time. Here, since the system is unknown, in addition to learning the system via discrete-time samples, the safety guarantee itself ultimately depends future evolution of states, a mapping that has to be learned. Under Sobolev continuity assumptions, the main result here is an algorithm that guarantees safety with high probability during training and deployment, while ensuring that the parameters themselves are estimated to arbitrary prespecified accuracies.

This paper has 3 positive reviews (including a clear accept) and a borderline reject. On the negative side, two of the reviewers point out that the synthetic evaluations are limited. However, all reviewers appreciated that the theoretical analysis puts together convergence/sample complexity, performance and safety rigorously in very concrete terms.

Given this, it seems like the upsides outweigh the downsides here. We are happy to recommend this paper for acceptance,
Decision: Accept (poster)