Title: Safely Learning Controlled Stochastic Dynamics
Venue: NeurIPS 2025
Accepted: yes
Keywords: Kernel Methods

Dynamical Systems

Exploration

Density Estimation

Learning Theory
Abstract: We address the problem of safely learning controlled stochastic dynamics from discrete-time trajectory observations, ensuring system trajectories remain within predefined safe regions during both training and deployment. Safety-critical constraints of this kind are crucial in applications such as autonomous robotics, finance, and biomedicine. We introduce a method that ensures safe exploration and efficient estimation of system dynamics by iteratively expanding an initial known safe control set using kernel-based confidence bounds. After training, the learned model enables predictions of the system's dynamics and permits safety verification of any given control. Our approach requires only mild smoothness assumptions and access to an initial safe control set, enabling broad applicability to complex real-world systems. We provide theoretical guarantees for safety and derive adaptive learning rates that improve with increasing Sobolev regularity of the true dynamics. Experimental evaluations demonstrate the practical effectiveness of our method in terms of safety, estimation accuracy, and computational efficiency.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for safely learning the dynamics of a nonlinear continuous time system, where safety is defined as high probability constraint satisfaction at each instant and learning is in terms of the state probability density map. Under assumptions on known control inputs for ensuring safety and resets, and on the smoothness of the (unknown) dynamics, the authors provide a method which guarantees safety and learning. The method uses kernel density estimation and ridge regression to estimate safety/reset/state density functions and their uncertainties, and safety constrained uncertainty maximization for sample new controllers.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper considers an important and difficult problem, and tackles challenges related to handling safety in continuous time stochastic dynamics.\n2. The method comes with nice guarantees about about safety, accuracy, and sample complexity.\n3. The method's performance is demonstrated in numerical experiments which illustrate the effect of different parameters.\n\nWeaknesses:\n1. There are some important technical details that are not clear, especially in the assumptions (see questions)\n2. It seems that this method would suffer from a curse of dimensionality in the state or control parameter.\n3. The guarantees do not address the size of the feasible set and the definition of safety is somewhat weak.\n4. The text in the figures is too small. Additionally, it would be better to label the plots directly with the value of epsilon rather than in the caption.\n\nQUESTIONS:\n- Does this method suffer from a curse of dimensionality? Would it be feasible to present numerical experiments on systems with higher state dimensions or input parameters?\n- In assumptions A1 and A2, is the requirement for all theta, or there exists a theta? The former is very restrictive but appears at odds with the written text. \n- Can you expand more on how smoothness of dynamics relates to drift and diffusion terms?\n- It is hard to parse the definition of initial safe resettable set, can you explain the role of t and t'?\n- At the beginning of section 4.2, should it be index i instead of N? Or can you explain the role of these two indices?\n- Can you further explain the assumption on N in theorem 5.1? It seems unintuitive to upper bound N by assumption. But maybe its actually better understood as a bound on Q, the number of samples per trajectory?\n- Can you compare the size of the learned safe region to that of the known dynamics? If not theoretically, then at least empirically?\n- Why is the definition of safety meaningful? Shouldn't we care about absolute safety integrated over time, rather than just at each instant probabilistically?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles safe learning in controlled stochastic systems that begins with an initial safety-certified state–control set and models the unknown stochastic disturbance in an otherwise deterministic system under smoothness assumptions. \nTo do this, they make the following assumptions:\n- there is an initial known safe set of controls\n- there is an initial \"reset set\" from which the system can be reset to its initial state distribution. For example, a local region around the launchpad for a UAV from which I suppose someone could pick up the UAV and reset it.\n- smoothness of the dynamics wrt time, state, and control\n\nFrom these assumptions, they provide a method for updating a) the dynamics model, b) the safety model (e.g. safe set), and c) a reset model to estimate the probability of returning to the initial state distribution for repeated exploration.  This is all done iteratively based on discrete-time trajectory observations paired with kernel-based confidence bounds. Controls are selected such that the trajectories will reduce uncertainty in the dynamics while, with high probability, maintain safety while ending in the reset region.  This seems to be largely building on Felix Berkenkamp's work on safe exploration, with the addition of stochasticity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper tackles an important problem (data-efficient, safety-critical learning) relevant to different applications.\n2. The paper is well-written with a clear presentation.\n3. The assumptions are explicitly articulated and immediately followed by a discussion.\n4. The theorem certifies that every trajectory sampled during training satisfies safety and reset constraints, while also obtaining convergence rates.\n\nWeaknesses:\n1. Evaluation is restricted to a single low-dimensional synthetic environment, drawing questions about the practicality of the approach. The paper does not test higher-dimensional states or longer horizons, nor discuss scalability.\n2. The algorithm requires both an initial set of controls that are guaranteed safe and a separate set that reliably reset the system. These assumptions (especially the latter) may be quite limiting in certain contexts\n3. Ablation study of kernel choice or safety/reset thresholds should be included.\n\nQUESTIONS:\n1. Could the authors comment on how sensitive exploration is to mis-specification of the sets from Assumptions?\n2. Can this algorithm be applied to higher-dimensional and longer-horizon problems?\n\nLIMITATIONS:\nNo limitations section, but assumptions are made clear.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo enable safe exploration and learning in control systems subject to noise disturbances, this paper proposes a novel safe learning framework that incorporates three key components: a dynamics model, a safety model, and a reset model. The resulting safe region is theoretically guaranteed by a provable bound.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**：\n\n1.This paper is well-written and proposes a novel safe exploration framework to address the challenges posed by the dynamic nature of controlled continuous-time stochastic systems.\n\n2.The theoretical analysis and derivations effectively support the authors' claims.\n\n3.The numerical experiments are solid, and the visualizations clearly illustrate the process of safe region coverage.\n\n**Weaknesses**：\n\n1.Some notations and writing could be improved for clarity. For example, w_i in Equation (5) and subsequent equations appears to represent noise but is not explicitly defined in the paper. Additionally, M_k in Equations (5) and (6) may correspond to T_{\\text{max}} as defined earlier (Line 83), but this should be clarified.\n\n2.The paper would benefit from the inclusion of an algorithm table or pseudocode to enhance clarity and reproducibility. Overall, the presentation could be improved.\n\n3.While the experimental results demonstrate the effectiveness of the proposed method, it would be valuable to include comparisons with existing safe exploration approaches as baseline models to better prove the performance and contribution.\n\nQUESTIONS:\n1.It would be helpful if the authors could include a practical case study or provide a clearer explanation of the paper's contributions-particularly how the proposed method differs from existing safe exploration approaches.\n\n2.Does the computation of the safety model introduce significant overhead during data collection? A discussion on the computational cost and efficiency would be beneficial.\n\nLIMITATIONS:\nYes, the author discuss limitations, but it will be helpful to include the practical discussion of this safe exploration method in real-world method.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for learning the dynamics of controlled stochastic systems from trajectory data in the presense of safety constraints, and provides safety guarantees during training and deployment. The method is based on expanding a safe control set using kernel-based confidence bounds. Also, adaptive learning rates that improve with the smoothness are derived. The method is validated on a 2D second order dynamical system.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThere is a solid theoretical analysis which provides safety guarantees and adaptive convergence rates under Sobolev regularity assumptions. \nThe paper is generally well-written with clear problem motivation and clear description of the approach. \nThe addressed problem in control of safety-critical systems is important with applications in robotics, autonomous systems, and other domains.\nThe paper extends safe UCB methods to continuous-time stochastic differential equations and provides joint (probabilistic) guarantees for dynamics estimation and safety.\n\nWeaknesses\n\nThe method relies on assumptions as Sobolev regularity, RKHS membership, eventually known safe/reset sets, which might be limiting.\nThe validation of the method is very limited (a 2D toy example) and does not show or address the scalability of the method to more complex systems or higher dimensionality, especially in terms of computational complexity.\nThe comparison with other safe learning approaches is limited. It is unclear what is the impact of the reset in more realistic applications.\n\nQUESTIONS:\nIf I understand it correctly, the method requires the upper bounds ||s||_G, ||r||_G, which are not obviously available (as also bounds of RKHS norms).\nTo increase the evaluation, it would be good to see how can these bounds be non conservatively estimated for real systems or system models? Does that impact the practical use of the method?  How sensitive are the safety guarantees to misspecification of  ||s||_G, ||r||G, and ||p||? Can you demonstrate similar performance with large underestimation of the bounds?\nHow are the hyparparameters β^s, β^r, λ, γ, R tuned?\nAnother suggestion for more convincing demonstration is to increase the complexity and dimensionality of the proposed numerical example and show how the method works in a scenario closer to reality.\nAlso, it would help to understand how the method performs with a comparison to safe learning control methods like model-based safe RL and control barrier functions, or more baseline method comparisons.\n\nLIMITATIONS:\n- the method relies on known bounds, and it is not specified how to estimate them or how sensitive the guarantees are to misspecification\n- the numerical simulation example is limited", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}