Title: Safe and Stable Control via Lyapunov-Guided Diffusion Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: safe and stable control

diffusion for control

Lyapunov-guided sampling
Abstract: Diffusion models have made significant strides in recent years, exhibiting strong generalization capabilities in planning and control tasks. However, most diffusion-based policies remain focused on reward maximization or cost minimization, often overlooking critical aspects of safety and stability. In this work, we propose Safe and Stable Diffusion ($S^2$Diff), a model-based framework that explores how diffusion models can ensure safety and stability from a Lyapunov perspective. We demonstrate that $S^2$Diff eliminates the reliance on both complex gradient-based solvers (e.g., quadratic programming, non-convex solvers) and control-affine structures, leading to globally valid control policies driven by the learned certificate functions. Additionally, we uncover intrinsic connections between diffusion sampling and almost Lyapunov theory, enabling the use of trajectory-level control policies to learn better certificate functions for safety and stability guarantees. To validate our approach, we conduct experiments on a wide variety of dynamical control systems, where $S^2$Diff consistently outperforms both certificate-based controllers and model-based diffusion baselines in terms of safety, stability, and overall control performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the $S^2$ framework, which combines diffusion policies with Control Lyapunov Barrier Functions (CLBFs) to ensure safety and stability in control tasks. For this purpose, the authors propose an iterative training procedure that alternates between training a diffusion policy and a CLBF. The diffusion policy is trained to generate actions that are safe and stable, while the CLBF is trained to ensure that it captures accurately the safety and stability constraints of the system. The paper demonstrates the effectiveness of the $S^2$ framework through experiments on various control tasks (e.g., Inverted Pendulum, Car, Segway, Neural Lander, and F-16) and against various baselines, including robust CLBF-QP, MPC, Model-Based Diffusion, showing that it outperforms existing methods in terms of stability/convergence behavior and safety at a greater computational efficiency than MPC and Model-Based Diffusion.\nAn additional advantage of the proposed method compared to MPC and rCLBF-QP is that it also admits to non-control-affine systems, as demonstrated in the experiments on the F-16 aircraft.\nFinally, the paper connects the proposed framework to Almost Lyapunov theory, showing that the policy almost-always guarantees convergence to the desired equilibrium point with exponential convergence rate.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- The paper is a timely contribution to the field of safe and stable control by integrating safety and stability guarantees into state-of-the-art diffusion policies.\n- The proposed $S^2$ framework is an interesting, novel, and valuable contribution and offers interesting theoretical and technical insights into the integration of diffusion models with control theory.\n- The proposed framework is comprehensively quantitatvely evaluated on a variety of control tasks and against various relevant baselines, demonstrating its effectiveness in terms of safety, stability, and computational efficiency.\n\n## Weaknesses\n\n- The current manuscript fails to clearly and in detail define the overall training procedure that consists of this iterative training procedure alternating between training the diffusion policy and the CLBF. For example, it is not clear according to which objectives the diffusion policy is trained in order to ensure adherence to the CLBF constraints. The reviewer recommends a clear and easily understandable description of the overall training procedure and the algorithm in the main text, before each of the components are described in detail. Also, the manuscript is missing illustrations and figures that illustrate the methodology and the training procedure.\n- The quantitative evaluation is lacking in some aspects: (1) unconstrained diffusion policies should be included as a baseline, (2) different configurations of MPC should be included as a baseline (i.e, some with short, medium, and long horizons) in order to demonstrate that long-horizon MPC can achieve high safety rates, (3) adopting SafeDiffuser as a baseline would be interesting, (4) arguably **most importantly**, an evaluation metric for the task performance is missing: while the paper evaluates the convergence/stability of the methods, the safety rate, and the computational efficiency, it does not evaluate the task performance of the methods (i.e., reward, cost to goal, etc.).\n- The paper shows the connections to Almost Lyapunov theory. However, the paper does not take the next step of actually applying Theorem 3.1 to practical examples. For the considered tasks, what is the epsilon value? Does this epsilon value together with the learned Lyapunov function satisfy the conditions/asssumptions? Can the authors conduct a post-training analysis of the learned CLBF function and how it connects to the Almost Lyapunov theory? This would be a valuable addition to the paper.\n\n### Detailed Comments on Minor Issues\n\n- Figure 1: The authors introduce a color map from \"violation to satisfaction\". However, it is using the same colormap as for the \"update Lyapunov function\". This is confusing and unclear. Please revise.\n- Page 2, line 85: \"... the stability properties of such methods are still unexplored\". The reviewer disagrees with this statement as CBF+CLF approaches guarantee simultatenously safety and stability. Please clarify this statement.\n- Section 2 (Preliminary) is missing a clear structure. The reviewer suggests to carefully reflect about the order and structure of this section and to possibly split it into subsections.\n- Section 2 \"Challenges in Gradient-Based Methods\": Despite the subsection title, many of the concepts discussed in this section (e.g., safety filters via CBFS, CLBFs, etc.) are not fundamentally connected to gradient-based methods but instead rather formulations that allow ensuring safety via the introduction of dynamics-based constraints. Furthermore, the referenced MPC setting could also be solved using sampling-based techniques (e.g., MPPI). Please clarify this and revise the structure.\n- Page 3, line 110: \"Traditionally, cost minimization problems for control policies are solved using LQR or MPC approaches, often without incoporating any constraints.\". The reviewer disagrees with this statement as a vanilla, safety-unaware MPC formulation would at least consider the dynamics of the system as an equality constraint. Please clarify this statement.\n- Section 3.1: You could stress here that you are proving \"**Exponential** Almost Lyapunov Stability\" and not just \"Almost Lyapunov Stability\". This is a strong result and should be highlighted.\n- Everywhere: Inconsistent capitalization of \"Almost Lyapunov theory\".\n- Table 1 has some deficiencies: (1) \"Safe and Stable\" should be two separate columns, (2) why does MPC not ensure safety and stability? This can be done via constraints - at least in the horizon? (3) to the best of the reviewer's knowledge, MPC can also be used for non-control-affine systems, although this will be more difficult and computationally expensive. See for example the Embotech Forces Pro solver documentation.\n- Equation (3): Add some underbraces what is the meaning of each of the constraints.\n- Section 4.1: Please include a concise overview over the characteristics of the tasks (e.g., control-affine, presence of safety constraints, definition of the equilibrium point, etc.).\n- Table 2: Add references for the baseline methods.\n- Figure 4: What do you encounter unsafe trajectories for both rCLBF-QP method and for the MPC methods? Is it because of the slack variables and the short horizons, respectively? In the reviewer's opinion, you should be able to avoid such unsafe trajectories for CLBF-QP-based and MPC-based methods with appropriate hyperparameter tuning.\n- Appendix D3.1: The reviewer suggests to be more explicit pointing out that most of the systems/tasks are control-affine and that in some tasks (e.g., Car) safety constraints are not established as the safe set is the whole state space.\n\nQUESTIONS:\n- How is the diffusion policy trained to ensure adherence to the CLBF constraints? Furthermore, how many iterations alternating between training the diffusion policy and the CLBF are performed? Is there a stopping criterion for this iterative training procedure? Finally, the reviewer would like to see an ablation study that compares the iterative training procedure with a joint training procedure of the diffusion policy and the CLBF. How do you sample the trajectories, which you mention in Section 3.2? What are the initial conditions?\n- Could you plot the results of Table 3, preferably with a smaller discretization, in a Pareto fron style plot, to better visualize the trade-off between safety and stability/convergence? Furthermore, why is the best convergence rate achieved for an intermediate temperature value? The reviewer would expect the convergence characteristics to improve with a lower temperature. Could this possibly point to issues in the evaluation procedure and/or the definition of the evaluation metric for convergence?\n- Re Table 4: Why does adding a term based on the discrete approximation of the Lie derivative actually improve the performance compared to the sole reliance on the analytical Lie derivative? This is unintuitive. Do the authors have an explanation for this?\n- Re Results Tables and Appendix F: The selection of the discretization and the horizon for MPC seems arbitrary and might partly explain why the MPC is not able to accomplish high/perfect safety rates. Therefore, the reviewer would like to see different MPC configurations reported in the results tables (e.g., MPCs with short, medium and long horizons). One would expect the short-horizon MPC to be computationally more efficient but to achieve lower safety rates, while the long-horizon MPC to be computationally less efficient but to achieve higher safety rates.\n\nThe questions touch on the presentation and the quantitative evaluation in the paper. The reviewer expects that a satisfactory answer to these questions will be provided in the rebuttal and that the paper score could be adjusted accordingly.\n\nLIMITATIONS:\nThe paper mentions a single limitation at the end of the conclusion regarding its slower inference speed compared to QP methods. However, the reviewer imagines that there must be other limitations to this method and the authors should transparently discuss those in the main text and possibly more extensively in the appendix.\n\n- For example, another limitation is that the proposed method does not formally guarantee convergence in all situations, but rather \"just\" almost-always guarantees.\n- Related to the previous point, the reviewer understands that the \"almost lypunov\" guarantees only hold if the loss function actually converges to zero. Is this indeed the case? Please clarify this in the main text - preferably also in the introduction and the conclusion.\n- In the introduction, you claim that \"However, designing valid CBFs priors remains a challenge.\". Indeed, while the proposed approach learns the CBFs, it still requires a manual definition of the safe state set, which should be explicitly discussed in the main text.\n- Finally, another limitation is that the proposed method requires knowledge about the system dynamics, which is not always (accurately) available in practice.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a control algorithm for Lipschitz-continuous systems that combines a learned Lyapunov function with guided sampling. It provides general safety and stability theorems and evaluates the approach on several benchmark problems.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The formulation applies to a broad class of systems.\n- Systematic empirical evaluation on challenging simulation tasks.\n- Clear presentation.\n\n**Weaknesses:**\n\n- Gap between theory and implementation:\n\nThe link between Theorem 3.1 and the learned Lyapunov function is unclear. Is the theorem merely an existence result, or can it yield explicit guarantees for the algorithm via the results in C.1? The paper should state this connection and clearly state what assumptions are necessary to apply Theorem 3.1 to the output of algorithm.\n\n- Insufficient method details:\n\nThe training loop for the CLBF and diffusion policy is under-specified. Do you retrain the CLBF after every denoised trajectory, on a growing dataset, or some other schedule? A step-by-step description (e.g., concise pseudocode) of how data are collected, how the CLBF is updated, and how policies are evaluated would make the method reproducible and easier to assess.\n\nQUESTIONS:\n**Questions:**\nI list here the questions that result from the weakness section above:\n\n- What is the connection of Theorem 3.1 with the learned Lyapunov function?\n- Is Theorem 3.1 only an existence result?\n- Can we make explicit statements about the algorithm using the results in Appendix C.1?\n- What types of results does the paper actually provide, which results are missing, and what additional assumptions would be needed to make explicit statements about the learned policy?\n- Is the CLBF trained during the diffusion process?\n- Do we retrain a new CLBF every time we denoise a trajectory?\n- Or is the CLBF trained on a separate dataset collected from past trajectories? If so, how big are the datasets used for the results in Section 4?\n\nLIMITATIONS:\nSee weaknesses and questions", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-27"}

comments: SUMMARY:
This paper proposes a diffusion-based policy which can incorporate safety and stability into the training objective.

STRENGTHS AND WEAKNESSES:
Strengths
- This paper proposes an interesting and novel approach for certifying safety and stability for systems with non-affine dynamics. 
- Overall, the text is very clear and easy to follow.
- Equations are motivated well and are followed by a clear description.

Weaknesses
- Figure 1 is quite small and difficult to understand. In particular, it is difficult to interpret the difference between violation and satisfaction. 
- The methods/results section is missing information about how the diffusion model is trained and how many denoising steps are required.
- There is no concrete algorithm provided that the audience can follow to easily implement the approach proposed in this paper.
- It is difficult to interpret the surface plot shown in Figure 6 as the colour map makes the surface look like a coloured silhouette.

QUESTIONS:
- What is $\alpha_k$ on line 174?
- What do $U^0$ and $U^i$ represent in Equation 9?
- Why is the distance between the fixed horizon terminal state and the equilibrium state much larger for the F-16 task?
- What do the grey boxes represent in Table 2?
- What does the color scale in Figure 3 represent, and should each environment have the same scale?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to modify the trajectory sampling in diffusion models by considering both safety and stability factors. It leverages Almost Lyapnunov Function (ALF) to incorporate safety and stability constraints into the sampling probability. So in the diffusion sampling phase, the score function is updated to the joint probability between cost, safety and stability. The paper analyses the proposed ALF and its relaxations and evaluates such formation in benchmark control tasks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\n**Quality**: The paper presents a comprehensive analysis, including both theory and empirical evaluation,  of Almost Lyapnunov Function and its applications in modelling safety and stability constraints. The empirical results are complete and convincingly shows that the proposed modification to the score function generates target trajectory distribution. The method reads solid. \n\n### Weakness\nFrom my perspective and understanding, **the main weaknesses for this paper in its current version is the clarity in technical details, especially on the diffusion sampling, and originality in modifying the probability distribution** (maybe my understanding was wrong?). \n\n* The diffusion sampling, as one of the main components in the method, is a bit hard to understanding. The paper sometimes refers to this sampling as a way to generate “control policies” and sometimes means the sampled trajectories. Since $U^0$ is the original trajectory distribution, it’s not clear how to generate control policies. Furthermore, the score function is written as $\\nabla\\log p(U)$, not even in the product form as specified in Eq. (6), or in the approximate as specified in Eq. (7). \n\n* Re the originality, the proposed target distribution and the later-on formulation is similar to the classifier guidance formulation (only differs in the classifier guidance paper: there is no classifier but a likelihood model, which makes the formulation a bit even easier). My understanding is that the main idea proposed by this paper is to modify directly the score function by incorporating the safety and stability constraints (which is defined by ALF but then relaxed to soft constraint). This can be achieve by modifying the diffusion sampling directly, even without the ALF (please correct me if wrong). So the baselines should at least include such naïve modification of the score function by directly modifying it via the safety and stability constraints (e.g., soft regularization).\n\nQUESTIONS:\n1. In ALF Guidance (line 157), it’s a bit unclear how the soft constraint is used as guidance. Does the paper mean that soft constraint is used as a guidance in the diffusion sampling? \n2. In line 172, if Monte Carlo score ascent is adopted within diffusion sampling to iteratively denoise trajectories toward the target distribution, then why the forward process is required in Eq. (9)? \n3. What’s $p(U^0|U^i)$ in Eq. (10)? \n4. Line 180, “After generating control policies through diffusion sampling”. In what way are the control policies generated? The diffusion sampling only gives the trajectory samples. There is no control policy generated through diffusion sampling. \n5. it’s unclear how Eq. (11) is optimized. \n6. In abstract, “uncover intrinsic connections between diffusion sampling and almost Lyapunov theory”. Can the authors explain what are the intrinsic connections between diffusion sampling and almost Lyapunov theory?\n\nLIMITATIONS:\nno. the paper should discuss the limitations of using the relaxed constraints in Eq. (11) and whether it's possible to learn $\\hat{V}$ that minimizes the objective in Eq. (11)", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-07-02"}