SafeDiffuser: Safe Planning with Diffusion Probabilistic Models

Authors: Wei Xiao, Tsun-Hsuan Wang, Chuang Gan, Ramin Hasani, Mathias Lechner, Daniela Rus
Primary area: generative models
Keywords: Diffusion model, Safety guarantees, Planning and control

Abstract:
Diffusion models have shown promise in data-driven planning. While these planners are commonly employed in applications where decisions are critical, they still lack established safety guarantees. In this paper, we address this limitation by introducing SafeDiffuser, a method to equip diffusion models with safety guarantees via control barrier functions. The key idea of our approach is to embed finite-time diffusion invariance, i.e., a form of specification consisting of safety constraints, into the denoising diffusion procedure. This way we enable data generation under safety constraints. We show that SafeDiffusers maintain the generative performance of diffusion models while also providing robustness in safe data generation. We evaluate our method on a series of tasks, including maze path generation, legged robot locomotion, and 3D space manipulation, and demonstrate the advantages of robustness over vanilla diffusion models.

Decision: accept (poster)
Reviewer scores: 6, 8, 8, 5

Reviews and discussion:
<thread>
creation date:
1730438681083

thread_type:
official_review

<thread_object>
creator: reviewer_gj8x

<content>
summary:
The paper introduces SafeDiffuser, a method to add safety guarantees to diffusion models when used for planning tasks. The key innovation is incorporating control barrier functions (CBFs) into the diffusion process to ensure generated trajectories satisfy safety constraints.

strengths:
The approach maintains the model performance while adding safety guarantees.

weaknesses:
1. The definition of diffusion invariance seems too strict. Essentially we only care about the safety at t=0, and trajectories satisfying t=0 safety don’t necessarily need to satisfy t=1 safety. However the method actively encourages the intermediate trajectories to be safe. This doesn’t mean the method is wrong, but I’m hoping to see a softer way to guarantee the safety. See Q2.
2. Along with the above weakness, the design choice seems too complex. There’re $\epsilon$, $w$, $r$, $\gamma$, $\sigma$, all of them seem hyper-parameters that need careful tuning.
3. CBF itself is meant to design for dynamics that are (1) time-invariant; and (2) deterministic. The diffusion process doesn’t belong to any of these two parts. For (1), the alpha bar changes over time steps. For (2), most of the diffusion sampler constantly adds noise to the dynamics (unless the approach is specifically using DDIM). Applying CBF-based technique to the diffusion seems counter-intuitive. 
4. Scaling up seems challenging. I’m hoping to see a highly nonlinear dynamics like drone/quadcopter to see whether the method could scale up. Another challenge for scaling up is from the selection criteria. Page 6 discusses the principle to select different safediffusers. As mentioned, it requires the unsafe set to be known to determine the convexity. This seems challenging for high-dimensional control tasks.

questions:
1. Regarding to the design choice, the TVS diffuser seems to be the most general safediffuser. Is it possible to further simplify the forward invariance equation (theorem 3.4) by fully discarding the reliance on b, and directly using sigma? For example, imagine sigma at t=0 converges to exactly b. This sigma can be used in analogy to the time-varying CBF and should impose a more relaxed restriction to the intermediate samples. I guess it would need clever design of sigma (or even learning-based), but do you think it is worth the effort?
2. Another question is the motivation of using CBF. From first principle, all we care about here is the safety at the final timestep, ie, t=0. It sounds more similar to a goal-reaching/liveness problem, where you don’t need to enforce every intermediate sample satisfy the safety. Compared to control Lyapunov function or reachability-based method, why is CBF used for the problem? Applying the other two concepts seem more natural to me.
3. In control, CBF can usually be compared to potential-field based method to show their effective in terms of avoiding obstacle in a longer time horizon rather than reactively avoiding. Similarly, is it possible to include [1] as a baseline to show the approach’s effectiveness? 

[1] Luo, Y., Sun, C., Tenenbaum, J. B., & Du, Y. (2024). Potential based diffusion motion planning. arXiv preprint arXiv:2407.06169.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730686335943

thread_type:
official_review

<thread_object>
creator: reviewer_vnzo

<content>
summary:
This paper presents control barrier function-based methods to enforce safety constraints on diffusion models. The main idea is to modify the diffusion denoizing step to satisfy safety constraints within a finite number of steps (finite-time diffusion invariance). This is done by formulating a quadratic program to compute a perturbation that steers the generated trajectory towards the safe set in every denoizing step. Evaluations in simulated maze and robotics domains show that the proposed method is superior to baselines in satisfying safety constraints at the cost of longer planning time.

strengths:
- Diffusion models are an important and effective class of conditional generative models. However, existing methods for guidance do no provide safety guarantees. Hence, the propose algorithm is an important contribution.
    
- Applying CBF-based technique to diffusion models is novel to my knowledge. In particular, modifying the denoizing steps seem to be more effective than generating safe trajectories through classifier-free guidance.
    
- Proposed extension to handle local traps are effective.
    
- Experimental results in simulation demonstrate the effectiveness and robustness of the algorithm.

weaknesses:
- Safety specifications have to be differentiable.
    
- SafeDiffuser is significantly slower (by an order of magnitude) than vanilla diffusion.

questions:
- The forward invariance in control theory (equation 2) assumes that the control system is governed by Lipschitz functions. Is any such requirement not required for extending the definition to diffusion neural networks?
    
- Diffusion dynamics is estimated online. How does error in this estimation affect the guarantees and performance in practice?
    
- In theorem 3.2 what is the intuitive meaning of the robust term?
    
- If would be interesting to continually retrain the diffusion models on the generated safe data. Can this improve performance or speed up safediffuser?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730704891625

thread_type:
official_review

<thread_object>
creator: reviewer_hg4a

<content>
summary:
The authors present SafeDiffuser, a modified version of the Diffuser model proposed by Janner et al. (2022), adapted to ensure the satisfaction of safety constraints. They introduces "Finite-Time Diffusion Invariance," leveraging Control Barrier Functions (CBFs) within the denoising diffusion process to enforce specified safety constraints. The authors propose three approaches for implementing their methodology—ROS, RES, and TVS—that demonstrate competitive performance compared to baseline methods, achieving significantly fewer constraint violations. Experiments were conducted across Maze2D-large, locomotion, and manipulation tasks, validating the effectiveness of the proposed approaches in diverse settings.

strengths:
1. The proposed method introduces a novel formulation of Control Barrier Functions (CBFs) within diffuser-family models, ensuring that safety constraints are satisfied throughout the diffusion denoising process.
2. The authors provide a systematic approach for selecting the appropriate variant based on the safety constraints.
3. Results demonstrate improvements in safety-related metrics while maintaining comparable performance in the downstream task.
4. The method can be trained using data that violates safety constraints, yet during testing, it reliably generates safe trajectories.

weaknesses:
1. The proposed methods require solving Quadratic Programs (QPs) at each diffusion step, which is computationally inefficient.
2. [Minor Weakness] The approach requires predefined safety constraints. However, I think that this limitation is relatively minor, such a setup is not a major weakness.
3. It would strengthen the paper if the authors included comparisons with other methods, such as CDT [1], which also employ sequential modeling techniques to address offline RL problems with safety constraints.

[1] Liu, Zuxin, et al. "Constrained decision transformer for offline safe reinforcement learning." *International Conference on Machine Learning*. PMLR, 2023.

Despite the weaknesses, I find the formulation and results are enough to have the paper above the acceptance threshold.

questions:
1. In terms of data and experimental results, could you clarify the rationale for selecting D4RL over [2], which is specifically designed for Safe Offline RL?
2. Can you clarify the motivation of applying CBFs in the diffusion denoising process steps instead of trajectory planning steps? It was not very clear.
3. Why were the experiments conducted solely in Maze2D-large? Could you clarify the choice of tasks for the locomotion experiments? Specifically, why was only the Medium-Expert variant used for the locomotion environment? What about Cheetah?
4. Could you elaborate on the generalization capabilities of the model when encountering new safety constraints beyond those specified during training? It would be valuable to understand how well the approach adapts to previously unseen constraints.


[2] Liu, Zuxin, et al. "Datasets and benchmarks for offline safe reinforcement learning." arXiv preprint arXiv:2306.09303 (2023).
</content>
</thread_object>

</thread>
<thread>
creation date:
1731020422381

thread_type:
official_review

<thread_object>
creator: reviewer_4uns

<content>
summary:
The authors propose a method to use control barrier functions (CBFs) with diffusion planning, enabling safety guarantees. The results are demonstrated on a 2D maze problem, Walker2D and a manipulation problem from related work.

strengths:
- Safety guarantees are important for some applications and of interest to the safe planning community
- This may be the first (only?) safety method for diffusion planning.

weaknesses:
- Some parts are a bit difficult to read and have unusual terminology (e.g., "local traps"?) and the paper contains some minor grammar and cosmetic issues which make the quite complex approach harder to understand.
- The method is actually three methods, and the user is to pick the one most suitable for their problem. This seems a bit complicated, why not go for the most general?

Example presentation issues (I recommend you do another polish pass):
- General: excessive underlining
- 470: typo: physical simulator
- 474: typo: can work in general safety guarantees

questions:
1) Are "local traps" = local minima?
2) Isn't it a big problem that the method gets stuck in "local traps" between 39% and 100% of the time for some of the benchmarks? What does this mean in practice?
3) What is trap rate 1 vs. trap rate 2 in the table?
4) Could you comment on why you chose to use CBF here instead of e.g. lagrangian objectives common in safe RL?
5) Why three methods instead of the most general, or one that picks the best solution approach?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732126320356

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of our response to address reviewers' main concerns

comment:
We appreciate all the reviewers for their constructive comments. We have added extensive experiments to fully address the limitations of our SafeDiffusers, and the paper (PDF) has also been updated accordingly.  Here we summarize our response to their main concerns.

**1. New experiments of closed-form solution to SafeDiffusers [Reviewers hg4A, vnZo, gj8X].**  We have fully addressed the computational issue by finding their closed-form solutions.  **The closed-form solution is given in Appendix section B of the revised manuscript**. We have rerun all the experiments using the closed-form solution of safediffusers, which shows that the computation time is almost the same as the original diffuser (given in the following three tables).

Maze planning (CF is short for closed-form solution):
|  method |  S-spec($\geq0$)  | C-spec($\geq0$) | Score ($\uparrow$)  | Time ($\downarrow$) | NLL|  Trap rate 1| Trap rate 2|
|---|---|---|---|---|---|---|---|
| Diffuser |-0.983 | -0.894 |$1.598{\small\pm 0.174}$ | ${\color{red}0.006}$| 4.501| | |
| Trunc |$-1.192e^{-7}$ | -0.759 |$1.577{\small\pm 0.242}$ | 0.024| 4.494| | |
| CG |-0.789 | -0.979 |$0.384{\small\pm 0.020}$ | 0.053| 6.962| | |
| CG-$\epsilon$ |-0.853 | -0.995 |$0.383{\small\pm 0.017}$ | 0.061| 6.975| | |
| InvODE|14.000 | $1.657e^{-5}$ |$-0.025{\small\pm 0.000}$ | 0.018| - | | |
| RoS-diffuser (ours)|0.010 | 0.010 |$1.519{\small\pm 0.330}$ | 0.106| 4.584|$100\%$ | $100\%$|
| RoS-diffuser-CF (ours)|0.010 | 0.010 |$1.536{\small\pm 0.306}$ | ${\color{purple}0.007}$| 4.481|$100\%$ | $100\%$|
| ReS-diffuser (ours)|0.010 | 0.010 |$1.557{\small\pm 0.289}$ | 0.107| 4.434|$46\%$ | $17\%$|
| ReS-diffuser-CF (ours)|0.010 | 0.010 |$1.544{\small\pm 0.280}$ | ${\color{purple}0.007}$| 4.619|$36\%$ | $16\%$|
| TVS-diffuser (ours)|0.003 | 0.003 |$1.543{\small\pm 0.303}$ | 0.107| 4.533|$47\%$ | $21\%$|
| TVS-diffuser-CF (ours)|0.003 | 0.003 |$1.588{\small\pm 0.231}$ | ${\color{purple}0.007}$| 4.462|$48\%$ | $18\%$|
| ReS-diffuser-l10 (ours)|0.010 | 0.010 |$1.527{\small\pm 0.291}$ | 0.011| 4.571|$39\%$ | $8\%$|


Locomotion (CF is short for closed-form solution):
|Experiment|  method |  S-spec($\geq0$)  | C-spec($\geq0$) | Score ($\uparrow$)  | Time($\downarrow$) |
|---|---|---|---|---|---|
| | Diffuser |-9.375 | -4.891 |$0.346{\small\pm 0.106}$ | ${\color{red}0.037}$|
| | Trunc |0.0 | $\times$ |$0.286{\small\pm 0.180}$ | 0.105|
|Walker2D| CG |-0.575 | -0.326 |$0.208{\small\pm 0.140}$ | 0.053| 
| | RoS-diffuser (ours)|0.000 | 0.010 |$0.312{\small\pm 0.165}$ | 0.183|
| | RoS-diffuser-CF (ours)|0.000 | 0.010 |$0.321{\small\pm 0.119}$ | ${\color{purple}0.040}$|
|---|---|---|---|---|---|
| | Diffuser |-2.180 | -1.862 |$0.455{\small\pm 0.038}$ | ${\color{red}0.038}$|
| | Trunc |0.0 | $\times$ |$0.436{\small\pm 0.067}$ | 0.046|
|Hopper| CG |-0.894 | -0.524 |$0.478{\small\pm 0.038}$ | 0.047| 
| | RoS-diffuser (ours)|0.000 | 0.010 |$0.430{\small\pm 0.040}$ | 0.170|
| | RoS-diffuser-CF (ours)|0.000 | 0.010 |$0.464{\small\pm 0.028}$ | ${\color{purple}0.040}$|

Manipulation (CF is short for closed-form solution):
|  method |  S-spec($\geq0$)  | C-spec($\geq0$) | Score ($\uparrow$)  | Time($\downarrow$) |
|---|---|---|---|---|
| Diffuser |-0.057 | -0.065 |$0.650{\small\pm 0.107}$ | ${\color{red}0.038}$|
| Trunc |$1.631e^{-8}$ | $\times$ |$0.575{\small\pm 0.112}$ | 0.069|
| CG |-0.050 | -0.053 |$0.800{\small\pm 0.328}$ | 0.075| 
| RoS-diffuser (ours)|0.072 | 0.069 |$0.925{\small\pm 0.107}$ | 0.088|
| RoS-diffuser-CF (ours)|0.093 | 0.002 |$0.800{\small\pm 0.114}$ | ${\color{purple}0.039}$|


**2. Comparison to more baselines to explain why we propose SafeDiffusers instead of using other methods (potential-based, reachability, Lagrange-based RL) [Reviewers 4UNS, gj8X]**
We have added new experiments to compare the potential-field based method and our methods (see the table below). In summary, potential-field based/existing methods have no strict safety guarantees (see Fig. 5a in ``Luo, Y., Sun, C., Tenenbaum, J. B., & Du, Y. (2024). Potential based diffusion motion planning. arXiv preprint arXiv:2407.06169'' showing there are still collisions with potential-based methods), while our methods do have guarantees. 

Maze planning:
|  method |  S-spec($\geq0$)  | C-spec($\geq0$) | Score ($\uparrow$)  | Time ($\downarrow$) | NLL| 
|---|---|---|---|---|---|
| Potential-based diffuser |-0.798 | -0.908 |$0.916{\small\pm 0.556}$ | 0.015| 5.289|
| RoS-diffuser-CF (ours)|0.010 | 0.010 |$1.536{\small\pm 0.306}$ | $0.007$| 4.481|
| ReS-diffuser-CF (ours)|0.010 | 0.010 |$1.544{\small\pm 0.280}$ | $0.007$| 4.619|
| TVS-diffuser-CF (ours)|0.003 | 0.003 |$1.588{\small\pm 0.231}$ | $0.007$| 4.462|

**We hope our responses below convincingly address all the reviewers’ concerns. We thank all the reviewers for their time and efforts again.**
</content>
</thread_object>

</thread>
<thread>
creation date:
1733252586242

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Rebuttal and Paper Revision

comment:
We thank all the reviewers and AC for reviewing our paper, and for being active in the rebuttal phase. We have revised the paper accordingly.  Here, we summarize our rebuttal and revision below:

 1. We have added the closed-form solution of SafeDiffusers (Appendix Section B of the revision) to fully address the computational concerns, and we have rerun all the experiments to show the effectiveness of the closed-form solution (reviewers hg4A and vnZo).

2. We have added a comparative study between the proposed SafeDiffusers and potential-based diffusion models, with results showing the advantage of our methods in safety guarantees (reviewers hg4A and gj8X).

3. We have added a formal definition of the local trap (Def. 3.2 in the revision) in navigation problems, which makes the concept clear (reviewer 4UNS).

4. We have fully clarified the Lipschitz problem for the diffusion model by referring to existing literatures, and revised the paper accordingly (after equation (5) of the revision) (reviewer vnZo).

5. We have added a unified and simple definition of the proposed SafeDiffusers (the general form of SafeDiffusers, as shown in Def. 3.3 of the revision), and have made connections to the three SafeDiffusers after Thms. 3.4-3.6 (reviewers 4UNS and gj8X).

All the reviewers have responded that we have addressed their concerns indicated by raising their scores in the rebuttal phase. We appreciate all the reviewers and AC again for taking the time in reviewing our paper.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes SafeDiffuser, a framework that integrates control barrier functions (CBFs) into diffusion processes to ensure the safety of diffusion-based planners. The model operates by using a quadratic program during each step of the denoising process to identify a perturbation that guides the generated trajectory towards the safety set. The paper evaluates SafeDiffuser alongside contemporary baselines in terms of task performance and adherence to safety constraints on various robot planning tasks.

The paper was reviewed by four referees, who agree on several of the paper's key strengths and weaknesses. Among its strengths, the ability to ensure that diffusion-based planners adhere to safety constraints becomes increasingly important as more and more people use diffusion models for planning. Further, at least two reviewers note the novelty in the paper's integration of CBFs within diffusion models. The experiments support SafeDiffuser's ability to ensure the safety of the planned trajectories, while maintaining performance that is comparable to traditional diffusion-based planners. However, several reviewers raised concerns about SafeDiffuser's computational complexity due to the need to solve a quadratic program at each diffusion step. Notably, the authors derived a closed-form solution of the model and re-ran all of the experiments, the result of which reveals that the computational cost is not significantly higher than that of the vanilla Diffuser model. Additionally, several reviewers called into question the difficulty of implementing SafeDiffuser due to the need to choose among the three variants of the model as well as the need to design the CBFs, which can be particularly challenging for systems with highly nonlinear dynamics. The authors addressed some of these concerns during the rebuttal period, proposing a unified version of SafeDiffuser and connected it to the three variants proposed in the original submission. Additionally, at least two reviewers questioned the validity of the assumption that the diffusion process is Lipschitz, which the authors clarified in their responses.

Overall, the authors put significant effort into revising the paper to address the reviewers' initial concerns. It is evident to the AC that this effort went a log way in clarifying the paper's contributions.