SafeDiffuser: Safe Planning with Diffusion Probabilistic Models

## Abstract

Abstract Diffusion model-based approaches have shown promise in data-driven planning, but there are no safety guarantees, thus making it hard to be applied for safety-critical applications. To address these challenges, we propose a new method, called SafeDiffuser, to ensure diffusion probabilistic models satisfy specifications by using a class of control barrier functions. The key idea of our approach is to embed the proposed finite-time diffusion invariance into the denoising diffusion procedure, which enables trustworthy diffusion data generation. Moreover, we demonstrate that our finite-time diffusion invariance method through generative models not only maintains generalization performance but also creates robustness in safe data generation. We test our method on a series of safe planning tasks, including maze path generation, legged robot locomotion, and 3D space manipulation, with results showing the advantages of robustness and guarantees over vanilla diffusion models 2 2 2 Videos can be found in the anonymous website: https://safediffuser.github.io/safediffuser/ .

## 1 Introduction

Figure: Figure 1: Our proposed SafeDiffuser (lower) generates safe trajectories with guarantees, while the diffuser (upper) fails (from $\bigodot$ to $\bigotimes$).
Refer to caption: /html/2306.00148/assets/x1.png

Data-driven approaches have received increasing attentions due to their representation flexibility. Diffusion models Sohl-Dickstein et al. () Ho et al. () are data-driven generative models whose primary applications are in image generations Dhariwal and Nichol () Du et al. () Saharia et al. (). Recently, diffusion models, termed diffusers Janner et al. (), have shown promise in trajectory planning for a variety of robotic tasks. Diffusers enable flexible behavior synthesis that makes it well generalized in novel environments.

During inference, the diffuser, conditioned on the current state and objectives, starts from Gaussian noise to generate clean planning trajectories based on which we can get a control policy. After applying this control policy one step forward, we get a new state and run the diffusion procedure again to get a new planning trajectory. This process is repeated until the objective is achieved. However, one big challenge in this method is that there are no safety guarantees. For instance, the planning trajectory could easily violate safety constraints in the maze (as shown in Fig. [1](#S1.F1)). This shortcoming demands a fundamental fix to diffusion models to ensure the safe generation of planning trajectories in safety-critical applications such as trustworthy policy learning and optimization.

In this paper, we propose to ensure diffusion models with specification guarantees using finite-time diffusion invariance. An invariance set is a form of specification mainly consisting of safety constraints in planning tasks.
We ensure that diffusion models are invariant to uncertainties in diffusion procedure.
We achieve safety by combining receding horizon control with stable diffusion. In receding horizon control, we compute safe paths incrementally. The key insight is to replace each path computation with diffusion-based path generation, allowing a broader exploration of the path space and makes it relatively easy to include additional constraints. The computed path is combined with simulation to validate that it can be safely actuated.

To ensure diffusers with specifications guarantees, we first find diffusion dynamics for the denoising diffusion procedure. Then, we use Lyapunov-based methods with forward invariance properties such as a class of control barrier functions (CBFs) Ames et al. () Glotfelter et al. () Nguyen and Sreenath () Xiao and Belta (), to formally guarantee the satisfaction of specifications at the end of the diffusion procedure. CBFs works well in planning time using robot dynamics. However, doing this in diffusion models poses extra challenges since the generated data is not directly associated with robot dynamics which makes the use of CBFs non-trivial. As oppose to existing literature, 1. we propose to embed invariance into the diffusion time for diffusers. Thus, finite-time invariance is required in diffusers since specifications are usually violated as the trajectory is initially Gaussian noise. 2. We propose to add diffusion time components in invariance to address local trap problems that are prominent in planning. 3. We propose a quadratic program approach to incorporate finite-time diffusion invariance into the diffusion to maximally preserve the performance.

In summary, we make the following new contributions:

- •
We propose formal guarantees for diffusion probabilistic models via control-theoretic invariance.
- •
We propose a novel notion of finite-time diffusion invariance, and use a class of CBFs to incorporate it into the diffusion time of the procedure. We proposed three different safe diffusers, and show how we may address the local trap problem from specifications that are prominent in planning tasks.
- •
We demonstrate the effectiveness of our method on a variety of planning tasks using diffusion models, including safe planning in maze, robot locomotion and manipulation.

## 2 Preliminaries

In this section, we provide background on diffusion models and forward invariance in control theory.

Diffusion Probabilistic Models. Diffusion probabilistic models Sohl-Dickstein et al. () Ho et al. () Janner et al. () are latent variable models representing a data generation process as an iterative denoising procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i})$, $i\in\{1,\dots,N\}$, where $\bm{\tau}^{1},\dots,\bm{\tau}^{N}$ are latent variables of the same dimensionality of the clean (noiseless) data $\bm{\tau}^{0}\sim q(\bm{\tau}^{0})$, and $N$ is the total denoising steps. This denoising procedure is the reverse of a forward diffusion process $q(\bm{\tau}^{i}|\bm{\tau}^{i-1})$ that gradually corrupts the clean data by adding noise. The denoising data generation is denoted by

$$ $p_{\theta}(\bm{\tau}^{0})=\int p_{\theta}(\bm{\tau}^{0:N})d\bm{\tau}^{1:N}=\int p(\bm{\tau}^{N})\prod_{i=1}^{N}p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i})d\bm{\tau}^{1:N},$ (1) $$

where $p(\bm{\tau}^{N})$ is a standard Gaussian prior distribution, and the joint distribution $p_{\theta}(\bm{\tau}^{0:N})$ is defined as a Markov chain with learned Gaussian transitions starting at $p(\bm{\tau}^{N})$. The parameter $\theta$ is optimized by minimizing the usual variational bound on the negative log-likelihood of the reverse process: $\theta^{*}=\arg\min_{\theta}\mathbb{E}_{\bm{\tau}^{0}}\left[-\textbf{log }p_{\theta}(\bm{\tau}^{0})\right].$
The forward diffusion process $q(\bm{\tau}^{i}|\bm{\tau}^{i-1})$ is usually prespecified. The reverse process is often parameterized as Gaussian with time-dependent mean and variance.

Notations. There are two “times” involved in the paper: that of the diffusion process and that of the planning horizon. We use superscripts ($i$ when unspecified) to denote the diffusion time of a trajectory (state) and subscripts ($k$ when unspecified) to denote the planning time of a state on the trajectory. For instance, $\bm{\tau}^{0}$ denotes the planning trajectory at denoising diffusion time step 0 0 (i.e., a noiseless trajectory), and $\bm{x}_{k}^{0}$ denotes the state on the trajectory at planning time step k during the denoising diffusion time step 0 (i.e., a noiseless state). We note it as $\bm{x}_{k}=\bm{x}_{k}^{0}$ ($\bm{\tau}=\bm{\tau}^{0}$ as well) when there is no ambiguity. Further, a trajectory $\bm{\tau}^{i}$ is defined as a planning-time sequence of discretized states, i.e., $\bm{\tau}^{i}=(\bm{x}_{0}^{i},\bm{x}_{1}^{i},\dots,\bm{x}_{k}^{i},\dots,\bm{x}_{H}^{i})$, where $H\in\mathbb{N}$ is the planning horizon. During the denoising diffusion procedure, the diffusion time changes from $N$ to 0 0, while the planning time varies from 0 0 to $H$.

Forward Invariance in Control Theory.
Consider an affine control system of the form:

$$ $\dot{\bm{x}}_{t}=f(\bm{x}_{t})+g(\bm{x}_{t})\bm{u}_{t}$ (2) $$

where $\bm{x}_{t}\in\mathbb{R}^{n}$, $f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$
and $g:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times q}$ are locally
Lipschitz, and $\bm{u}_{t}\in U\subset\mathbb{R}^{q}$, where $U$ denotes a control constraint set. $\dot{\bm{x}}_{t}$
denotes the (planning) time derivative of state $\bm{x}_{t}$.

###### Definition 2.1 .

(Set invariance):
A set $C\subset\mathbb{R}^{n}$ is forward invariant for system ([2](#S2.E2)) if its solutions for some $\bm{u}\in U$ starting at any $\bm{x}_{0}\in C$ satisfy $\bm{x}_{t}\in C,$ $\forall t\geq 0$.

###### Definition 2.2 .

(Extended class $\mathcal{K}$ function Khalil ()):
A Lipschitz continuous function $\alpha:[-b,a)\rightarrow(-\infty,\infty),b>0,a>0$ belongs to extended class $\mathcal{K}$ if it is strictly increasing and $\alpha(0)=0$.

Consider a safety constraint $b(\bm{x}_{t})\geq 0$ for system ([2](#S2.E2)), where $b:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is continuously differentiable, we define a safe set in the form: $C:=\{\bm{x}_{t}\in\mathbb{R}^{n}:b(\bm{x}_{t})\geq 0\}$.

###### Definition 2.3 .

(Control Barrier Function (CBF) Ames et al. ()):
A function $b:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a CBF if there exists an extended class $\mathcal{K}$ function $\alpha$ such that

$$ $\sup_{\bm{u}_{t}\in U}[L_{f}b(\bm{x}_{t})+[L_{g}b(\bm{x}_{t})]\bm{u}_{t}+\alpha(b(\bm{x}_{t}))]\geq 0,$ (3) $$

for all $\bm{x}_{t}\in C$. $L_{f}$ and $L_{g}$ denote Lie derivatives w.r.t. $\bm{x}$ along $f$ and $g$, respectively.

###### Theorem 2.4 ( Ames et al. ( 2017 ) ) .

Given a CBF $b(\bm{x}_{t})$ from Def. [2.3](#S2.Thmtheorem3), if $\bm{x}_{0}\in C$, then any Lipschitz continuous controller $\bm{u}_{t}$ that satisfies the constraint in ([3](#S2.E3)), $\forall t\geq 0$ renders $C$ forward invariant for system ([2](#S2.E2)).

If we need to differentiate $b(\bm{x}_{t})$ more than once along the dynamics ([2](#S2.E2)) until the control $\bm{u}_{t}$ explicitly shows, we use a high-order CBF Nguyen and Sreenath () Xiao and Belta () as a general form of CBF to guarantee safety for ([2](#S2.E2)).
In this work, we map the forward invariance in control theory to finite time diffusion invariance in diffusion models, where we incorporate CBFs into the diffusion time as opposed to their regular applications in planning time. In addition, we show how we may address the local traps during diffusion.

###### Definition 2.1 .

###### Definition 2.2 .

###### Definition 2.3 .

###### Theorem 2.4 ( Ames et al. ( 2017 ) ) .

## 3 Safe Diffuser

In this section, we propose three different safe diffusers to ensure the safe generation of data in diffusion, i.e., to ensure the satisfaction of specifications $b(\bm{x}_{k})\geq 0,\forall k\in\{0,\dots,H\}$. Each of the proposed safe diffusers have their own flexibility, such as avoiding local traps in planning. We consider discretized system states in the sequel. Safety in continuous planning time can be guaranteed using a lower hierarchical control framework employing other CBFs, as in Ames et al. (); Nguyen and Sreenath (); Xiao and Belta ().

In the denoising diffusion procedure, since the learned Gaussian transitions starts at $p(\bm{x}^{N})\sim\mathcal{N}(0,\bm{I})$, it is highly likely that specifications are initially violated, i.e., $\exists k\in\{0,\dots,H\},b(\bm{x}_{k}^{N})<0$. For safe data generation, we wish to have $b(\bm{x}_{k}^{0})\geq 0(\textit{i.e., }b(\bm{x}_{k})\geq 0),\forall k\in\{0,\dots,H\}$. Since the maximum denoising diffusion step $N$ is limited, this needs to be guaranteed in a finite diffusion time step. Therefore, we propose the finite-time diffusion invariance of the diffusion procedure as follows:

###### Definition 3.1 (Finite-time Diffusion Invariance) .

If there exists $i\in\{0,\dots,N\}$ such that $b(\bm{x}_{k}^{j})\geq 0,\forall k\in\{0,\dots,H\},\forall j\leq i$, then a denoising diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{1,\dots,N\}$ with respect to a specification $b(\bm{x}_{k})\geq 0,\forall k\in\{0,\dots,H\}$ is finite-time diffusion invariant.

The above definition can be interpreted as that if $b(\bm{x}_{k}^{N})\geq 0,k\in\{0,\dots,H\}$, then we require $b(\bm{x}_{k}^{i})\geq 0,\forall i\in\{0,\dots,N\}$ (similar to the forward invariance definition as in Def. [2.1](#S2.Thmtheorem1)); otherwise, we require that $b(\bm{x}_{k}^{j})\geq 0,\forall j\in\{0,\dots,i\},i\in\{0,\dots,N\}$, where $i$ is a finite diffusion time.

In the following, we propose three different methods to achieve finite-time diffusion invariance. The first method is a general form of the safe- diffuser, and the other two are variants to address local traps in planning.

###### Definition 3.1 (Finite-time Diffusion Invariance) .

### 3.1 Robust-Safe Diffuser

Figure: Figure 2: The proposed SafeDiffuser workflow. SafeDiffuser performs an additional step of invariance QP solver in the diffusion dynamics to ensure safety. The final control signal is inferred from safe planning trajectories.
Refer to caption: /html/2306.00148/assets/x2.png

The safe denoising diffusion procedure is considered at every diffusion step. Following ([1](#S2.E1)), the data generation at the diffusion time $j\in\{0,\dots,N-1\}$ is given by:

$$ $p_{\theta}(\bm{\tau}^{j})=\int p(\bm{\tau}^{N})\prod_{i=j+1}^{N}p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i})d\bm{\tau}^{j+1:N}$ (4) $$

A sample $\bm{\tau}^{j},j\in\{0,\dots,N-1\}$ follows the data distribution in ([4](#S3.E4)), i.e., we have

$$ $\bm{\tau}^{j}\sim p_{\theta}(\bm{\tau}^{j}).$ (5) $$

The denoising diffusion dynamics are then given by:

$$ $\dot{\bm{\tau}}^{j}=\lim_{\Delta\tau\rightarrow 0}\frac{\bm{\tau}^{j}-\bm{\tau}^{j+1}}{\Delta\tau}$ (6) $$

where $\dot{\bm{\tau}}$ is the (diffusion) time derivative of $\bm{\tau}$. $\Delta\tau>0$ is a small enough diffusion time step length during implementations, and $\bm{\tau}^{j+1}$ is available from the last diffusion step.

In order to impose finite-time diffusion invariance on the diffusion procedure, we wish to make diffusion dynamics ([6](#S3.E6)) controllable. We reformulate ([6](#S3.E6)) as

$$ $\dot{\bm{\tau}}^{j}=\bm{u}^{j},$ (7) $$

where $\bm{u}^{j}$ is a control variable of the same dimensionality as ${\bm{\tau}}^{j}$. On the other hand, we wish $\bm{u}^{j}$ to stay close to $\frac{\bm{\tau}^{j}-\bm{\tau}^{j+1}}{\Delta\tau}$ in order to maximally preserve the performance of the diffusion model. The above model can be rewritten in terms of each state on the trajectory $\bm{\tau}^{j}$: $\dot{\bm{x}}_{k}^{j}=\bm{u}_{k}^{j},$
where $\bm{u}_{k}^{j}$ is the $k^{th}$ component of $\bm{u}^{j}$. Then, we can define CBFs to ensure the satisfaction of $b(\bm{x}_{k}^{j})\geq 0$ (in finite diffusion time):
$h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j}):=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))\geq 0,k\in\{0,\dots,H\},j\in\{0,\dots,N-1\},$
where $\alpha(\cdot)$ is an extended class $\mathcal{K}$ function. We have the following theorem to show the finite-time diffusion invariance (proof is given in appendix):

###### Theorem 3.2 .

Let the diffusion dynamics defined as in ([6](#S3.E6)) whose controllable form is defined as in ([7](#S3.E7)). If there exists an extended class $\mathcal{K}$ function $\alpha$ such that

$$ $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j})\geq 0,\forall k\in\{0,\dots,H\},\forall j\in\{0,\dots,N-1\},$ (8) $$

where $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j})=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))$, then the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{1,\dots,N\}$ is finite-time diffusion invariant with almost probability 1.

One possible issue in the robust-safe diffusion procedure is that if $b(\bm{x}_{k}^{j})\geq 0$ when $j$ is close to the initial diffusion step $N$, then the state $\bm{x}_{k}^{j}$ can never violate the specification after diffusion step $j<N$. When there are local traps from specifications, the state $\bm{x}_{k}^{j}$ may get stuck there during the denoising diffusion process, which may adversely affect the performance. In order to address this issue, we propose relaxed-safe diffuser and time-varying-safe diffuser in the following subsections.

###### Theorem 3.2 .

### 3.2 Relaxed-Safe Diffuser

In order to address the local trap problems imposed by specifications during the denoising diffusion procedure, we propose a variation of the robust-safe diffuser. We define the diffusion dynamics and their controllable form as in ([6](#S3.E6)) - ([7](#S3.E7)). The modified versions for CBFs are in the form:

$$ $h(\bm{u}_{k}^{j},r_{k}^{j}|\bm{x}_{k}^{j}):=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))-w_{k}(j)r_{k}^{j}\geq 0,k\in\{0,\dots,H\},j\in\{0,\dots,N-1\},$ (9) $$

where $r_{k}^{j}\in\mathbb{R}$ is a relaxation variable that is to be determined (shown in the next section). $w_{k}(j)\geq 0$ is a diffusion time-varying weight on the relaxation variable such that it gradually decrease to 0 as $j\rightarrow 0$.

When $w_{k}(j)$ decreases to 0, the condition ([9](#S3.E9)) becomes a hard constraint. One problem in such cases is that the diffusion performance may be adversely affected by such a hard constraint. In order to address this issue, we may run additional $N_{a}\in\mathbb{N}$ diffusion steps, while setting the diffusion time to 0 when $j<0$ in the reverse process. The corresponding CBF conditions to ([9](#S3.E9)) are to change the domain of $j$ to $j\in\{-N_{a},\dots,N-1\}$.We also have the following theorem to show the finite-time diffusion invariance (proof is given in appendix):

###### Theorem 3.3 .

Let the diffusion dynamics defined as in ([6](#S3.E6)) whose controllable form is defined as in ([7](#S3.E7)). If there exist an extended class $\mathcal{K}$ function $\alpha$, a large enough extra diffusion step $N_{a}\in\mathbb{N}$, and a time-varying weight $w_{k}(j)$ where $w_{k}(j)=0$ for all $j\leq 0$ such that

$$ $h(\bm{u}_{k}^{j},r_{k}^{j}|\bm{x}_{k}^{j})\geq 0,\forall k\in\{0,\dots,H\},\forall j\in\{-N_{a},\dots,N-1\},$ (10) $$

where $h(\bm{u}_{k}^{j},r_{k}^{j}|\bm{x}_{k}^{j})=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))-w_{k}(j)r_{k}^{j}$, then the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{-N_{a},\dots,N\}$ is finite-time diffusion invariant with almost probability 1.

After the denoising diffusion procedure is done at step $-N_{a}$, we would set the $\bm{\tau}^{-N_{a}}$ as the output data of the diffusion model, i.e., $\bm{\tau}^{0}=\bm{\tau}^{-N_{a}}$.

###### Theorem 3.3 .

### 3.3 Time-Varying-Safe Diffuser

As an alternative to the relaxed-safe diffuser, we propose another safe diffuser called time-varying-safe diffuser in this subsection. The proposed time-varying-safe diffuser can also address the local trap issues induced by specifications.

In this case, we directly modify the specification $b(\bm{x}_{k}^{j})\geq 0$ by a diffusion time-varying function $\gamma_{k}:j\rightarrow\mathbb{R}$ in the form:

$$ $b(\bm{x}_{k}^{j})-\gamma_{k}(j)\geq 0,k\in\{0,\dots,H\},j\in\{0,\dots,N\},$ (11) $$

where $\gamma_{k}(j)$ is continuously differentiable, and is defined such that $\gamma_{k}(N)\leq b(\bm{x}_{k}^{N})$ and $\gamma_{k}(0)=0$.

The modified time-varying specification can then be enforced using CBFs: $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j},\gamma_{k}(j)):=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}-\dot{\gamma}_{k}(j)+\alpha(b(\bm{x}_{k}^{j})-\gamma_{k}(j))\geq 0,k\in\{0,\dots,H\},j\in\{0,\dots,N-1\},$
where $\dot{\gamma}_{k}(j)$ is the diffusion time derivative of $\gamma_{k}(j)$.
Finally, we have the following theorem to show the finite-time diffusion invariance (proof is given in appendix):

###### Theorem 3.4 .

Let the diffusion dynamics defined as in ([6](#S3.E6)) whose controllable form is defined as in ([7](#S3.E7)). If there exist an extended class $\mathcal{K}$ function $\alpha$ and a time-varying function $\gamma_{k}(j)$ where $\gamma_{k}(N)\leq b(\bm{x}_{k}^{N})$ and $\gamma_{k}(0)=0$ such that

$$ $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j},\gamma_{k}(j))\geq 0,\forall k\in\{0,\dots,H\},\forall j\in\{0,\dots,N-1\},$ (12) $$

where $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j},\gamma_{k}(j))=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}-\dot{\gamma}_{k}(j)+\alpha(b(\bm{x}_{k}^{j})-\gamma_{k}(j))$, then the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{0,\dots,N\}$ is finite-time diffusion invariant.

###### Theorem 3.4 .

## 4 Enforcing Invariance in Diffuser

In this section, we show how we may incorporate the three proposed invariance methods from the last section into diffusion models. Enforcing the finite-time invariance in diffusion models is equivalent to ensure the satisfaction of the conditions in Thms. [3.2](#S3.Thmtheorem2)-[3.4](#S3.Thmtheorem4) in the diffusion procedure. In this section, we propose a minimum-deviation quadratic program (QP) approach to achieve that. We wish to enforce these conditions at every step of the diffusion (as shown in Fig. [2](#S3.F2)) as those states that are far from the specification boundaries $b(\bm{x}_{k}^{j})=0$ can also be optimized accordingly, and thus, the model may generate more coherent trajectories.

Enforcing Invariance for Robust-Safe (RoS) and Time-Varying-Safe Diffusers.
During implementation, the diffusion time step length $\Delta\bm{\tau}$ in ([6](#S3.E6)) is chosen to be small enough, and we wish the control $\bm{u}^{j}$ to stay close to the right-hand side of ([6](#S3.E6)). Thus, we can formulate the following QP-based optimization to find the optimal control for $\bm{u}^{j}$ that satisfies the condition in Thms. [3.2](#S3.Thmtheorem2) or [3.4](#S3.Thmtheorem4):

$$ $\displaystyle\bm{u}^{j*}$ $\displaystyle=\arg\min_{\bm{u}^{j}}||\bm{u}^{j}-\frac{\bm{\tau}^{j}-\bm{\tau}^{j+1}}{\Delta\tau}||^{2},\text{ s.t., }(\ref{eqn:diff_cbf})\text{ if RoS diffuser }\text{ else s.t., }(\ref{eqn:diff_cbft}),$ (13) $$

where $||\cdot||$ denotes the 2-norm of a vector. If we have more than one specification, we can add the corresponding conditions in Thm. [3.2](#S3.Thmtheorem2) for each of them to the above QP. After we solve the above QP and get $\bm{u}^{j*}$, we update ([7](#S3.E7)) by setting $\bm{u}^{j}=\bm{u}^{j*}$ within the time step and get a new state for the diffusion procedure. Note that all of these happen at the end of each diffusion step.

Figure: Algorithm 1 Enforcing invariance in diffusion models

Enforcing Invariance for Relaxed-Safe Diffuser.
In this case, since we have relaxation variables for each of the safety specification, we wish to minimize these relaxations in the cost function to drive all the state towards the satisfaction of specifications. In other words, we have the following QP:

$$ $\displaystyle\bm{u}^{j*},r^{j*}$ $\displaystyle=\arg\min_{\bm{u}^{j},r^{j}}||\bm{u}^{j}-\frac{\bm{\tau}^{j}-\bm{\tau}^{j+1}}{\Delta\tau}||^{2}+||r^{j}||^{2},\text{ s.t., }(\ref{eqn:diff_rcbf}),$ (14) $$

where $r^{j}$ is the concatenation of $r_{k}^{j}$ for all $k\in\{0,\dots,H\}$. As an alternative, all the constraints above may share the same relaxation variable, i.e., the dimension of $r^{j}$ is only one. After we solve the above QP and get $\bm{u}^{j*}$, we update ([7](#S3.E7)) by setting $\bm{u}^{j}=\bm{u}^{j*}$ within the time step and get a new state.

Complexity of enforcing invariance The computational complexity of a QP is $\mathcal{O}(q^{3})$, where $q$ is the dimension of the decision variable. When there is a set $S$ of specifications, we just add the corresponding constraints for each specification the QP. The complexity of the three proposed safe diffuser are similar.

The algorithm for enforcing invariance is straight forward, which includes the construction of proper conditions, the solving of QP, and the update of diffusion state. We summary the algorithm in Alg. [1](#alg1).

Figure: Figure 3: Maze planning (blue to red) denoising diffusion procedure with classifier-based guidance (Left to right: diffusion time steps 256, 4, 3, -50, respectively). Red ellipse and super-ellipse (outside) denote safe specifications. The safe classifier-based guidance approach adversely affects the diffusion procedure without guarantees.
Refer to caption: /html/2306.00148/assets/figs/maze/gd/0.png

## 5 Experiments

**Table 1: Maze safe planning comparisons with benchmarks. Items are short for satisfaction of simple specifications (S-SPEC) and complex specifications (C-SPEC), score of planning tasks (SCORE), computation time at each diffusion step (TIME) in seconds, respectively. In the method column, items are short for robust-safe diffuser (RoS-DIFFUSER), relaxed-safe diffuser (ReS-DIFFUSER), and time-varying-safe diffuser (TVS-DIFFUSER), respectively. The classifier guidance$-\epsilon$ method applies (safe) gradient to the model when the state is $\epsilon>0$ close to the boundary.**
| Method | S-spec($\uparrow\&\geq 0$) | C-spec($\uparrow\&\geq 0$) | Score ($\uparrow$) | time |
| --- | --- | --- | --- | --- |
| Diffuser (baseline) Janner et al. () | -0.983 | -0.894 | $1.016{\small\pm 0.712}$ | 0.007 |
| Truncate Brockman et al. () | $-1.192e^{-7}$ | -0.759 | $0.754{\small\pm 0.779}$ | 0.024 |
| Classifier guidance Dhariwal and Nichol () | -0.789 | -0.979 | $0.502{\small\pm 0.328}$ | 0.053 |
| Classifier guidance$-\epsilon$ Dhariwal and Nichol () | -0.853 | -0.995 | $0.470{\small\pm 0.366}$ | 0.061 |
| RoS-diffuser (Ours) | $-2.384e^{-7}$ | $-5.960e^{-7}$ | $0.770{\small\pm 0.782}$ | 0.106 |
| ReS-diffuser (Ours) | $-2.384e^{-7}$ | $-4.768e^{-7}$ | $0.762{\small\pm 0.746}$ | 0.107 |
| TVS-diffuser (Ours) | $-2.384e^{-7}$ | $-5.364e^{-7}$ | $0.806{\small\pm 0.783}$ | 0.107 |

We set up experiments to answer the following questions:

- •
Does our method match the theoretical potential in various tasks quantitatively and qualitatively?
- •
How does our method compare with state-of-the-art approaches in enforcing safety specifications?
- •
How does our proposed method affect the performance of diffusion under guaranteed specifications?

### 5.1 Safe Planning in Maze

In this experiment, we aim to impose trajectory constraints on the planning path of a maze. The training data is publicly available from Janner et al. (), in which initial positions and destinations in maze are randomly generated. The diffusion model is conditioned on the initial positions and destinations.

Figure: Figure 4: Maze planning (blue to red) denoising diffusion procedure with the proposed time-varying safe diffuser (Left to right: diffusion time steps 256, 4, 3, -50, respectively). Red ellipse and super-ellipse (outside) denote safe specifications. The proposed time-varying safe diffuser can guarantee specifications at the end of diffusion while not significantly affecting the diffusion procedure.
Refer to caption: /html/2306.00148/assets/figs/maze/cbf/0.png

The diffuser cannot guarantee the satisfaction of any specifications, as shown in Fig. [1](#S1.F1). When using classifier-based guidance in diffusion for safety specifications, the performance could be significantly affected (Fig. [3](#S4.F3)). As a result, the generated trajectory will largely deviates from the desired one with no safety. The proposed robust-safe diffuser (RoS-diffuser), relaxed-safe diffuser (ReS-diffuser), and time-varying-safe diffuser (TVS-diffuser) can all guarantee the satisfaction of specifications, even when the specifications are complex (as long as they are differentiable), as shown in Table [1](#S5.T1). The satisfaction scores of the proposed methods are not strictly positive, and this may be due to the computation errors or inter-sampling effect as even the truncation method cannot strictly satisfy the simple specifications. The proposed methods can also maximally preserve the performance of diffusion models, as shown by the score in Table [1](#S5.T1), as well as in Fig. [4](#S5.F4). The ReS-diffuser and TVS-diffuser can address the local trap problem from specifications, as shown by figures in appendix.

### 5.2 Safe Planning for Robot Locomotion

For robot locomotion (in MuJoCo), we wish the robot to avoid collisions with obstacles, such as the roof. In this case, since there is no local trap problem, we only consider robust-safe diffuser (RoS-diffuser). Others work similarly. The training data set is publicly available from Janner et al. ().

**Table 2: Robot safe planning comparisons with benchmarks. Items are short for satisfaction of simple specifications (S-SPEC), satisfaction of complex specifications (C-SPEC), score of planning tasks (SCORE), computation time at each diffusion step (TIME) in seconds, respectively.**
| Experiment | Method | S-spec($\uparrow\&\geq 0$) | C-spec($\uparrow\&\geq 0$) | Score ($\uparrow$) | time |
| --- | --- | --- | --- | --- | --- |
|  | Diffuser (baseline) Janner et al. () | -9.375 | -4.891 | $0.346{\small\pm 0.106}$ | 0.037 |
| Walker2D | Truncate Brockman et al. () | $0.0$ | $\times$ | $0.286{\small\pm 0.180}$ | 0.105 |
|  | Classifier guidance Dhariwal and Nichol () | -0.575 | -0.326 | $0.208{\small\pm 0.140}$ | 0.053 |
|  | RoS-diffuser (Ours) | $0.0$ | $-6.706e^{-8}$ | $0.312{\small\pm 0.782}$ | 0.183 |
|  | Diffuser (baseline) Janner et al. () | -2.180 | -1.862 | $0.455{\small\pm 0.038}$ | 0.038 |
| Hopper | Truncate Brockman et al. () | $0.0$ | $\times$ | $0.436{\small\pm 0.067}$ | 0.046 |
|  | Classifier guidance Dhariwal and Nichol () | -0.894 | -0.524 | $0.478{\small\pm 0.038}$ | 0.047 |
|  | RoS-diffuser (Ours) | $0.0$ | $-5.960e^{-8}$ | $0.430{\small\pm 0.040}$ | 0.170 |

Figure: Figure 5: Walker2D planning denoising diffusion procedure with the proposed robust-safe diffuser (Up to down: diffusion time steps 20, 10, 0, respectively). The red line denotes the roof the walker needs to safely avoid during locomotion (safety specifications). Safety is violated at step 20 since the trajectory is initially Gaussian noise, but is eventually guaranteed (step 0).
Refer to caption: /html/2306.00148/assets/figs/gym/0.png

As expected, collisions with roof are very likely to happen in walker and hopper using the diffuser since there are no guarantees, as shown in Table [2](#S5.T2). The truncation method can work for simple specifications (S-spec), but not for complex specifications (C-spec). The classifier-based guidance can improve the satisfaction of specifications, but with no guarantees. Collision-free is guaranteed using the proposed Ros-diffuser, and one example of diffusion procedure is shown in Fig. [5](#S5.F5).

### 5.3 Safe Planning for Manipulation

For manipulation (in Pybullet), the diffusion models generate joint trajectories (as controls) for the robot, which are conditioned on the locations of the objects to grasp and place. The training data set is publicly available from Janner et al. (). Specifications are joint limitations to avoid collision in joint space.

**Table 3: Manipulation safe planning comparisons with benchmarks. Items are short for satisfaction of simple specifications (S-SPEC), satisfaction of complex specifications (C-SPEC), reward of planning tasks (REWARD), computation time at each diffusion step (TIME) in seconds, respectively. In the method column, the item is short for robust-safe diffuser (RoS-DIFFUSER).**
| Method | S-spec($\uparrow\&\geq 0$) | C-spec($\uparrow\&\geq 0$) | Reward ($\uparrow$) | time |
| --- | --- | --- | --- | --- |
| Diffuser (baseline) Janner et al. () | -0.057 | -0.065 | $0.650{\small\pm 0.107}$ | 0.038 |
| Truncate Brockman et al. () | $1.631e^{-8}$ | $\times$ | $0.575{\small\pm 0.112}$ | 0.069 |
| Classifier guidance Dhariwal and Nichol () | -0.050 | -0.053 | $0.800{\small\pm 0.328}$ | 0.075 |
| RoS-diffuser (Ours) | $0.072$ | $0.069$ | $0.925{\small\pm 0.107}$ | 0.088 |

Figure: Figure 6: Manipulation planning denoising diffusion procedure with the proposed robust-safe diffuser (Left to right: diffusion time steps 1000, 100, 0, and execution time step 100, respectively). The red dots denote the planning trajectory of the end-effector.
Refer to caption: /html/2306.00148/assets/figs/kuka/0.png

In this case, the truncation method still fails to work for complex specifications (speed-dependent joint limitations). Our proposed RoS-diffuser can work for all specifications as long as they are differentiable. An interesting observation is that the proposed RoS-diffuser can even improve the performance (reward) of diffusion models in this case, as shown in Table [3](#S5.T3). This may be due to the fact that the satisfaction of joint limitations can avoid collision in the joint space of the robot as Pybullet is a physical simulator. The computation time of the proposed RoS-diffuser is comparable to other methods. An illustration for the safe diffusion and manipulation procedure is shown in Fig. [6](#S5.F6).

## 6 Related Works

Diffusion models and planning Diffusion models Sohl-Dickstein et al. () Ho et al. () are data-driven generative modeling tools, widely used in applications to image generations Dhariwal and Nichol () Du et al. (), in planning Hafner et al. () Janner et al. () Ozair et al. () Janner et al. (), and in language Saharia et al. () Liu et al. (). Generative models are combined with reinforcement learning to explore dynamic models in the form of convolutional U-networks Kaiser et al. (), stochastic recurrent networks Ke et al. (), neural ODEs Du et al. (), generative
adversarial networks Eysenbach et al. (), neural radiance fields Li et al. (), and transformers Chen et al. (). Further, planning tasks are becoming increasingly important for diffusion models Lambert et al. () Ozair et al. () Janner et al. () as they can generalize well in all kinds of robotic problems.
However, there are no methods to equip diffusion models with specification guarantees, which is especially important for safety-critical applications. Here, we address this issue using the proposed finite-time diffusion invariance.

Set invariance and CBFs. An invariant set has been widely used to represent the safe behavior of dynamical systems Preindl () Rakovic et al. () Ames et al. () Glotfelter et al. () Xiao et al. (). In the state of the art of control, Control Barrier Functions (CBFs) are also widely used to prove set
invariance Aubin (), Prajna et al. (), Wisniewski and Sloth (). CBFs can be traced back to optimization problems
Boyd and Vandenberghe (), and are Lyapunov-like functions
Wieland and Allgöwer (). For time-varying systems, CBFs can also be adapted accordingly Lindemann and Dimarogonas (). Existing CBF approaches are usually applied in planning time since they are closely coupled with system dynamics. There are few studies of CBFs in other space, such as the diffusion time horizon in diffusion models. Our work addresses all these limitations.

Guarantees in neural networks.
Differentiable optimization methods show promise for neural network controllers with guarantees Pereira et al. (); Amos et al. (); Xiao et al. (). They are usually served as a layer (filter) in the neural networks. In Amos and Kolter (), a differentiable quadratic program (QP) layer, called OptNet, was introduced. OptNet with CBFs has been used in neural networks as a filter for safe controls Pereira et al. (), in which CBFs are not trainable, thus, potentially limiting the system’s learning performance. In Deshmukh et al. (); Zhao et al. (); Ferlez et al. (), safety guaranteed neural network controllers have been learned through verification-in-the-loop training. The verification approaches cannot ensure coverage of the entire state space. More recently, CBFs have been incorporated into neural ODEs to equip them with specification guarantees Xiao et al. (). However, none of these methods can be applied in diffusion models, which we address in this paper.

## 7 Conclusions, Discussions and Future Work

We have proposed finite-time diffusion invariance for diffusion models to ensure safe planning for safety-critical applications. We have demonstrated the effectiveness of our method on a series of robotic planning tasks.
Nonetheless, our method face a few shortcomings motivating for future work.

Specifically, specifications for diffusion models should be expressed as continuously differentiable constraints that may be unknown for planning tasks. Further work may explore how to learn specifications from history trajectory data. There is also a gap between planning and control using diffusion models. We may further investigate diffusion for safe control policies when robot dynamics are known or to be learned.

Broader Impact. Our proposed finite-time diffusion invariance can be applied to guarantees in other tasks, such as image generations, in addition to planning. In such cases, we can ensure the generation of desired patterns/objects, such as a cat in the generated images. We will further investigate those directions in our future work.

## 8 Acknowledgement

The research was supported in part by Capgemini Engineering. It was also partially sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. This research was also supported in part by the AI2050 program at Schmidt Futures (Grant G-
965 22-63172).

## 9 Proofs

### 9.1 Proof of Theorem 3.2

Proof: Given a continuously differentiable constraint $h(\bm{x}_{t})\geq 0$ ($h(\bm{x}_{0})\geq 0$), by Nagumo’s theorem Nagumo (), the necessary and sufficient condition for the satisfaction of $h(\bm{x}_{t})\geq 0,\forall t\geq 0$ is

$$ $\dot{h}(\bm{x}_{t})\geq 0,\textit{ when }h(\bm{x}_{t})=0,$ $$

If $b(\bm{x}_{k}^{N})\geq 0,k\in\{0,\dots,H\}$, then
the condition ([8](#S3.E8)) is equivalent to

$$ $\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\dot{\bm{x}}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))\geq 0,$ $$

where $\dot{\bm{x}}_{k}^{j}$ is the diffusion time derivative. The last equation is equivalent to

$$ $\dot{b}(\bm{x}_{k}^{j})+\alpha(b(\bm{x}_{k}^{j}))\geq 0,$ $$

Since $\alpha$ is an extended class $\mathcal{K}$ function, we have that

$$ $\alpha(b(\bm{x}_{k}^{j}))\rightarrow 0,\textit{ as }b(\bm{x}_{k}^{j})\rightarrow 0,$ $$

In other words, we have $\dot{b}(\bm{x}_{k}^{j})\geq 0$ when $b(\bm{x}_{k}^{j})=0$. Since $b(\bm{x}_{k}^{N})\geq 0,k\in\{0,\dots,H\}$, then by Nagumo’s theorem, we have $b(\bm{x}_{k}^{j})\geq 0,\forall j\in\{0,\dots,N-1\}$. Therefore, the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{1,\dots,N\}$ is finite-time diffusion invariant, and the finite time in diffusion invariance is $N$.

If, on the other hand, $b(\bm{x}_{k}^{N})<0,k\in\{0,\dots,H\}$, then we can define a Lyapunov function:

$$ $V(\bm{x}_{k}^{j})=-b(\bm{x}_{k}^{j}),k\in\{0,\dots,H\},j\in\{0,\dots,N\},$ (15) $$

and $V(\bm{x}_{k}^{N})>0$.

Since $\alpha$ is an extended class $\mathcal{K}$ function, replacing $b(\bm{x}_{k}^{j})$ by $V(\bm{x}_{k}^{j})$, the condition ([8](#S3.E8)) is equivalent to

$$ $\frac{dV(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\dot{\bm{x}}_{k}^{j}+\alpha(V(\bm{x}_{k}^{j}))\leq 0,$ $$

which is equivalent to

$$ $\dot{V}(\bm{x}_{k}^{j})+\alpha(V(\bm{x}_{k}^{j}))\leq 0,$ $$

Since $\dot{V}(\bm{x}_{k}^{j})\leq-\alpha(V(\bm{x}_{k}^{j}))<0$, we have that $V(\bm{x}_{k}^{j})$ will be stabilized to 0 by Lyapunov stability theory. In other words, the state $\bm{x}_{k}^{j}$ will be stabilized to the boundary $b(\bm{x}_{k}^{j})=0$. Specifically, when $\alpha$ is a linear function, the last equation can be rewritten as

$$ $\dot{V}(\bm{x}_{k}^{j})+\epsilon V(\bm{x}_{k}^{j}))\leq 0,$ (16) $$

where $\epsilon>0$.
Suppose we have

$$ $\dot{V}(\bm{x}_{k}^{j})+\epsilon V(\bm{x}_{k}^{j})=0,$ $$

the solution to the above equation is

$$ $V(\bm{x}_{k}^{j})=V(\bm{x}_{k}^{N})e^{-\epsilon(N-j)},$ $$

Using the comparison lemma Khalil (), equation ([16](#S9.E16)) implies that

$$ $V(\bm{x}_{k}^{j})\leq V(\bm{x}_{k}^{N})e^{-\epsilon(N-j)},j\in\{0,\dots,N\},$ $$

Therefore,

$$ $V(\bm{x}_{k}^{j})\rightarrow 0,\textit{ as }j\rightarrow 0,\textit{ if $N$ is sufficiently large},$ $$

and the state $\bm{x}_{k}^{j}$ will be exponentially stabilized to the boundary $b(\bm{x}_{k}^{j})=0$. If at diffusion time $j\in\{0,\dots.N-1\}$, the state $\bm{x}_{k}^{j}$ is close to the boundary, and the probability for the state $\bm{x}_{k}^{j}$ to jump into the set: $\{\bm{x}_{k}^{j}:b(\bm{x}_{k}^{j})\geq 0\}$ is $p$ (due to the Gaussian transitions in the diffusion process), then the probability for $\bm{x}_{k}^{0}$ to enter the set $\{\bm{x}_{k}^{j}:b(\bm{x}_{k}^{j})\geq 0\}$ is $1-(1-p)^{j}$. When $j$ is large enough, then the probability such that $b(\bm{x}_{k}^{0})\geq 0$ is almost 1. When $b(\bm{x}_{k}^{l})\geq 0$ at step $l\leq j$, then $b(\bm{x}_{k}^{r})\geq 0$ for all $r\leq l$ following the Nagumo’s theorem (as in the first case of the proof). Therefore, the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{1,\dots,N\}$ is finite-time diffusion invariant with almost probability 1. $\blacksquare$

### 9.2 Proof of Theorem 3.3

Proof: Suppose the weight $w_{k}(j)$ is chosen such that $w_{k}(j)=0$ when $j=0$, then the condition ([10](#S3.E10)) becomes a hard constraint when $j<0$. In other words, equation ([10](#S3.E10)) becomes:

$$ $h(\bm{u}_{k}^{j}|\bm{x}_{k}^{j}):=\frac{db(\bm{x}_{k}^{j})}{d\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\alpha(b(\bm{x}_{k}^{j}))\geq 0,k\in\{0,\dots,H\},j\in\{-N_{a},\dots,0\},$ $$

Then, the proof is simiar to that of the Thm. 3.2, and we have that the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{-N_{a},\dots,N\}$ is finite-time diffusion invariant with almost probability 1. $\hfill\blacksquare$

### 9.3 Proof of Theorem 3.4

Proof: Since $\gamma_{k}(N)\leq b(\bm{x}_{k}^{N})$, we have that $s(\bm{x}_{k}^{j},\gamma_{k}(j)):=b(\bm{x}_{k}^{j})-\gamma_{k}(j)\geq 0$ when $j=N$.

The condition ([12](#S3.E12)) is equivalent to

$$ $\frac{\partial s(\bm{x}_{k}^{j},\gamma_{k}(j))}{\partial\bm{x}_{k}^{j}}\bm{u}_{k}^{j}+\frac{\partial s(\bm{x}_{k}^{j},\gamma_{k}(j))}{\partial j}+\alpha(s(\bm{x}_{k}^{j},\gamma_{k}(j)))\geq 0,$ $$

which can be rewritten as

$$ $\dot{s}(\bm{x}_{k}^{j},\gamma_{k}(j))+\alpha(s(\bm{x}_{k}^{j},\gamma_{k}(j)))\geq 0,$ $$

Using the Nagumo’ theorem presented in the proof of Thm. 3.2, we have that

$$ $s(\bm{x}_{k}^{j},\gamma_{k}(j))\geq 0,\forall j\in\{0,\dots,N\}$ $$

since $s(\bm{x}_{k}^{N},\gamma_{k}(N))\geq 0$.

As $\gamma_{k}(0)=0$ and $s(\bm{x}_{k}^{j},\gamma_{k}(j)):=b(\bm{x}_{k}^{j})-\gamma_{k}(j)$, we have that $b(\bm{x}_{k}^{0})\geq 0,\forall k\in\{0,\dots,H\}$. Therefore, the diffusion procedure $p_{\theta}(\bm{\tau}^{i-1}|\bm{\tau}^{i}),i\in\{0,\dots,N\}$ is finite-time diffusion invariant, and the finite time in diffusion invariance is 0 0. $\hfill\blacksquare$

## 10 Experiment Details

### 10.1 Safe Planning in Maze

In this experiment, we aim to impose trajectory constraints on the planning path of a maze. The training data is publicly available from Janner et al. (), in which initial positions and destinations in maze are randomly generated. The diffusion model is conditioned on the initial positions and destinations.

Specifications. The simple safety specification for the planning trajectory is defined as an super-ellipse-shape obstacle:

$$ $\left(\frac{x-x_{0}}{a}\right)^{2}+\left(\frac{y-y_{0}}{b}\right)^{2}\geq 1,$ (17) $$

where $(x,y)\in\mathbb{R}^{2}$ is the state on the planning trajectory, $(x_{0},y_{0})\in\mathbb{R}^{2}$ is the location of the obstacle. $a>0,b>0$. Since the state $(x,y)$ is normalized in diffusion models, we also need to normalize the above constraint accordingly. In other words, we normalize $x_{0},a$ and $y_{0},b$ according to the normalization of $(x,y)$ along the $x$-axis and $y-$axis, respectively.

The complex safety specification for the planning trajectory is defined as an ellipse-shape obstacle:

$$ $\left(\frac{x-x_{0}}{a}\right)^{4}+\left(\frac{y-y_{0}}{b}\right)^{4}\geq 1,$ (18) $$

We also normalize the above constraint as in the simple case. In this case, it is non-trivial to truncate the planning trajectory to satisfy the constraint. When we have much more complex specifications, it is too hard for the truncation method to work.

Model setup, training and testing. The diffusion model structure is the same as the open source one (Maze2D-large-v1) provided in Janner et al. (). We set the planning horizon as 384, the diffusion steps as 256 with an additional $N_{a}=50$ diffusion steps for the proposed methods. The learning rate is $2e^{-4}$ with $2e^{6}$ training steps. The training of the model takes about 10 hours on a Nvidia RTX-3090 GPU. More parameters are provided in the attached code: “safediffuser/config/maze2d.py”. The switch of different (proposed) methods in testing can be modified in “safediffuser/diffuser/models/diffusion.py” through “GaussianDiffusion.p_sample()” function.

In Fig. [7](#S10.F7), we present a diffusion procedure using the diffuser, in which case the generated trajectory can easily violate safety constraints. Using the proposed robust-safe diffuser, the generated trajectory can guarantee safety, but some points on the trajectory may get stuck in local traps, as shown in [8](#S10.F8). Using the proposed relaxed-safe diffuser and time-varying-safe diffuser, the local trap problem could be addressed.

Figure: Figure 7: Maze planning (blue to red) denoising diffusion procedure with diffuser (Left to right: diffusion time steps 256, 4, 3, 0, respectively). Red ellipse and superellise (outside) denote safe specifications. Both specifications are violated with the trajectory from diffuser.
Refer to caption: /html/2306.00148/assets/figs/maze/diff/0.png

Figure: Figure 8: Maze planning (blue to red) denoising diffusion procedure with robust-safe diffuser at diffusion time step 0. Red ellipse and superellise (outside) denote safe specifications. Although with safety guarantees, some trajectory points may get stuck in local traps.
Refer to caption: /html/2306.00148/assets/figs/maze/260.png

### 10.2 Safe Planning for Robot Locomotion

For robot locomotion (in MuJoCo), we wish the robot to avoid collisions with obstacles, such as the roof. In this case, since there is no local trap problem, we only consider robust-safe diffuser (RoS-diffuser). Others work similarly. The training data set is publicly available from Janner et al. ().

Specifications. The simple safety specification for both the Walker2D and Hopper is collision avoidance with the roof. In other words, the height of the robot head $z\in\mathbb{R}$ should satisfy the following constraint:

$$ $z\leq h_{r},$ (19) $$

where $h_{r}>0$ is the height of the roof. We also need to normalize $h_{r}$ according to the normalization of the state $z$ in the diffusion model.

The complex safety specification for both the Walker2D and Hopper is a speed-dependent collision avoidance constraint:

$$ $z+\varphi v_{z}\leq h_{r},$ (20) $$

where $\varphi>0$, $v_{z}\in\mathbb{R}$ is the speed of the robot head along the $z$-axis. The speed-dependent safety constraint is more robust for the robot to avoid collision with the roof since when the robot jumps faster, we need to ensure a larger safe distance with respect to the roof in order to account for all kinds of uncertainties or perturbations. In this case, the simple truncation method is hard to work since it is not clear how to truncate both $z$ and $v_{z}$ at the same time.

Model setup, training and testing. The diffusion model structures are the same as the open source ones (Walker2D-Medium-Expert-v2 and Hopper-Medium-Expert-v2) provided in Janner et al. (). We set the planning horizon as 600, the diffusion steps as 20. The learning rate is $2e^{-4}$ with $2e^{6}$ training steps. The training of the model takes about 16 hours on a Nvidia RTX-3090 GPU. More parameters are provided in the attached code: “safediffuser/config/locomotion.py”. The switch of different methods in testing can be modified in “safediffuser/diffuser/models/diffusion.py” through “GaussianDiffusion.p_sample()” function.

### 10.3 Safe Planning for Manipulation

For manipulation (in Pybullet), the diffusion models generate joint trajectories (as controls) for the robot, which are conditioned on the locations of the objects to grasp and place. The training data set is publicly available from Janner et al. (). Specifications are joint limitations to avoid collision in joint space.

Specifications. The simple safety specification for the robot is in the joint space, and we are trying to limit the joint angles of the robot within allowed ranges:

$$ $x_{min}\leq x\leq x_{max},$ (21) $$

where $x\in\mathbb{R}^{7}$ is the state of 7 joint angles, $x_{min}\in\mathbb{R}^{7}$ and $x_{max}\in\mathbb{R}^{7}$ denotes the minimum and maximum joint limits.
We need to normalize the limits according to how the state $x$ is normalized in the diffusion model.

The complex safety specifications are speed-dependent joint constraints:

$$ $x_{min}\leq x+\varphi v\leq x_{max},$ (22) $$

where $\varphi>0$, $v\in\mathbb{R}^{7}$ is the joint speed corresponding to the joint angle $x$. In this example, since the diffusion model does not directly predict $v$, we evaluate $v$ using $x(k)$ and $x(k+1)$ along the planning horizon. The joints limits are also normalized as in the simple specification case.

Model setup, training and testing. The diffusion model structure is the same as the open source one provided in Janner et al. (), and we use their pre-trained models to evaluate our methods when comparing with other approaches.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1) Are "local traps" = local minima?
2) Isn't it a big problem that the method gets stuck in "local traps" between 39% and 100% of the time for some of the benchmarks? What does this mean in practice?
3) What is trap rate 1 vs. trap rate 2 in the table?
4) Could you comment on why you chose to use CBF here instead of e.g. lagrangian objectives common in safe RL?
5) Why three methods instead of the most general, or one that picks the best solution approach?
soundness: 2
strengths: - Safety guarantees are important for some applications and of interest to the safe planning community
- This may be the first (only?) safety method for diffusion planning.
summary: The authors propose a method to use control barrier functions (CBFs) with diffusion planning, enabling safety guarantees. The results are demonstrated on a 2D maze problem, Walker2D and a manipulation problem from related work.
weaknesses: - Some parts are a bit difficult to read and have unusual terminology (e.g., "local traps"?) and the paper contains some minor grammar and cosmetic issues which make the quite complex approach harder to understand.
- The method is actually three methods, and the user is to pick the one most suitable for their problem. This seems a bit complicated, why not go for the most general?

Example presentation issues (I recommend you do another polish pass):
- General: excessive underlining
- 470: typo: physical simulator
- 474: typo: can work in general safety guarantees

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. In terms of data and experimental results, could you clarify the rationale for selecting D4RL over [2], which is specifically designed for Safe Offline RL?
2. Can you clarify the motivation of applying CBFs in the diffusion denoising process steps instead of trajectory planning steps? It was not very clear.
3. Why were the experiments conducted solely in Maze2D-large? Could you clarify the choice of tasks for the locomotion experiments? Specifically, why was only the Medium-Expert variant used for the locomotion environment? What about Cheetah?
4. Could you elaborate on the generalization capabilities of the model when encountering new safety constraints beyond those specified during training? It would be valuable to understand how well the approach adapts to previously unseen constraints.


[2] Liu, Zuxin, et al. "Datasets and benchmarks for offline safe reinforcement learning." arXiv preprint arXiv:2306.09303 (2023).
soundness: 3
strengths: 1. The proposed method introduces a novel formulation of Control Barrier Functions (CBFs) within diffuser-family models, ensuring that safety constraints are satisfied throughout the diffusion denoising process.
2. The authors provide a systematic approach for selecting the appropriate variant based on the safety constraints.
3. Results demonstrate improvements in safety-related metrics while maintaining comparable performance in the downstream task.
4. The method can be trained using data that violates safety constraints, yet during testing, it reliably generates safe trajectories.
summary: The authors present SafeDiffuser, a modified version of the Diffuser model proposed by Janner et al. (2022), adapted to ensure the satisfaction of safety constraints. They introduces "Finite-Time Diffusion Invariance," leveraging Control Barrier Functions (CBFs) within the denoising diffusion process to enforce specified safety constraints. The authors propose three approaches for implementing their methodology—ROS, RES, and TVS—that demonstrate competitive performance compared to baseline methods, achieving significantly fewer constraint violations. Experiments were conducted across Maze2D-large, locomotion, and manipulation tasks, validating the effectiveness of the proposed approaches in diverse settings.
weaknesses: 1. The proposed methods require solving Quadratic Programs (QPs) at each diffusion step, which is computationally inefficient.
2. [Minor Weakness] The approach requires predefined safety constraints. However, I think that this limitation is relatively minor, such a setup is not a major weakness.
3. It would strengthen the paper if the authors included comparisons with other methods, such as CDT [1], which also employ sequential modeling techniques to address offline RL problems with safety constraints.

[1] Liu, Zuxin, et al. "Constrained decision transformer for offline safe reinforcement learning." *International Conference on Machine Learning*. PMLR, 2023.

Despite the weaknesses, I find the formulation and results are enough to have the paper above the acceptance threshold.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: - The forward invariance in control theory (equation 2) assumes that the control system is governed by Lipschitz functions. Is any such requirement not required for extending the definition to diffusion neural networks?
    
- Diffusion dynamics is estimated online. How does error in this estimation affect the guarantees and performance in practice?
    
- In theorem 3.2 what is the intuitive meaning of the robust term?
    
- If would be interesting to continually retrain the diffusion models on the generated safe data. Can this improve performance or speed up safediffuser?
soundness: 3
strengths: - Diffusion models are an important and effective class of conditional generative models. However, existing methods for guidance do no provide safety guarantees. Hence, the propose algorithm is an important contribution.
    
- Applying CBF-based technique to diffusion models is novel to my knowledge. In particular, modifying the denoizing steps seem to be more effective than generating safe trajectories through classifier-free guidance.
    
- Proposed extension to handle local traps are effective.
    
- Experimental results in simulation demonstrate the effectiveness and robustness of the algorithm.
summary: This paper presents control barrier function-based methods to enforce safety constraints on diffusion models. The main idea is to modify the diffusion denoizing step to satisfy safety constraints within a finite number of steps (finite-time diffusion invariance). This is done by formulating a quadratic program to compute a perturbation that steers the generated trajectory towards the safe set in every denoizing step. Evaluations in simulated maze and robotics domains show that the proposed method is superior to baselines in satisfying safety constraints at the cost of longer planning time.
weaknesses: - Safety specifications have to be differentiable.
    
- SafeDiffuser is significantly slower (by an order of magnitude) than vanilla diffusion.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Regarding to the design choice, the TVS diffuser seems to be the most general safediffuser. Is it possible to further simplify the forward invariance equation (theorem 3.4) by fully discarding the reliance on b, and directly using sigma? For example, imagine sigma at t=0 converges to exactly b. This sigma can be used in analogy to the time-varying CBF and should impose a more relaxed restriction to the intermediate samples. I guess it would need clever design of sigma (or even learning-based), but do you think it is worth the effort?
2. Another question is the motivation of using CBF. From first principle, all we care about here is the safety at the final timestep, ie, t=0. It sounds more similar to a goal-reaching/liveness problem, where you don’t need to enforce every intermediate sample satisfy the safety. Compared to control Lyapunov function or reachability-based method, why is CBF used for the problem? Applying the other two concepts seem more natural to me.
3. In control, CBF can usually be compared to potential-field based method to show their effective in terms of avoiding obstacle in a longer time horizon rather than reactively avoiding. Similarly, is it possible to include [1] as a baseline to show the approach’s effectiveness? 

[1] Luo, Y., Sun, C., Tenenbaum, J. B., & Du, Y. (2024). Potential based diffusion motion planning. arXiv preprint arXiv:2407.06169.
soundness: 2
strengths: The approach maintains the model performance while adding safety guarantees.
summary: The paper introduces SafeDiffuser, a method to add safety guarantees to diffusion models when used for planning tasks. The key innovation is incorporating control barrier functions (CBFs) into the diffusion process to ensure generated trajectories satisfy safety constraints.
weaknesses: 1. The definition of diffusion invariance seems too strict. Essentially we only care about the safety at t=0, and trajectories satisfying t=0 safety don’t necessarily need to satisfy t=1 safety. However the method actively encourages the intermediate trajectories to be safe. This doesn’t mean the method is wrong, but I’m hoping to see a softer way to guarantee the safety. See Q2.
2. Along with the above weakness, the design choice seems too complex. There’re $\epsilon$, $w$, $r$, $\gamma$, $\sigma$, all of them seem hyper-parameters that need careful tuning.
3. CBF itself is meant to design for dynamics that are (1) time-invariant; and (2) deterministic. The diffusion process doesn’t belong to any of these two parts. For (1), the alpha bar changes over time steps. For (2), most of the diffusion sampler constantly adds noise to the dynamics (unless the approach is specifically using DDIM). Applying CBF-based technique to the diffusion seems counter-intuitive. 
4. Scaling up seems challenging. I’m hoping to see a highly nonlinear dynamics like drone/quadcopter to see whether the method could scale up. Another challenge for scaling up is from the selection criteria. Page 6 discusses the principle to select different safediffusers. As mentioned, it requires the unsafe set to be known to determine the convexity. This seems challenging for high-dimensional control tasks.

Meta-review: The paper proposes SafeDiffuser, a framework that integrates control barrier functions (CBFs) into diffusion processes to ensure the safety of diffusion-based planners. The model operates by using a quadratic program during each step of the denoising process to identify a perturbation that guides the generated trajectory towards the safety set. The paper evaluates SafeDiffuser alongside contemporary baselines in terms of task performance and adherence to safety constraints on various robot planning tasks.

The paper was reviewed by four referees, who agree on several of the paper's key strengths and weaknesses. Among its strengths, the ability to ensure that diffusion-based planners adhere to safety constraints becomes increasingly important as more and more people use diffusion models for planning. Further, at least two reviewers note the novelty in the paper's integration of CBFs within diffusion models. The experiments support SafeDiffuser's ability to ensure the safety of the planned trajectories, while maintaining performance that is comparable to traditional diffusion-based planners. However, several reviewers raised concerns about SafeDiffuser's computational complexity due to the need to solve a quadratic program at each diffusion step. Notably, the authors derived a closed-form solution of the model and re-ran all of the experiments, the result of which reveals that the computational cost is not significantly higher than that of the vanilla Diffuser model. Additionally, several reviewers called into question the difficulty of implementing SafeDiffuser due to the need to choose among the three variants of the model as well as the need to design the CBFs, which can be particularly challenging for systems with highly nonlinear dynamics. The authors addressed some of these concerns during the rebuttal period, proposing a unified version of SafeDiffuser and connected it to the three variants proposed in the original submission. Additionally, at least two reviewers questioned the validity of the assumption that the diffusion process is Lipschitz, which the authors clarified in their responses.

Overall, the authors put significant effort into revising the paper to address the reviewers' initial concerns. It is evident to the AC that this effort went a log way in clarifying the paper's contributions.

Decision: Accept (Poster)