DiffuSolve: Diffusion-Based Solver for Non-Convex Trajectory Optimization

## Abstract

Abstract Optimal trajectory design is computationally expensive for nonlinear and high-dimensional dynamical systems.
The challenge arises from the non-convex nature of the optimization problem with multiple local optima, which usually requires a global search.
Traditional numerical solvers struggle to find diverse solutions efficiently without appropriate initial guesses.
In this paper, we introduce DiffuSolve , a general diffusion model-based solver for non-convex trajectory optimization.
An expressive diffusion model is trained on pre-collected locally optimal solutions and efficiently samples initial guesses, which then warm-starts numerical solvers to fine-tune the feasibility and optimality.
We also present DiffuSolve+ , a novel constrained diffusion model with an additional loss in training that further reduces the problem constraint violations of diffusion samples.
Experimental evaluations on three tasks verify the improved robustness, diversity, and a 2 × \times × to 11 × \times × increase in computational efficiency with our proposed method, which generalizes well to trajectory optimization problems of varying challenges.

## 1 Introduction

Optimal trajectory design is fundamental in decision-making for autonomous agents.
In the open-loop case, with a predetermined initial configuration, the goal is to plan an optimal path and control sequence for an agent to reach a target while satisfying the dynamics and safety constraints.
In the case of complex nonlinear system dynamics and environmental constraints, these problems often have non-convex structures.
For example, within a chaotic dynamical system like the Circular Restricted Three-Body Problem (CR3BP) (Koon et al., ),
the trajectory optimization problem for the spacecraft has multiple local optima of different qualitative behaviors with various tradeoffs (Hartmann et al., ; Russell, ).
Identifying these diverse, locally optimal, and feasible solutions efficiently remains a significant challenge in the field.

Control transcription of the open-loop trajectory optimization problem involving nonlinear functions results in a nonlinear program (NLP), where an initial guess is required to get the solution (Betts, ).
When one is interested in solving for several solutions, a two-step global search is needed: (i) sampling a distribution on the control space, and (ii) using this sample as an initial guess to the NLP solver that hopefully generates a solution.
For each solution from the NLP, optimality (and also feasibility) is often verified by the solver by checking whether the first-order necessary conditions, the Karush-Kuhn-Tucker (KKT) conditions, are satisfied (Kuhn & Tucker, ; Karush, ).
However, the high dimensionality and non-convex nature of the problem often make the two-step global search process time-consuming such that the efficiency demands of autonomous systems are
not met.

Figure: Figure 1: Three trajectory optimization tasks in our experiments (from left to right): two-car reach-avoid problem; quadrotor navigation problem; cislunar transfer problem. The dimensions of the decision variables are 81, 241, and 64, respectively.
Refer to caption: extracted/5897449/figures/experiment_intro.png

Machine learning has also gained popularity in trajectory optimization, but primarily focuses on solving the convexified problem rather than learning the solution distribution to the original non-convex problem.
For example, investigations have been made on warm-starting Quadratic Programs (QP) using neural networks (Chen et al., ; Cauligi et al., ; Sambharya et al., ).
Recently, generative models, particularly diffusion models, have shown good performance in generating trajectories or controls from a distribution (Janner et al., ; Ajay et al., ; Chi et al., ; Carvalho et al., ; Sridhar et al., ).
When conditioned on the environmental parameters, diffusion models can efficiently generate appropriate and diverse trajectories even in an a priori unknown environment.
However, the diffusion model alone without the NLP solver may output trajectories that violate the dynamic and safety constraints, causing catastrophic damage to safety-critical systems.
It also lacks the ability to verify the optimality of the sampled trajectories.
While efforts have been made to integrate safety constraints into generative models (Li et al., ; Chang et al., ; Yang et al., ; Xiao et al., ; Power et al., ; Botteghi et al., ), it is still challenging to achieve solution quality, diversity, feasibility, and computational efficiency at the same time.

In this paper, we propose DiffuSolve: a diffusion model-based solver for non-convex trajectory optimization with improved efficiency, diversity, and robustness.
Our contribution is two-fold:
First, we train a conditional diffusion model to learn the complex solution distribution of non-convex optimization problems, and efficiently sample a set of diverse and high-quality trajectory predictions.
These samples are then used as warm starts for an NLP solver to derive locally optimal and feasible solutions, ensuring the robustness of DiffuSolve.
Second, to mitigate the effect of diffusion samples that violate the problem constraints, we developed DiffuSolve+ with a novel constrained diffusion model that incorporates an additional constraint violation loss in the training process.

DiffuSolve has the following appealing properties:
(a). Computational Efficiency. The diffusion model samples high-quality solutions that are close to local optima as warm starts, from which the NLP solver can quickly converge.
(b). Robustness and Diversity. A constrained diffusion model produces diverse samples with minimal constraint violations, further refined by an NLP solver for enhanced feasibility and optimality.
(c). Automatic Data Generation. With an NLP solver, we can generate diverse problems and collect corresponding solutions ourselves, allowing for on-demand data augmentation whenever required.
(d). Generalization. Our framework is suitable for trajectory optimization across various system dynamics and environment settings, with the potential for optimization problems in other areas like finance (Sambharya et al., ), etc.

We demonstrate the efficacy of our method on three non-convex trajectory optimization problems: two-car reach-avoid, quadrotor navigation, and cislunar low-thrust transfer for a spacecraft.

## 2 Related Work

##### Machine Learning for Optimization.

Machine learning methods have been widely used to solve a variety of optimization problems with different techniques, including convex Quadratic Programming (QP) (Zeilinger et al., ; Zhang et al., ; Chen et al., ; Sambharya et al., ), mixed-integer programming (Cauligi et al., ), combinatorial optimization (Sun & Yang, ; Huang et al., ) and black-box optimization (Kumar & Levine, ; Trabucco et al., ; Krishnamoorthy et al., ), etc.
For the Combinatorial Optimization problem, graph-based diffusion models are used to directly generate solutions (Sun & Yang, ; Huang et al., ).
To improve the constraint satisfaction in optimization problems, neural networks have been incorporated with fully differentiable constraint function (Donti et al., ).
Conditional variational autoencoder (CVAE) and long short-term memory (LSTM) are proposed to warm-start the general nonlinear trajectory optimization problems (Li et al., ).
Another branch of work leverages machine learning methods and pre-collected solution data points for solving partial differential equations (PDEs) (Cai et al., ). One of the most pupular work is physics-informed neural networks (PINN) (Raissi et al., ), which introduces the solution prediction model with a hybrid training loss for labeled solution prediction and violation of the original PDEs.

##### Diffusion Models.

The diffusion models (Sohl-Dickstein et al., ; Song et al., ) is able to sample the multi-modal distribution with a stochastic differential equation (SDE). The models are further developed into denoising diffusion probabilistic models (DDPM) (Ho et al., ), denoising diffusion implicit models (DDIM) (Song et al., ), etc., for application of image generation. Both classifier-guided diffusion (Dhariwal & Nichol, ) and classifier-free guidance (Ho & Salimans, ) are proposed to achieve conditional generation with diffusion models.

##### Diffusion-based Trajectory Generation with Constraints.

Diffusion models are becoming popular in robotics for generating controls or trajectories (Ajay et al., ; Janner et al., ; Liang et al., ; Sridhar et al., ; Chi et al., ; Mishra et al., ; Ding et al., ).
Many efforts have been made to improve the diffusion model for creating safe trajectories for safety-critical systems.
Control barrier functions (Xiao et al., ; Botteghi et al., ) and collision-avoidance kernels (Chang et al., ) are employed to guide diffusion models to sample feasible trajectory solutions.
To adapt to new constraints in testing, prior work investigated composing diffusion models to improve the generalizability (Power et al., ; Yang et al., ).

Different from existing work, our paper focuses on high-dimensional, highly nonlinear trajectory optimization problems with many local optima, and aims to find diverse solutions efficiently.
Instead of incorporating constraints in the sampling process, our constrained diffusion model in DiffuSolve+ includes a novel loss in training, which can be combined with the current work to further reduce the violation.
Moreover, DiffuSolve and DiffuSolve+ are equipped with an NLP solver that finetunes the diffusion samples until feasibility and optimality are obtained at its best.

## 3 Preliminaries

##### Non-Convex Trajectory Optimization.

We consider a general open-loop trajectory optimization problem: assume a known agent’s dynamical model, the predefined initial configuration, and the obstacle settings, we aim to plan an optimal trajectory and controls for the agent to reach a target while satisfying the dynamics and safety constraints.
With the direct transcription (Betts, ), this trajectory optimization problem can be discretized as the following nonlinear program $\mathcal{P}_{y}$:

$$ $\displaystyle\mathcal{P}_{y}\coloneqq\begin{cases}\underset{x}{\min}\quad&J(x; y)\\ s.t.,\quad&g_{i}(x;y)\leq 0,\ i=1,2,...,l\\ \quad&h_{j}(x;y)=0,\ j=1,2,...,m\\ \end{cases}$ (1) $$

where $x\in\mathcal{X}\subset\mathbb{R}^{n}$ is the decision variable to be optimized, and $y\in\mathcal{Y}\subset\mathbb{R}^{k}$ represents various problem parameters.
$J\in C^{1}(\mathbb{R}^{n},\mathbb{R}^{k};\mathbb{R})$ is the objective function.
Each $h_{i}\in C^{1}(\mathbb{R}^{n},\mathbb{R}^{k};\mathbb{R})$ and $g_{j}\in C^{1}(\mathbb{R}^{n},\mathbb{R}^{k};\mathbb{R})$ are equality and inequality constraint functions, respectively.
In trajectory optimization, $x$ might include a sequence of control $u$, the time variable $t$ required to reach the target, etc.
Parameters $y$ can include control limits, target and obstacle characteristics such as position and shape, and other relevant factors.
The objective function $J$ can be time-to-reach, fuel expenditure, etc.
The constraint functions $g$ and $h$ could contain requirements for dynamical models, goal-reaching, obstacle avoidance, etc.

When the problem parameter $y$ is fixed, $\mathcal{P}_{y}$ in Eq. (1) typically presents a non-convex optimization problem with multiple local optima.
To solve this problem $\mathcal{P}_{y}$, a gradient-based NLP solver $\pi$ can be used.
A local optimum (feasible) $x^{*}$ to $\mathcal{P}_{y}$ is verified by the solver $\pi$ that satisfies the KKT conditions with no constrained violation.
To find different $x^{*}$, the user first provides multiple initial guesses $x^{0}$ for the decision variables $x$ to the solver $\pi$, typically drawn from a uniform distribution or some heuristic approach.
Then the solver $\pi$ performs an iterative, gradient-based optimization until a local optimum $x^{*}$ is found.
The popular methods to conduct such gradient-based optimization in non-convex problems include the Interior Point Method (IPM) (Wright, ) and the Sequential Quadratic Programming (SQP) (Boggs & Tolle, ).

##### Diffusion Probabilistic Model.

As expressive generative models, the diffusion probabilistic models (Sohl-Dickstein et al., ; Ho et al., ; Song et al., ) generate samples from random noises through an iterative denoising process. The forward process (*i.e.*, diffusion process) iteratively adds Gaussian noise to sample $x_{k}$ at the $k$-th step according to the following equation:

$$ $q(x_{k+1}|x_{k})=\mathcal{N}(x_{k+1};\sqrt{1-\beta_{k}}x_{k},\beta_{k}\mathbf{ I}),\quad 0\leq k\leq K-1.$ (2) $$

where $\beta_{k}$ is a given variance schedule.
With the diffusion steps $K\rightarrow\infty$, $x_{K}$ becomes a random Gaussian noise. From above one can write:

$$ $x_{k}=\sqrt{\widebar{\alpha}_{k}}x_{0}+\sqrt{1-\widebar{\alpha}_{k}}\varepsilon,$ (3) $$

where $\widebar{\alpha}_{k}=\prod_{k^{\prime}=1}^{K}(1-\beta_{k^{\prime}})$ and $\varepsilon\sim\mathcal{N}(0,\mathbf{I})$ is a random sample from a standard Gaussian.
The data generation process is the reverse denoising process that transforms random noise into data sample:

$$ $p_{\theta}(x_{k-1}|x_{k})=\mathcal{N}\big{(}x_{k-1};\mu_{\theta}(x_{k}),\sigma _{k}\mathbf{I}\big{)},\quad 1\leq k\leq K$ (4) $$

with initial noisy data sampled from a random Gaussian distribution $x_{K}\sim\mathcal{N}(0,\mathbf{I})$. $p_{\theta}$ is the parameterized sampling distribution, and it can be optimized through an alternative $\varepsilon_{\theta}(x_{k},k)$ which is to predict $\varepsilon$ injected for $x_{k}$, for every diffusion step $k$. For conditional generation modeling, a conditional variable $y$ is added to both processes $q(x_{k+1}|x_{k},y)$ and $p_{\theta}(x_{k-1}|x_{k},y)$, respectively. The classifier-free guidance (Ho & Salimans, ) can be applied to further promote the conditional information, which learns both conditioned and unconditioned noise predictors as $\varepsilon_{\theta}(x_{k},k,y)$ and $\varepsilon_{\theta}(x_{k},k,\varnothing)$. Specifically, $\varepsilon_{\theta}$ is optimized with the following loss function:

$$ $\mathcal{L}_{\text{diff}}=\operatorname{\mathbb{E}}_{(x_{0},y)\sim\mathcal{X} \times\mathcal{Y},k,\varepsilon,b}\left\|\varepsilon_{\theta}\Big{(}x_{k}(x_{0 },\varepsilon),k,(1-b)\cdot y+b\cdot\varnothing\Big{)}-\varepsilon\right\|^{2} _{2}$ (5) $$

where $x_{0}$ and $y$ are sampled from groundtruth data,
$\varepsilon\sim\mathcal{N}(0,\mathbf{I})$,
$k\sim\text{Uniform}(1,K)$,
$b\sim\text{Bernoulli}(p_{\text{uncond}})$ with given unconditioned probability $p_{\text{uncond}}$,
and $x_{k}$ follows Eq. (3).

## 4 Methodology

We first propose DiffuSolve, a general framework for efficient and robust non-convex trajectory optimization using diffusion models and NLP solvers.
Then we introduce a novel constrained diffusion model specifically designed to guide the diffusion models to generate feasible solutions, which is included in DiffuSolve+.

### 4.1 DiffuSolve Framework

Figure: Figure 2: The DiffuSolve framework for solving non-convex trajectory optimization problems.
Refer to caption: extracted/5897449/figures/workflow.png

It has been observed that for similar trajectory optimization problems $\mathcal{P}_{y}$ in Eq. (1), their solutions $x^{*}$ often exhibit similar structures (Amos et al., ; Li et al., ).
Similar problems are those $\mathcal{P}_{y}$ with the same objective $J$ and constraints $g,h$, but with different parameters $y$.
Thus, the key insight of DiffuSolve is to use a diffusion model to learn $p(x^{*}|y)$, the conditional probability distribution of locally optimal solutions based on pre-solved similar problems.
Then the diffusion model can generalize and predict diverse solutions $\tilde{x}_{\text{new}}^{*}$ to new scenarios where the condition $y_{\text{new}}$ value is unseen.
However, diffusion models inevitably make prediction errors, which can result in constraint violations. To address this, we use diffusion samples $\tilde{x}_{\text{new}}^{*}$ as initial guesses for the NLP solver, which fine-tunes them to derive the final solutions $x_{\text{new}}^{*}$, increasing both feasibility and optimality. This approach acts like a safety net, enhancing the robustness of the solutions.

We illustrate our proposed DiffuSolve framework in Fig. 2.
In the offline data generation process, we use an NLP solver $\pi$ to collect locally optimal and feasible solutions $x^{*}$ for various problem instances $\mathcal{P}_{y}$ with uniformly sampled initial guesses.
The solutions with sub-optimal objective values are filtered out manually and the remaining dataset is used as the training data.
For DiffuSolve training, we use a conditional diffusion model to learn $p(x^{*}|y)$ with classifier-free guidance (Ho & Salimans, ), with the loss function in Eq. 5.
In the online testing process when a new problem $\mathcal{P}_{y_{\text{new}}}$ is presented with a priori unknown value $y_{\text{new}}$, we first use the diffusion model to predict diverse solution candidates $\tilde{x}_{\text{new}}^{*}$ as initial guesses.
Then the NLP solver $\pi$ only requires minor adjustments to derive the final solutions $x_{\text{new}}^{*}$ with improved feasibility and optimality.
This warm-starting approach is fully parallelizable for both diffusion sampling and the solving process of $\pi$.

### 4.2 DiffuSolve+ with Constrained Diffusion Model

The original diffusion models in DiffuSolve aim to learn the distribution of locally optimal solutions $p(x|y),x\in\mathcal{X},y\in\mathcal{Y}$ with $\mathcal{X}$ as the variable space and $\mathcal{Y}$ as the problem parameter space. DiffuSolve is optimized through the loss function as Eq. (5) without knowing any constraint information.
As a result, the sampled solutions could be close to the groundtruth but still largely violate the constraints.
To address this issue, we propose an improved DiffuSolve+ method with a constrained diffusion model (CDM) to minimize the constraint violations in the sampled trajectories.
Inspired by the equation loss in PINN (Raissi et al., ), we define a violation function $V(x,y):\mathcal{X}\times\mathcal{Y}\rightarrow\mathbb{R}$ for differentiable constraints in Eq. (1):

$$ $\displaystyle V=\sum_{i=1}^{l}\max(g_{i},0)+\sum_{j=1}^{m}|h_{j}|,$ (6) $$

where $V(\cdot,\cdot)\in PC^{1}(\mathbb{R}^{n},\mathbb{R}^{k};\mathbb{R})$ (piecewise differentiable except at $g_{i}=0$ or $h_{j}=0$) maps from a sample $x$ and parameter $y$ and outputs the total constraint violation as a scalar value.
In fact, each constraint term can be further customized with different weights that allow different treatments.
The violation value is expected to decrease during the training process of diffusion models.
We introduce the newly proposed loss function and the corresponding training process as follows.

##### DiffuSolve+ Training.

For the diffusion process, we try to generate samples from solution set $x_{0}\in\mathcal{X}$.
By reformulating the forward sampling process as in Eq. (3), we have $x_{0}=\frac{x_{k}-\sqrt{1-\widebar{\alpha}_{k}}\varepsilon}{\sqrt{\widebar{
\alpha}_{k}}},\varepsilon\sim\mathcal{N}(0,\mathbf{I})$.
The noise $\varepsilon$ is approximated as the neural-network parameterized model $\varepsilon_{\theta}(x_{k},k,y)$ for condition $y$ at timestep $k$. Replacing it into previous formula, the clean sample at timestep $k=0$ is predicted as $\widehat{x}_{0}=\frac{x_{k}(x_{0},\varepsilon)-\sqrt{1-\widebar{\alpha}_{k}}
\varepsilon_{\theta}(x_{k},k,y)}{\sqrt{\widebar{\alpha}_{k}}}$. The benefits of direct $\widehat{x}_{0}$ prediction is that it does not need iterative diffusion sampling thus avoids gradient backpropagation through multi-step sampling.
For constrained diffusion generation with DiffuSolve+, we introduce a constraint violation loss $\mathcal{L}_{\text{vio}}$ on the predicted $\widehat{x}_{0}$ from $x_{k}$, $\forall k\in[K]$:

$$ $\displaystyle\mathcal{L}_{\text{vio}}$ $\displaystyle=\mathbb{E}_{(x_{0},y)\sim\mathcal{X}\times\mathcal{Y},k\in[K], \varepsilon\sim\mathcal{N}(0,\mathbf{I})}\bigg{[}\frac{1}{k}V\Big{(}\text{clip }\big{(}\widehat{x}_{0}(x_{k},\varepsilon_{\theta}),x_{0}^{\text{min}},x_{0}^{ \text{max}}\big{)},y\Big{)}\bigg{]}$ (7) $$

where the clip function preserves the range of $\widehat{x}_{0}$ to be within the lower $x_{0}^{\text{min}}$ and upper bound $x_{0}^{\text{max}}$ for $x_{0}\in\mathcal{X}$, ensuring the valid calculation of violation function $V(\cdot,\cdot)$.
The reason for the clipping is that at the early phase of the training, the parameterized $\varepsilon_{\theta}$ as an approximation of $\varepsilon$ is not accurate thus the predicted $\widehat{x}_{0}$ may not lie on the original range of the data $x_{0}$.
The scheduling factor $\frac{1}{k}$ regulates the scale of the violation function $V$ for different $x_{k},k\in[K]$, since the predicted $\widehat{x}_{0}$ will be more noisy when $k$ is large and the estimation of violation function can be less reliable.
This may not be a theoretically principled choice but is found effective and convenient in our experiments.

For the violation loss $\mathcal{L}_{\text{vio}}$, we always let the condition variable $y$ appear in the noise prediction $\varepsilon_{\theta}(x_{k},k,y)$ without masking it out, contrary to the probabilistic masking in the original training loss $\mathcal{L}_{\text{diff}}$.
An intuitive interpretation of the additional loss term is that it guides the diffusion model training with the gradients of minimizing the violation value for each denoising step.

Finally, our proposed constrained diffusion training loss is a combination of the original diffusion model training loss in DiffuSolve as Eq. 5 and the above violation loss:

$$ $\displaystyle\mathcal{L}=\mathcal{L}_{\text{diff}}+\lambda\mathcal{L}_{\text{ vio}}$ (8) $$

where $\lambda$ is a hyperparameter for adjusting the strength of $\mathcal{L}_{\text{vio}}$.
As the training proceeds, the original loss function $\mathcal{L}_{\text{diff}}$ will encourage $\widehat{x}_{0}$ to be close to true $x_{0}$, and $\mathcal{L}_{\text{vio}}$ further enforces the constraint satisfaction of the generated samples.

##### Conditional Diffusion Sampling.

For sampling from trained diffusion models, we adopt guided sampling with classifier-free guidance as introduced in Sec. 3. The guided prediction noise with guidance weight $c$ is:

$$ $\displaystyle\widehat{\varepsilon}_{\theta}\leftarrow c\cdot\varepsilon_{ \theta}(x_{k},k,y)+(1-c)\cdot\varepsilon_{\theta}(x_{k},k,\varnothing)$ (9) $$

Then we plug $\widehat{\varepsilon}_{\theta}$ into $\varepsilon_{\theta}$ in the sampling process (Ho et al., ; Ho & Salimans, ), and through iterative sampling $x_{k-1}$ from $x_{k}(k=K,\dots,1)$ with initial random noise $x_{K}\sim\mathcal{N}(0,\mathbf{I})$, it produces the generated samples $x_{0}$. The diffusion sample serves as the predicted initial guess in Sec. 4.1 for a new task: $\tilde{x}^{*}_{\text{new}}:=x_{0}$.

## 5 Experiment

In this section, we first evaluate DiffuSolve with a vanilla diffusion model (DM) on three high-dimensional and highly nonlinear trajectory optimization tasks.
Secondly, we present the results of DiffuSolve+ with our novel constrained diffusion model (CDM).
As a standalone method, the CDM reduces problem constraint violation of the samples compared to DM.
Within DiffuSolve+, CDM helps further improve the efficiency of obtaining locally optimal and feasible solutions compared to DiffuSolve.

The experiments mainly involve three stages: 1. Use an NLP solver to collect solutions on each task domain; 2. Train the data-driven models to fit the solution dataset; 3. For unseen new tasks within each domain, sample the initial guesses from each model that warm-start the solver to derive final solutions.
The details of data collection and model training refer to Appendix A.

### 5.1 Task setup

Our experiments are conducted on three complex non-convex trajectory optimization tasks as illustrated in Figure 1.
These tasks span diverse domains and present unique challenges:
(i). Two-Car Reach-Avoid: Multi-agent game-theoretical problem is difficult to scale and features multiple equilibria.
(ii). Quadrotor Navigation: High-dimensional nonlinear dynamical systems lead to computational inefficiency.
(iii). Cislunar Transfer: The highly nonlinear chaotic dynamical system makes spacecraft trajectory optimization time-consuming with numerous local optima.
The detailed dynamical settings for each task are as follows.
The numerical settings can be seen in Appendix A.

##### Two-Car Reach-Avoid.

This is a two-player game inspired by (Chen et al., ). Each car tries to find a minimum-time path to reach the goal while avoiding collision with another car and obstacles, shown in Figure 1.
Specifically, both car1 and car2 follows the same dynamics as follows:

$$ $\displaystyle\dot{p}_{x}=v\cos{\theta},\quad\dot{p}_{y}=v\sin{\theta},\quad \dot{v}=a,\quad\dot{\theta}=\omega,\quad$ $$

where $(p_{x},p_{y},v,\theta)$ denotes the state of each car.
$p_{x},p_{y}$ are the position, $v$ is the speed, and $\theta$ is the orientation of the car.
Each car has two-dimensional controls $u^{1}=(a^{1},\omega^{1}),u^{2}=(a^{2},\omega^{2})$, where $a\in[-1,1]$ is the acceleration and $\omega\in[-1,1]$ is the angular speed.

##### Quadrotor Navigation.

This a navigation problem of a 10D quadrotor, inspired by (Herbert et al., ).
The quadrotor needs to find the minimum time to reach the goal position while avoiding obstacles shown in Figure 1, where the goal and obstacles are randomly sampled.
The quadrotor’s dynamics is defined as follows:

$$ $\displaystyle\dot{x}$ $\displaystyle=v_{x},$ $\displaystyle\dot{v}_{x}$ $\displaystyle=g\tan\theta_{x},$ $\displaystyle\dot{\theta}_{x}$ $\displaystyle=-d_{1}\theta_{x}+\omega_{x},$ $\displaystyle\dot{\omega}_{x}$ $\displaystyle=-d_{0}\theta_{x}+n_{0}a_{x},$ $\displaystyle\dot{z}$ $\displaystyle=v_{z},$ $\displaystyle\dot{y}$ $\displaystyle=v_{y},$ $\displaystyle\dot{v}_{y}$ $\displaystyle=g\tan\theta_{y},$ $\displaystyle\dot{\theta}_{y}$ $\displaystyle=-d_{1}\theta_{y}+\omega_{y},$ $\displaystyle\dot{\omega}_{y}$ $\displaystyle=-d_{0}\theta_{y}+n_{0}a_{y},$ $\displaystyle\dot{v}_{z}$ $\displaystyle=K_{T}a_{z}-g$ (10) $$

where $x,y,z$ are the position of the quadrotor.
$\theta_{x}$, $\theta_{y}$ are the pitch and roll angle, and $\omega_{x}$, $\omega_{y}$ are the corresponding rates.
$v_{x},v_{y},v_{z}$ are the speed of the quadrotor.
$d_{0},d_{1},K_{T},n_{0}$ are system parameters, and $g=9.81$.
The controls are $u=(a_{x},a_{y},a_{z})$.

##### Cislunar Transfer.

In this task, we consider a minimum-fuel cislunar low-thrust transfer with the dynamical model to be a Circular Restricted Three-Body Problem (CR3BP) (Koon et al., ).
As shown in Figure 1, the example trajectory (orange) starts from the end of a geostationary transfer spiral (blue) and ends at an arc of the invariant manifold (black).
Assuming the mass of the spacecraft is negligible, the CR3BP describes the equation of motion of a spacecraft under the gravitational force from the Earth and moon.
Let $m_{1}$ be the mass of the Earth and $m_{2}$ be the mass of the moon, and $\mu=m_{2}/(m_{1}+m_{2})$, the dynamics of the spacecraft in the CR3BP is as follows:

$$ $\displaystyle\ddot{x}-2\dot{y}=-\widebar{U}_{x},\quad\ddot{y}+2\dot{x}=- \widebar{U}_{y},\quad\ddot{z}=-\widebar{U}_{z},$ $$

where $\widebar{U}(r_{1}(x,y,z),r_{2}(x,y,z))=-\frac{1}{2}\left((1-\mu)r_{1}^{2}+\mu r
_{2}^{2}\right)-\frac{1-\mu}{r_{1}}-\frac{\mu}{r_{2}}$ is the effective gravitational potential, $r_{1}=\sqrt{(x+\mu)^{2}+y^{2}+z^{2})}$ and $r_{2}=\sqrt{(x-(1-\mu))^{2}+y^{2}+z^{2})}$ are the distance from the spacecraft to the sun and moon, respectively.
This is a chaotic dynamical system.

### 5.2 Experiment Setup

##### Numerical Solvers.

We assume fully known dynamics and environment for the open-loop trajectory optimization problem.
For all three tasks, we formulate and solve the optimization problem with Sparse Nonlinear OPTimizer (SNOPT) (Gill et al., ) as the solver $\pi$, which uses the SQP method and supports warm-starting.
For the cislunar transfer, we adopt pydylan, a Python interface of the Dynamically Leveraged Automated (N) Multibody Trajectory Optimization (DyLAN) (Beeson et al., ) to formulate the CR3BP dynamics in addition to SNOPT.
The detailed problem formulation and solving for each task are included in Appendix A.

##### Baselines.

For comparison with the proposed DiffuSolve, we introduce the following baselines:

- •
Uniform: This method solely uses an NLP solver with uniformly sampled initial guesses $x\in\mathcal{X}$ for the non-convex trajectory optimization problem.
It solves problems from scratch without leveraging any problem-specific prior information.
- •
Optimal Uniform: This method uniformly samples from collected locally optimal solutions $\{x^{*}\}$ as initial guesses for the NLP solver.
It establishes a performance lower bound for data-driven methods since without learning it cannot generalize to new problems.
- •
CVAE_LSTM: This method (Li et al., ) combines the convolutional variational auto-encoder (CVAE) with long short-term memory (LSTM) to learn the conditional distribution of locally optimal solutions.
Specifically, a CVAE with Gaussian Mixture Model (GMM) prior is first used to sample the non-control variables, e.g. time or mass variables, and then an LSTM is adapted to sample the control sequence.

Although there exist graph-based methods for path planning, such as the Rapidly-exploring Random Trees (RRT) (LaValle, ), A^∗ (Hart et al., ), etc., they do not aim to produce optimal solutions, nor handle the case of nonlinear dynamical systems in complex environmentally constrained problems well.
Thus, the comparison against them is not provided in our experiments.

##### Diversity Measure.

We adopt the Vendi Score (Friedman & Dieng, ) as a quantitative metric for measuring the diversity of the model samples.
The similarity kernel we choose is $k(x,y)=e^{-||x-y||}$.

### 5.3 DiffuSolve Results

In this section, we evaluate DiffuSolve’s performance with a vanilla diffusion model (DM) against baseline methods, focusing on solution optimality, feasibility, diversity, and computational efficiency.

#### 5.3.1 Optimality and Feasibility

We generate 600 samples from each method and use them as warm starts for the NLP solver.
In Table 1, we present the percentage (ratio) of locally optimal and feasible solutions over 600 samples which are fine-tuned by the NLP solver.
Note that before finetuning, this ratio is zero for all methods’ raw samples.
The optimality and feasibility are verified by checking the KKT condition and the constraint violation based on the task setup in Sec. 5.1.
As shown in Table 1, no method is able to directly generate locally optimal and feasible solutions.
When using the samples as warm starts for the NLP solver, our proposed DiffuSolve obtains a greater number of locally optimal and feasible solutions than any non-diffusion methods.

We also visualize the sampled trajectories and the corresponding solver-derived trajectories in Figure 3.
For the Uniform and CVAE_LSTM method, the generated trajectories are far from feasible, thus many of them cannot serve as good initial guesses for the NLP solver.
DiffuSolve already generates near-feasible trajectories and thus makes NLP solver easier to derive locally optimal and feasible solutions.

**Table 1: Percentage (%) (ratio) of locally optimal and feasible solutions over 600 samples.**
| Task | Method |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| DiffuSolve+ | DiffuSolve | CVAE_LSTM | Uniform | Optimal Uniform |  |
| Two-car | 94.83 | 94.50 | 63.17 | 64.17 | 83.00 |
| Quadrotor | 100.00 | 99.83 | 99.67 | 99.17 | 98.50 |
| Cislunar | – | 54.00 | 29.17 | 17.50 | 23.33 |

Figure: Figure 3: In two-car reach-avoid problem: First row: raw samples (no solver); Second row: corresponding solver derived locally optimal and feasible solutions.
Refer to caption: extracted/5897449/figures/solver_refined.png

#### 5.3.2 Computational Efficiency

For the locally optimal and feasible solutions obtained in Table 1, Table 2 presents the computational time statistics, involving both sampling time from the model and the NLP solver time.
In Table 2, the DiffuSolve with a vanilla DM is 2 $\times$ to 11 $\times$ faster than the uniform method based on the top $25\%$-quantile of the computational time for deriving the locally optimal and feasible solutions.
Also in the histogram plots in Figure 5, DiffuSolve obtains more solutions than baseline methods within a short period of time.
These results coincide with Figure 3 that since the DiffuSolve generates samples that are close to local optima, it takes a shorter time for the NLP solver to converge.

Thus, it motivates us to set a proper cut-off time for the solver in the future, which can allow more initial guesses to be fed.
This could further improve the efficiency of DiffuSolve to derive locally optimal and feasible solutions.

**Table 2: Computational time (sampling + solving) statistics and Vendi Score.**
| Task | Method | Solving Time | Vendi Score |  |  |
| --- | --- | --- | --- | --- | --- |
| Mean($\pm$STD) | 25%-Quantile | Median |  |  |  |
| Two-car | DiffuSolve+ (Ours) | 17.86 $\pm$ 14.28 | 7.68 | 13.83 | 3873.60 |
| DiffuSolve (Ours) | 18.82 $\pm$ 14.33 | 8.77 | 15.61 | 3574.66 |  |
| CVAE_LSTM | 36.24 $\pm$ 20.55 | 19.45 | 33.26 | 1162.72 |  |
| Uniform | 46.17 $\pm$ 20.63 | 29.62 | 43.54 | 5256.75 |  |
|  | Optimal Uniform | 25.15 $\pm$ 19.75 | 10.82 | 20.89 | 3708.58 |
| Quadrotor | DiffuSolve+ (Ours) | 6.65 $\pm$ 3.55 | 4.47 | 6.05 | 5458.23 |
| DiffuSolve (Ours) | 7.30 $\pm$ 4.79 | 4.78 | 6.62 | 5428.63 |  |
| CVAE_LSTM | 9.05 $\pm$ 8.30 | 5.00 | 6.74 | 309.18 |  |
| Uniform | 16.87 $\pm$ 13.60 | 9.28 | 12.48 | 5985.10 |  |
|  | Optimal Uniform | 13.94 $\pm$ 12.08 | 7.66 | 10.13 | 5575.83 |
| Cislunar | DiffuSolve (Ours) | 50.81 $\pm$ 70.30 | 9.97 | 26.68 | 5999.96 |
| CVAE_LSTM | 141.31 $\pm$ 125.60 | 38.41 | 106.48 | 1403.38 |  |
| Uniform | 199.34 $\pm$ 116.59 | 114.02 | 174.35 | 6000.00 |  |
| Optimal Uniform | 177.35 $\pm$ 123.22 | 70.03 | 153.97 | 5918.08 |  |

#### 5.3.3 Diversity

In this paper, our proposed DiffuSolve aims to learn the solution distribution of the non-convex problem, instead of predicting a single solution.
For example, for the two-car reach-avoid problem, there are multiple equilibrium points that are locally optimal and feasible.
In Figure 4, we present 4 qualitatively different solutions with DiffuSolve for each task, showing its ability to generate diverse solutions even within a shorter period of time.

The Vendi Score is also evaluated on the samples from each method in Table 2.
While the Uniform method certainly generates the most diverse samples, the DiffuSolve is comparable to Optimal Uniform and outperforms CVAE_LSTM in sample diversity.
Especially for the quadrotor navigation, though the solving time for CVAE_LSTM is competitive in Table 2, the Vendi Score shows that its generated solutions are not diverse at all.

Figure: Figure 4: Diverse trajectory solutions for two-car reach-avoid, quadrotor navigation and cislunar transfer problem with DiffuSolve method, given the same conditional input.
Refer to caption: extracted/5897449/figures/diverse_traj.png

### 5.4 DiffuSolve+ Results

In this section, we present the results of DiffuSolve+ on the quadrotor navigation and two-car reach-avoid task.
The proposed CDM is evaluated both as a standalone method for reducing constraint violations and within the DiffuSolve+ framework for improving computational efficiency.

Since the current cislunar transfer task originally adopted from (Beeson et al., ) uses an adaptive-stepsize integrator for the dynamics constraint, it requires extra effort to incorporate it into the neural network and is beyond our scope.
We leave the CDM for this task as future work.

#### 5.4.1 Constraint violation

As a standalone method without the NLP solver, our proposed CDM from Sec. 4.2 has fewer constraint violations compared to the vanilla DM and CVAE_LSTM, displayed in Table 3.
For the quadrotor navigation, the CDM is able to directly generate feasible solutions more than any other methods.
Also in Figure 3 for the two-car reach-avoid problem, the CDM generates a trajectory that is the closest to the locally optimal and feasible solutions - only car 1 cannot reach the goal but it is already near.
It will be easy for the NLP solver to close this gap within DiffuSolve+.

**Table 3: Left: The constraint violation of the model samples. Right: The ratio of feasible solutions. Both are measured directly on the model samples without the NLP solver. Here DiffuSolve+: constrained diffusion model, DiffuSolve: vanilla diffusion model**
| Task | Method | Constraint Violation (No Solver) | Feasible Ratio ‰ |  |  |
| --- | --- | --- | --- | --- | --- |
| Mean($\pm$STD) | 25%-Quantile | Median | (No solver) |  |  |
| Two-car | DiffuSolve+ | 3.00 $\pm$ 15.36 | 0.73 | 1.44 | 0 |
| DiffuSolve | 10.99 $\pm$ 35.59 | 1.98 | 4.52 | 0 |  |
| CVAE_LSTM | 282.72 $\pm$ 186.40 | 147.09 | 292.90 | 0 |  |
| Quadrotor | DiffuSolve+ | 3.06 $\pm$ 7.47 | 0.10 | 0.38 | 81.8 |
| DiffuSolve | 3.85 $\pm$ 8.52 | 0.16 | 0.63 | 49.6 |  |
| CVAE_LSTM | 20.19 $\pm$ 22.32 | 3.36 | 12.96 | 16.3 |  |

#### 5.4.2 Computational Efficiency

For DiffuSolve+ with the proposed CDM, the computational efficiency is further improved as shown in Table 2.
The intuition is clear from Figure 3: when the DiffuSolve+ sample is closer to the local optima, it takes the NLP solver an even shorter time to derive the final solution.
We also present the histogram of the computational time in Figure 5.
Here we visualize the density of locally optimal and feasible solutions obtained within a time range, where DiffuSolve+ in red is able to obtain more solutions than DiffuSolve within a short period of time from 600 initial guesses.

Figure: (a) Two-car reach-avoid.
Refer to caption: extracted/5897449/figures/car_histogram.png

## 6 Conclusion and Discussion

This paper presents DiffuSolve: a general diffusion model-based solver for non-convex trajectory optimization. It can generate locally optimal, feasible, and diverse solutions with high computational efficiency.
We also propose DiffuSolve+ which includes a novel constrained diffusion model.
It has fewer constraint violations as a standalone method and further improves the computational efficiency when integrated with DiffuSolve+.
It is also a general acceleration framework that has the potential to solve optimization problems in other areas beyond trajectory optimization such as finance, computational chemistry, etc.

Limitations also exist for the current paper.
We don’t spend extra effort to optimize the diffusion model’s sampling speed.
As a result, the current DiffusSolve and CDM may not be able to achieve real-time optimization for high-dimensional systems or highly dynamic environments.
This can be improved with existing acceleration techniques such as stride sampling, DDIM (Song et al., ), etc.
In addition, although most constraint functions in trajectory optimization are differentiable (almost everywhere), the current CDM does not work for non-differentiable violation functions.
For non-differentiable constraints, transforming them into differentiable functions or applying approximation with appropriate bump functions or neural networks is possible.

## Appendix A Implementation and Task Details

### A.1 Model Details

For both vanilla and constrained diffusion models (DMs), we apply a UNet structure with three hidden layers of 512, 512, and 1024 neurons.
We also use a fully connected layer of 256 and 512 neurons to embed the conditional input $y$.
The sampling step is set to be 500.
Both DMs are trained with 200 epochs using the Adam optimizer (Kingma & Ba, ).
The constrained and unconstrained DMs usually take 14 hours and 9 hours respectively as well as 20 GB VRAM to train on one NVIDIA A100 GPU, for every 100k training data with batch size of 512.

### A.2 Two Car Reach-Avoid

##### Problem formulation and solving.

In this task $\mathcal{P}_{y}$, we aim to minimize the time for the two cars to reach each own goal while avoiding colliding with the other car and obstacles.
We fixed the start $p_{\text{start}}$ and goal $p_{\text{goal}}$ position for car1 to be $(0,10)$ and $(10,0)$, and for car2 to be $(10,0)$ and $(0,0)$.
The conditional parameters $y\in\mathbb{R}^{6}$ include the two obstacle positions $p_{\text{obs}}$ randomly sampled from $[2.0,8.0]$, and the obstacle radius $r_{\text{obs}}$ randomly sampled from $[0.5,1.5]$. We formulate this trajectory optimization problem with a forward shooting method, with the discretized time step $T=20$.
The variable to optimize for each car is $x=(t_{\text{final}},u^{1}_{1},u^{1}_{2},...,u^{1}_{T},u^{2}_{1},u^{2}_{2},...,
u^{2}_{T})\in\mathbb{R}^{41}$, where $u^{1}_{i}$ and $u^{2}_{i}$ are two control variables for each cars.
The problem parameter is $y=(p_{\text{obs}},r_{\text{obs}})$.
The objective is defined as $J(x;y)=t_{\text{final}}$.
The constraint $g$ includes the goal-reaching constraints at $t_{\text{final}}$, and the collision avoidance for obstacles and between each car for all time.
We use the 4-th order Runge-Kutta (RK4) method to integrate the dynamics and formulate the constraints.

##### Data collection and model training.

In this task, we collect locally optimal solutions $x^{*}$ from 3k different $y$, each with 100 uniformly sampled initial guesses using SNOPT (Gill et al., ).
We filter out those solutions whose objective is higher than 12 s and use the remaining 114k solutions as the training data.
Within this 114k data, 10% of the data are used as the validation set.
The whole data collection process takes around 8 hours with 200 AMD EPYC 9654 CPU cores.

We train each of our proposed constrained and unconstrained DMs and baseline models with 3 different random seeds and test the warm-starting performance with 600 initial guesses sampled from 3 random seeds across 20 unseen obstacle settings, i.e. unseen $y$ values.

### A.3 Quadrotor Navigation

##### Problem formulation and solving.

In this task $\mathcal{P}_{y}$, we aim to minimize the time for the quadrotor to reach the goal while avoiding the obstacles.
The start position is fixed at $(-12,0,0)$, and the reference goal position is $(12,0,0)$.
The conditional parameter $y$ includes the perturbation $d_{\text{goal}}$ randomly chosen from $[-2,2]$ that is added to the goal position, 4 obstacle positions $p_{\text{obs}}$ randomly sampled between the start and goal, and the obstacle radius $r_{\text{obs}}$ randomly sampled from $[1.5,3.5]$. We discretize this problem into $T$ timesteps where $T=80$.
The variable to optimize is $x=(t_{\text{final}},u_{1}^{1},..,u_{T}^{1},u_{1}^{2},...u_{T}^{2},u_{1}^{3},..
.u_{T}^{3})\in\mathbb{R}^{241}$, and the conditional parameter is $y=(d_{\text{goal}},p_{\text{obs}},r_{\text{obs}})\in\mathbb{R}^{17}$.
The objective is defined as $J(x;y)=t_{\text{final}}$.
The constraint $g$ includes the goal-reaching constraints at $t_{\text{final}}$ and the collision avoidance for obstacles at all time.
We use the 4-th order Runge-Kutta (RK4) method to integrate the dynamics and formulate the constraints.

##### Data collection and model training.

In this task, to collect the training data we sample 4000 different $y$, each with 50 uniformly sampled initial guesses.
Then we use the SNOPT (Gill et al., ) solver for each problem instance to collect locally optimal solutions.
Finally, we filter out those solutions with objective values greater than 4.57 s and use the remaining 179k data as training data.
Within this 179k data, 10% of the data are used as the validation set.
The whole data collection process takes around 20 hours with 200 AMD EPYC 9654 CPU cores.

We train each of our proposed constrained and unconstrained DMs and baseline models with 3 different random seeds.
The models are tested for the warm-starting performance with 600 initial guesses from 3 random seeds across 20 unseen locations of the goal and obstacles, i.e. unseen $y$ values.

### A.4 Cislunar Transfer

##### Problem formulation and solving.

In this problem $\mathcal{P}_{y}$, as shown in Fig. 1, the spacecraft is planned to start from a Geostationary Transfer Spiral (blue) and reach a stable manifold arc (green) of a Halo orbit near $L_{1}$ Lagrange point (green).
A candidate trajectory solution is plotted in orange.
The CR3BP is a chaotic system, where a smaller perturbation will lead to a significantly different final trajectory.
Therefore, trajectory optimization of this problem contains many local optima.
We choose the conditional parameter $y\in[0.1,1.0]$ Newton to be the maximum allowable thrust of the spacecraft.

We use the forward-backward shooting method to formulate the trajectory optimization problem in Eq. (1).
For the spacecraft, we set the constant specific impulse (CSI) $I_{sp}=1000$ s. The initial dry mass is 300 kg and the initial fuel mass is 700 kg.
We choose $T$ discretized control segments where $T=20$ and the control $u$ are uniformly applied on each segment.
The variable $x$ to optimize is as follows:

$$ $\displaystyle x=(t_{\text{burn}},t_{\text{coast}}^{\text{initial}},t_{\text{ coast}}^{\text{final}},m_{f},u_{1},u_{2},...,u_{N})\in\mathbb{R}^{64}$ (11) $$

where $t_{\text{burn}},t_{\text{coast}}^{\text{initial}},t_{\text{coast}}^{\text{
final}}$ are the effective burning time for the engine, initial and final coast time, $m_{f}$ is the final mass, and $u_{i}$ is a three-dimensional thrust control of the spacecraft.
The objective is defined as $J(x;y)=-m_{f}$, which is to minimize fuel expenditure.
The constraint $g$ is the midpoint agreement when integrating dynamics from forward and backward.
The problem parameter $y$ is the maximum allowable thrust for the spacecraft and is bounded by $[0.1,1]$ Newton.
To integrate the dynamics, we use the Runge-Kutta-Fehlberg 5-4th order (RK54) method (Fehlberg, ).

##### Data collection and model training.

To collect the training data, we sample 12 different $y$ values from a grid in $[0.1,1]$ Newton, and for each $y$ we collect 25k locally optimal solutions with $m_{f}\geq 415kg$ total amount of 300k data.
The problem is solved using pydylan, a python interface of the Dynamically Leveraged Automated (N) Multibody Trajectory Optimization (DyLAN) (Beeson et al., ) and the solver SNOPT (Gill et al., ).
Within this 300k data, 10% of the data are used as the validation set.
The whole data collection process takes around 3 days with 500 Intel Skylake CPU cores.

We train the unconstrained DM and baseline models on 3 random seeds, and we test the warm-starting performance with 600 initial guesses sampled from 3 random seeds on $y=0.15$ Newton, which is unseen in training data.

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. Are there any tasks that can illustrate the advantage of DiffuSolve+ over the DiffuSolve when combined with NLP solvers (maybe more safety critical applications)?

2. In the related work, the authors mention several works of diffusion-based trajectory generation with constraints. Why they are not included in the experiment baselines?

3. What value is assigned to the hyperparameter $\lambda$  during the experiments? An ablation study on this hyperparameter may help better understanding the contribution of the proposed loss function.
soundness: 2
strengths: 1. The paper is clear and well-written. 

2. The use of diffusion models helps better warm-up the trajectory optimization solvers.
summary: This paper proposes to use diffusion models (DMs) to enhance the nonlinear program (NLP) solvers for trajectory optimization. The authors first collect local-optimal trajectories of related tasks to train a DM, then use the samples generated from the DM to warmup the NLP solver. They also derive a new loss function to enforce the DM to consider feasibility during sampling. The experiment results from 3 constrained trajectory optimization tasks demonstrate that the proposed DiffuSolve and Diffusolve+ generate better initial samples compared to non diffusion-based baselines, reducing the solving time of NLP solvers.
weaknesses: 1. The authors basically directly incorporate DMs in the DiffuSolve pipeline, where the new proposed loss function seems contribute marginally when combined with NLP solvers. From Table 1, DiffuSolve+ produces only two additional feasible solutions in the first task and one more in the second task compared to the original DiffuSolve.

2. The demonstrated effectiveness of DiffuSolve relies on in-distribution, near-optimal data derived from tasks with similar parameters. It would be intersting to  assess the generalization capabilities of DiffuSolve when the distribution of training parameters differs from the distribution encountered during online testing.

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 3
questions: 1) In line 272, how are values for $\lambda$ determined? 
2) I have also raised some questions in the weaknesses section. I would certainly appreciate a response to it!
3) All the current simulation results are being tested against "CVAE_LSTM" and some uniform samples. The paper doesn't mention, but is "CVAE_LSTM" the best-known baseline comparison? The experimental results might seem more meaningful if they were more robustly tested against a larger variety of algorithms from the literature...
soundness: 4
strengths: 1) The paper is well-written - the technical details are explained clearly, and the experimental results are fairly presented.
2) The paper demonstrates good simulation results in three diverse settings.
summary: The paper presents two algorithms: DiffuSolve and DiffuSolve+. In DiffuSolve, a conditional diffusion model is first trained to learn $p(x^*|y)$ with a loss function derived from Ho & Salimans, 2022. An offline data generation process uses a nonlinear program solver, $\pi$, to collect locally optimal and feasible solutions $x^*$ for problem instances $P_y$ with uniformly sampled initial guesses. It manually filters out samples corresponding to suboptimal objectives. The diffusion model predicts diverse guesses $\tilde{x}^*_{new}$, which the NLP solver $\pi$ then uses to derive final solutions. Intuitively, this warm-starting approach with diverse samples helps find a global minimizer in the presence of multiple local minima, making it amenable to nonconvex optimization. The paper also develops DiffuSolve+ which ensures that no constraints are violated. For this, the algorithmic insight is to use a “closed-form” prediction of the sample $\hat{x}_0$, while minimizing a “clipped” objective which intuitively keeps predicted solutions within the feasible region.  The paper then runs three experiments: quadrotor navigation, two-car reach avoid, and cislunar transfer, and compares its algorithms against the baselines to demonstrate speed and efficacy results.
weaknesses: 1) The term "control transcription" (line 37) is not defined and is not used anywhere else in the paper. I think readers would appreciate it if you were to remove it if it is unnecessary, or if you were to explain it further, otherwise.

2) This is merely a pedantic point, but $\epsilon_{\theta}$ in lines 171 and 175 are being used for two things: one is a function of two variables, and the other is a function of three (not to mention there is an $\epsilon$ without a subscript!). I think this is fine, but maybe add a note saying “with abuse of notation”, or use a different variable for more clarity?

3) I just read "Model-Based Diffusion for Trajectory Optimization" (Pan, Yi, Chi, Qu, NeurIPS 24), which seems to have the same problem formulation and a similar algorithmic approach. Upon a closer look, I think there is a significant amount of overlap in the work.

4) While the literature review presented in the submission seems to touch on much of the existing work in the area, I'm not convinced that the submission contains any meaningful novelty. For instance, as the authors mention, it has been known that diffusion models are amenable to nonconvex optimization. In this sense, constructing a diffusion model and passing it to an optimizer is not novel, per se, which makes me question the contribution of DiffuSolve. On the other hand, DiffuSolve+ presents a nontrivial improvement with the clipping function to prevent violations; however, I wonder if there isn't already existing work in the literature to address preventing the infeasibility of solutions... In this regard, it would have been good to know the limitations of existing works in this area, which the literature review fails to highlight. For instance, the above reference (Pan et. al. 2024) solves some related problems in non-convex high-dimensional trajectory optimization and further references many other works, specifically on non-convex non-smooth high-dimensional trajectory optimization problems.

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: To understand the contributions of different components in the "DiffuSolve" and "DiffuSolve+" frameworks, I suggest the authors try to generate the initial guesses with other existing approaches for a diffusion-based trajectory with constraints (in addition to "CVAE_LSTM"). Adding additional comparison benchmarks will help us understand whether most of the improvement comes from the proposed training method/loss functions or the idea of fine-tuning a diffusion model's guess by the NLP solver.
soundness: 3
strengths: Planning optimally in a nonlinear dynamical system with constraints is a difficult but critical problem for many applications, such as robotics and autonomous driving. It is good to see that a diffusion model can help solve the problem more efficiently and with a higher success rate. The improvement in the experiment results looks significant.
summary: This paper proposes a diffusion-based solver for solving the optimal trajectory in a nonlinear dynamical system with constraints. Traditional nonlinear programs (NLP) rely on the initial guess to find locally optimal and feasible solutions. The authors propose a framework called “DiffuSolve” that first samples high-quality solutions as warm starts and then uses the NLP solvers to refine them. The goal of the training process is to make the diffusion model generate diverse samples with minimal constraint violations and be able to generalize across different dynamics and constraints.
weaknesses: My major concern is the novelty and the significance of “DiffuSolve” compared with previous works on diffusion-based trajectory generation with constraints. A straightforward idea is to use previous approaches to sample the warm-start trajectory and then use the NLP solver to fine-tune. In Related Works, the authors claim the difference is that this work focus on “high-dimensional”, “highly nonlinear”, and “diverse solutions”. Further, the design of the violation function (6) helps “DiffuSolve+” achieve better performances in problem instances with the above features. I do not see sufficient evidence on benchmark comparison to support these claims.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 3
questions: see above.
soundness: 2
strengths: see above.
summary: The manuscript presents an algorithm for generating optimal trajectories for nonlinear dynamical systems. The presentation is mostly clear and the writing is OK. I have two major concerns:

1) Novelty

The manuscript suggests to use diffusion models to generate initial guesses for parametric nonlinear optimization problems. The initial guess is then refined by applying a nonlinear programming solver that also guarantees constraint satisfaction. Both building blocks (diffusion models) and NLP solvers have been used in the context of trajectory planning and combining the two, while certainly useful for solving an important applied problem, brings very little novelty/creativity. 

As a result, I believe that the manuscript should maybe be better published at a robotics conference, rather than a machine learning venue.


2) Experiments

The experiments are somewhat limited (in particular, in view of the limited novelty). The planning problems that are considered have essentially about eight states (Two-Car Reach-Avoid), ten states (quadrotor navigation), six states (cislunar transfer), which is low dimensional. While nonconvex, the problems are also smooth and well-behaved (no contacts, friction, discontinuities, etc.). For context, similar planning problems are routinely solved in engineering, for more than twenty years (e.g. SNOPT dates back to 2005 and has been routinely used since then).
The diffusion model needs to be trained on a dataset that contains numerous solutions to the parametric program for different sets of parameters and initial conditions. The manuscript does not seem to consider the computational effort needed to assemble the training data when presenting the result, which would mean that the results are heavily skewed towards the authors' approach. In addition, from the presented Table (e.g. Table 2), I cannot see a 10-fold decrease in solution time, as claimed by the authors.


Summary: The manuscript is certainly interesting to read, however falls short of providing significant conceptual novelty or the solution of an open real-world problem in a convincing way.
weaknesses: see above.