Learning Multiple Initial Solutions to Optimization Problems

## Abstract

Abstract Sequentially solving similar optimization problems under strict runtime constraints is essential for many applications, such as robot control, autonomous driving, and portfolio management. The performance of local optimization methods in these settings is sensitive to the initial solution: poor initialization can lead to slow convergence or suboptimal solutions. To address this challenge, we propose learning to predict multiple diverse initial solutions given parameters that define the problem instance. We introduce two strategies for utilizing multiple initial solutions: (i) a single-optimizer approach, where the most promising initial solution is chosen using a selection function, and (ii) a multiple-optimizers approach, where several optimizers, potentially run in parallel, are each initialized with a different solution, with the best solution chosen afterward. We validate our method on three optimal control benchmark tasks: cart-pole, reacher, and autonomous driving, using different optimizers: DDP, MPPI, and iLQR. We find significant and consistent improvement with our method across all evaluation settings and demonstrate that it efficiently scales with the number of initial solutions required. The code is available at https://github.com/EladSharony/miso .

## 1 Introduction

Many applications, ranging from trajectory optimization in robotics and autonomous driving to portfolio management in finance, require solving similar optimization problems sequentially under tight runtime constraints (Paden et al., ; Ye et al., ; Mugel et al., ). The performance of local optimizers in these contexts is often highly sensitive to the initial solution provided, where poor initialization can result in suboptimal solutions or failure to converge within the allowed time (Michalska & Mayne, ; Scokaert et al., ). The ability to consistently generate high-quality initial solutions is, therefore, essential for ensuring both performance and safety guarantees.

Conventional methods for selecting these initial solutions typically rely on heuristics or warm-starting, where the solution from a previously solved, related problem instance is reused. More recently, learning-based solutions have also been proposed, where neural networks are used to predict an initial solution. However, in more challenging cases, where the optimization landscape is highly non-convex or when consecutive problem instances rapidly change, predicting a single good initial solution is inherently difficult.

To this end, we propose *Learning Multiple Initial Solutions (MISO)* (Figure 1), in which we train a neural network to predict *multiple* initial solutions.
Our approach facilitates two key settings: (i) a single-optimizer method, where a selection function leverages prior knowledge of the problem instance to identify the most promising initial solution, which is then supplied to the optimizer; and (ii) a multiple-optimizers method, where multiple initial solutions are generated jointly to support the execution of several optimizers, potentially running in parallel, with the best solution chosen afterward.

More specifically, our neural network receives a parameter vector that characterizes the problem instance and outputs $K$ candidate initial solutions. The network is trained on a dataset of problem instances paired with (near-)optimal solutions and is evaluated on previously unseen instances. Crucially, the network is designed not only to predict *good* initial solutions—those close to the optimal—but also to ensure that these solutions are sufficiently diverse, potentially spanning all underlying modes of the problem in hand. To actively encourage this multimodality, we implement training strategies such as a winner-takes-all loss that penalizes only the candidate with the lowest loss, a dispersion-based loss term to promote dispersion among solutions, and a combination of both.

We evaluate MISO across three distinct local optimization algorithms applied to separate robot control tasks: First-order Box Differential Dynamic Programming (DDP), which utilizes first-order linearization for the cart-pole swing-up task; Model Predictive Path Integral (MPPI) control, a sampling-based method, for the reacher task; and the Iterative Linear Quadratic Regulator (iLQR), a trajectory optimization algorithm, for an autonomous driving task.
Our results show that MISO significantly outperforms existing initialization methods that rely on heuristics, learn to predict a single initial solution or use ensembles of independently learned models.

In summary, our key contributions are as follows:

- 1.
We present a novel framework for predicting *multiple* initial solutions for optimizers.
- 2.
We introduce two distinct strategies for utilizing the predicted initial solutions: (i) *single-optimizer*, where the most promising solution is chosen based on a selection function, and (ii) *multiple-optimizers*, where multiple optimizers are initialized, potentially in parallel, with the best solution chosen afterward.
- 3.
We design and implement specific training objectives to prevent mode collapse and ensure that the predicted solutions remain multimodal.
- 4.
We apply our framework to three distinct sequential optimization tasks and perform extensive evaluation.

Figure: Figure 1: As opposed to previous works that predict a single initial solution, MISO trains a single neural network to predict multiple initial solutions. We use them to either initialize a single optimizer or jointly initialize multiple optimizers.
Refer to caption: x1.png

## 2 Related Work

#### Learning for optimization.

Advancements in machine learning have introduced numerous learning-based approaches to optimization problems (Sun et al., ). Early work by Gregor & LeCun () replaced components of classical convex optimization algorithms with neural networks.
More recent works aim to replace optimization methods entirely with end-to-end neural networks (OpenAI et al., ; Mirowski et al., ) or generate new optimization algorithms (Chen et al., ) for specific classes of problems. Other works enhance optimization-based control algorithms (Sacks & Boots, ), learn constraints (Fajemisin et al., ), or learn objective functions and system dynamics (Lenz et al., ; Wahlström et al., ; Tamar et al., ; Hafner et al., ; Nagabandi et al., ; Xiao et al., ).

#### Learning initial solutions.

Previous studies have proposed heuristic approaches to generate initial solutions for optimizers (Johnson et al., ; Marcucci & Tedrake, ). More recently, learning-based methods for initializing optimizers have gained attention in various fields, aiming to enhance both computational efficiency and resulting solutions quality.
In mixed-integer programming, neural networks have enhanced solver performance by predicting variable assignments (Nair et al., ), branching decisions (Sonnerat et al., ), and integer variables (Bertsimas & Stellato, ).
Baker () employed Random Forests to predict solutions for AC optimal power flow problems.
Kang et al. () utilized nearest neighbor search to warm-start tight convex relaxations in nonconvex trajectory optimization problems.
In robot control, neural networks were used to predict initializations for trajectory optimizers or Model Predictive Control (MPC) (Chen et al., ; Wang & Ba, ; Lembono et al., ).
An exciting line of recent work developed *differentiable* optimization algorithms, which allow jointly learning objectives, constraints, and initializations by backpropagating through the optimization process (Amos et al., ; East et al., ; Karkus et al., ; Sambharya et al., ). In contrast, we learn multiple initializations instead of one, and we do so without strong assumptions about the task or the optimizer.
Notably, Bouzidi et al. () used multiple initializations by repurposing a motion prediction model and Bézier curve fitting for a downstream MPC; however, this approach is specifically tailored for autonomous driving, incorporating a dedicated motion prediction module.

#### Parallel optimizers.

Leveraging parallelism has a long history in optimization research (Betts & Huffman, ). With recent advances in parallel computing hardware, such as GPUs, methods that execute multiple optimizers in parallel have also emerged.
For example, Sundaralingam et al. () introduced cuRobo, a GPU-accelerated method combining L-BFGS and particle-based optimization for robotic manipulators. Similarly, Huang et al. () utilized massive parallel GPU computation for efficient inverse kinematics and trajectory optimization.
de Groot et al. () proposed a topology-driven method that plans for multiple evasive maneuvers in parallel.
Barcelos et al. () focused on initializing parallel optimizers through rough paths. However, these works have not utilized learning. Lembono et al. () explored learning-based strategies for initializing trajectory optimizers based on a database of previous solutions and ensemble-learned models, particularly in manipulation and humanoid control tasks. In contrast, we propose a single neural network to generate multiple initializations, which, as shown in our experiments, significantly outperforms the ensemble-based approach.

## 3 Initializing optimizers

#### Problem setup.

In the most general form, we need to solve instances of a parameterized optimization problem,

$$ $\displaystyle\bm{x}^{\star}(\bm{\psi})=\arg\min_{\bm{x}}J(\bm{x};\bm{\psi}) \quad\mathrm{s.t.}\quad\bm{g}(\bm{x};\bm{\psi})\leq\bm{0};\quad\bm{h}(\bm{x}; \bm{\psi})=\bm{0},$ (1) $$

where $\bm{x}\in\mathbb{R}^{n}$ is the variable vector to be optimized, $J$ is the objective function, $\bm{g}$ and $\bm{h}$ are collections of inequality and equality constraints, and $\bm{\psi}\in\mathbb{R}^{m}$ is a parameter vector that defines the problem instance, e.g., parameters of the objective function and constraints that differ across problem instances.
A local optimization algorithm, $\bm{\mathrm{Opt}}$, attempts to find an optimum of $J$, namely,

$$ $\hat{\bm{x}}^{\star}=\bm{\mathrm{Opt}}(J,\bm{\psi},t_{\mathrm{lim}};\bm{x}^{ \mathrm{init}}),$ $$

where $\bm{x}^{\mathrm{init}}$ is initial solution provided to the optimizer, and $t_{\mathrm{lim}}$ is the runtime limit.

#### Heuristic methods.

A common choice of the initial solution, $\bm{x}_{\mathrm{init}}$, is the solution to a previously solved similar problem instance, referred to as a *warm-start*. For example, in optimal control the warm-start is typically the solution from the previous timestep, shifted and padded with zeros, $\bm{x}^{\mathrm{w.s.}}:=\{\{\bm{x}_{t+k}^{\mathrm{cand}}\}_{k=1}^{H-2},\bm{0}\}$ (Otta et al., ). This heuristic often works well in practice, however, it can struggle when large changes in the problem instance, $\bm{\psi}$, occur between consecutive time steps, leading to significant shifts in the optimal solution. For example, in autonomous driving, abrupt events like a traffic light switch or the sudden appearance of a pedestrian might drastically alter the reference trajectory or constraints. In such cases, the previous solution becomes a poor initialization, and the optimizer may fail to find a good solution within the allocated time frame.

## 4 Learning Multiple Initial Solutions ( \scalerel * O MISO)

The main idea of MISO is to train a single neural network to predict multiple initial solutions to an optimization problem, such that the initial solutions cover promising regions of the optimization landscape, eventually allowing a local optimizer to find a solution close to the global optimum. The key questions are then how to design a multi-output predictor; how to utilize multiple initial solutions in existing optimizers; and how to train the predictor to output a diverse set of initial solutions. In the following, we discuss our proposed solutions to these questions, illustrate the need for multimodality with a toy example, and discuss applications to optimal control.

### 4.1 Multi-output predictor

Our multi-output predictor is a neural network that takes the problem instance, $\bm{\psi}$, as input and outputs $K$ initial solutions for the optimization problem,

$$ $\{\bm{\hat{x}}_{k}^{\mathrm{init}}\}_{k=1}^{K}=\bm{f}(\bm{\psi};\bm{\theta}),$ $$

where $\bm{\theta}$ are the learned parameters of the network. We train the network on a dataset of problem instances and their corresponding (near-)optimal solutions, $\{(\bm{\psi}_{i},\bm{x}_{i}^{\star})\}_{i=1}^{n}$. Such dataset can be generated offline, for example, by running a slow yet globally optimal solver, or allowing the same local optimizer to run with longer time limits, potentially many times from different initial solutions.

### 4.2 Optimization with multiple initial solutions

We propose two distinct settings to leverage multiple initial solutions: *single-optimizer* and *multiple-optimizers*. The resulting frameworks are illustrated in Fig. 1.

#### Single optimizer.

In the single-optimizer setting we run a single instance of the optimizer with the most promising initial solution, $\bm{\hat{x}^{\star}}=\bm{\mathrm{Opt}}(J,\bm{\psi},t_{\textrm{limit}};\bm{\hat
{x}}^{\mathrm{init}})$. We introduce a selection function, $\bm{\Lambda}$, which, given a set of candidate solutions and the problem instance $\bm{\psi}$, returns the most promising candidate, $\bm{\hat{x}}^{\mathrm{init}}=\bm{\Lambda}(\{\bm{\hat{x}}_{k}^{\mathrm{init}}\}
_{k=1}^{K},\bm{\psi})$. A reasonable choice for $\bm{\Lambda}$ used in our experiments is selecting the candidate that minimizes the objective function the optimizer aims to minimize, i.e., $\bm{\Lambda}:=\arg\min_{k}J(\bm{\hat{x}}_{k}^{\mathrm{init}};\bm{\psi})$. Other possibilities include risk measures, metrics based on performance stability, robustness, exploration, or domain-specific metrics that align with the objectives of the overall task.

#### Multiple optimizers.

In the multiple-optimizers setting, we assume multiple instances of the optimizer can be executed in parallel. We then initialize each optimizer with a different initial solution,
$\bm{x}_{k}^{\star}=\bm{\mathrm{Opt}}_{k}(J,\bm{\psi},t_{\mathrm{limit}};\bm{
\hat{x}}_{k}^{\mathrm{init}}),\quad k\in\{1,\dots,K\}.$
To select a single solution from the outputs of the optimizers, we can use the same selection function $\bm{\Lambda}$, as in the previous case, e.g., the solution that minimizes the objective function.

Our framework can be trivially generalized to allow a different number of optimizers and initial solution predictions, as well as using a heterogeneous set of optimization methods. Further, to maintain performance guarantees, one may include the default, e.g., warm-start, solution as one of the considered initial solutions, which ensures that even with poor predictions, the final solution quality does not degrade.

### 4.3 Training strategies

The ultimate goal is to predict multiple initial solutions so that the downstream optimizer can find a solution close to the global optima, i.e., $J(\hat{\bm{x}}^{\star};\bm{\psi})\approx J(\bm{x}^{\star};\bm{\psi})$.
Training a neural network directly for this objective is not feasible in general. Instead, we propose proxy training objectives that combine two terms: a regression term that encourages outputs to be close to the global optimum, e.g., $\mathcal{L}_{\mathrm{reg}}(\bm{\hat{x}}_{k}^{\mathrm{init}},\bm{x}^{\star})=\|
\hat{\bm{x}}_{\mathrm{init}}-\bm{x}^{\star}\|$, where $\|\cdot\|$ is a distance metric; along with a *diversity* term that promotes outputs being different from each other, thereby covering various regions of the solution space. An illustrative example is in  Sect. 4.4.
In the following, we present three simple training strategies promoting diversity and preventing mode collapse.
We discuss alternative formulations, with probabilistic modeling and reinforcement learning, in Sect. 7.

#### Pairwise distance loss.

A simple method to encourage the model’s outputs to differ from each other is to penalize the pairwise distance between all outputs. The overall loss combines this dispersion-promoting term with the regression loss,

$$ $\mathcal{L}_{\mathrm{MISO-PD}}=\frac{1}{K}\sum_{k=1}^{K}\mathcal{L}_{\mathrm{ reg}}(\bm{\hat{x}}_{k}^{\mathrm{init}},\bm{x}^{\star})+\alpha_{K}\frac{1}{K} \sum_{k=1}^{K}\mathcal{L}_{\mathrm{PD},k}(\bm{\hat{x}}_{k}^{\mathrm{init}},\bm {x}^{\star}),$ $$

$$ $\mathcal{L}_{\mathrm{PD},k}=\frac{1}{K-1}\sum_{\begin{subarray}{c}k^{\prime}=1 \\ k^{\prime}\neq k\end{subarray}}^{K}\|\bm{\hat{x}}_{k}^{\mathrm{init}}-\bm{\hat {x}}_{k^{\prime}}^{\mathrm{init}}\|,$ $$

where $\alpha_{K}$ is a hyperparameter that balances the trade-off between accuracy and dispersion.

#### Winner-takes-all loss.

A more interesting way to encourage multimodality is to select the best-predicted output at training time and only minimize the regression loss for this specific prediction,

$$ $\mathcal{L}_{\mathrm{MISO-WTA}}=\min_{k}\{\mathcal{L}_{\mathrm{reg}}(\bm{\hat{ x}}_{k}^{\mathrm{init}},\bm{x}^{\star})\}.$ $$

Intuitively, the model only needs one of its outputs to be close to the ground truth, while the other predictions are not penalized for deviating, potentially aligning with different regions of the underlying distribution. Similar losses have been used, e.g., in multiple-choice learning (Guzman-Rivera et al., ). One advantage of this approach is that it is hyperparameter-free.

#### Mixture loss.

Lastly, we consider a combination of the previous two approaches to potentially enhance performance, as it provides some measure of dispersion we can tune,

$$ $\mathcal{L}_{\mathrm{MISO-MIX}}=\min_{k}\left\{\mathcal{L}_{\mathrm{reg}}(\bm{ \hat{x}}_{k}^{\mathrm{init}},\bm{x}^{\star})+\alpha_{K}\Phi\left(\mathcal{L}_{ \mathrm{PD},k}(\bm{\hat{x}}_{k}^{\mathrm{init}},\bm{x}^{\star})\right)\right\},$ $$

here, $\Phi$ is an upper-bounded function, such as $\mathrm{min}$ or $\mathrm{tanh}$, designed to limit the contribution of the pairwise distance term.

Beyond the losses above, MISO could be integrated with other training paradigms, such as reinforcement learning or probabilistic modeling. We discuss these options in Sect. 7 but differ investigation to future work.

### 4.4 Illustrative example

Figure: Figure 2: Top: The one-dimensional cost function $c(x)$ with global minima at $\bm{\mathrm{A}}$ and $\bm{\mathrm{C}}$ and a local minimum at $\bm{\mathrm{B}}$. Bottom: Predicted initial solution for different methods, demonstrating why explicitly promoting multimodality is important.
Refer to caption: x3.png

To illustrate the advantage of using a single model with multimodal outputs compared to regression models or ensembles of regressors, we examine a straightforward one-dimensional optimization problem aimed at minimizing the cost function $c(x)$ shown in Fig. 2 (top). The function features two global minima, denoted as $\bm{\mathrm{A}}$ and $\bm{\mathrm{C}}$, with a local minimum located between them at $\bm{\mathrm{B}}$.

Applying our learning framework to this simple problem, the dataset of optimal solutions includes instances of $\bm{\mathrm{A}}$ and $\bm{\mathrm{C}}$. A single-output regression model has no means to distinguish the two modes and inevitably learns to predict the mean of examples in the dataset, somewhere near $\bm{\mathrm{B}}$. Consequently, the local optimizer is likely to converge to the suboptimal local minimum at $\bm{\mathrm{B}}$. Constructing an ensemble of such models to generate multiple initial solutions does not mitigate this issue, as each ensemble member tends to be biased toward the mean of the two modes near $\bm{\mathrm{B}}$.
We implemented the optimization problem and showed the predictions for different training strategies in Fig. 2 (bottom). Details are in Appendix A.5. Indeed, an ensemble of single-output predictors fails to predict a global optimum, while our multi-output predictor succeeds with winner-takes-all and mixture losses.

While the problem considered here is purposefully simplistic, the existence of local minima is the key challenge in most optimization problems.

### 4.5 Application to optimal control

MISO is applicable to a broad class of sequential optimization problems; however, for the sake of evaluation, we focus on optimal control problems. Optimal control has a wide range of applications, e.g., in robotics, autonomous driving, and many other domains with strict runtime requirements, and due to the complexity induced by constraints and non-convex costs, local optimization algorithms are highly sensitive to the initial solution.

In optimal control the optimization variable $\bm{x}$ represents a trajectory defined as a sequence of states and control inputs over discrete time steps: $\bm{\tau}=\{\bm{s}_{t},\bm{u}_{t}\}_{t=1:H}$. Here, $\bm{s}_{t}\in\mathcal{S}$ and $\bm{u}_{t}\in\mathcal{U}$ denote the state and control input at time step $t\in\mathbb{Z}^{+}$, and $H\in\mathbb{Z}^{+}$ is the optimization horizon. The constraints involve adhering to the system dynamics $\bm{f}_{d}(\bm{s}_{t+1},\bm{s}_{t},\bm{u}_{t})=\bm{0}$, starting from an initial state $\bm{s}_{0}=\bm{s}_{\textrm{curr}}$, where $\bm{s}_{\text{curr}}$ represents the system’s current state. The problem instance parameters $\bm{\psi}$ encompass the initial state $\bm{s}_{0}$, and other domain-specific variables that parameterize the objective function or constraints, such as target states, reference trajectories, obstacle positions, friction coefficients, temperature, etc.

A specific property of optimal control problems is that the relationship between optimization variables, states $\bm{s}_{t}$ and controls $\bm{u}_{t}$, are defined by the dynamics constraint $\bm{f}_{d}$; and the initial state $\bm{s}_{0}$ is given. Therefore, a sequence of controls uniquely defines an (initial) solution. We can leverage this property by learning to predict only a sequence of controls instead of the full optimization variable of state-control sequences. Further, one can define the training loss over either control, state, or state-control sequences and backpropagate gradients through the dynamics constraint as long as it is differentiable. In our experiments, we use state-control loss by default as we found it to improve both our and baseline learning methods. In Appendix A.7, we show that our conclusions hold with control-only loss as well.

Figure: Figure 3: Optimal control tasks used in our experiments.
Refer to caption: x4.png

## 5 Experimental setup

#### Tasks.

We evaluated our method on the three robot control benchmark tasks shown in Fig. 3, each employing a distinct local optimization algorithm.
Cart-pole. This task involves balancing a pole upright while moving a cart toward a randomly selected target position (Barto et al., ), using a first-order box Differential Dynamic Programming (DDP) optimizer (Amos et al., ).
Reacher. In this task, a two-link planar robotic arm needs to reach a target placed at a random positon (Tassa et al., ), using a Model Predictive Path Integral (MPPI) optimizer (Williams et al., ).
Autonomous Driving. Based on the nuPlan benchmark (Caesar et al., ), this task focuses on trajectory tracking in complex urban environments by following a reference trajectory generated by a Predictive Driver Model (PDM) planner (Dauner et al., ), using the Iterative Linear Quadratic Regulator (iLQR) optimizer (Li & Todorov, ).
Further details are in Appendix A.1 and Appendix A.2.

#### Baselines.

We compare MISO to a range of alternative methods to provide single or multiple initial solutions.
For a single initial solution, we considered: Warm-start, the default method that uses the optimizer output from the last problem instance; Regression, a single-output regression model (the $K$ = 1 version of MISO); Oracle Proxy, optimization with unlimited runtime, which we also used to generate our training data.
For methods that generate multiple initial solutions, we considered: Warm-start with perturbations, which extends the warm-start approach by adding Gaussian noise to the optimizer output from the last problem instance; Regression with perturbations, where Gaussian noise is introduced to the predictions of the single-output regression model; Multi-output regression, a naive multi-output regression model without a diversity-promoting objective; and Ensemble, which trains multiple single-output neural networks with different random initializations. Finally, we assessed variants of our proposed method with the different training losses discussed in Sect. 4.3: pairwise distance, winner-takes-all, and mix.

**Table 1: Results for the single optimizer setting. The mean cost of solutions found by the single optimizer using different initial solutions across tasks and evaluation settings.**
|  |  | One-Off Optimization | Sequential Optimization |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Method | $K$ | Reacher | Cart-pole | Driving | Reacher | Cart-pole | Driving |
| Single Optimizer |  |  |  |  |  |  |  |
| Warm-start | 1 | 13.48  $\pm$0.88 | 11.69  $\pm$0.84 | 283.86  $\pm$37.91 | 13.48  $\pm$0.88 | 11.69  $\pm$0.84 | 283.86  $\pm$37.91 |
| Regression | 1 | 13.40  $\pm$0.88 | 11.19  $\pm$0.80 | 74.23  $\pm$7.69 | 19.56  $\pm$0.52 | 6.18  $\pm$0.47 | 70.62  $\pm$7.38 |
| Warm-start w. perturb | 32 | 13.46  $\pm$0.88 | 11.64  $\pm$0.83 | 145.01  $\pm$23.01 | 14.71  $\pm$0.93 | 16.29  $\pm$0.44 | 164.75  $\pm$22.84 |
| Regression w. perturb | 32 | 13.38  $\pm$0.88 | 11.16  $\pm$0.80 | 67.69  $\pm$8.01 | 15.28  $\pm$0.58 | 5.74  $\pm$0.47 | 66.75  $\pm$6.56 |
| Multi-output regression | 32 | 13.41  $\pm$0.88 | 11.21  $\pm$0.80 | 70.25  $\pm$8.75 | 18.49  $\pm$0.55 | 6.62  $\pm$0.45 | 78.74  $\pm$8.99 |
| Ensemble | 32 | 13.39  $\pm$0.88 | 10.94  $\pm$0.79 | 47.22  $\pm$4.71 | 8.40  $\pm$0.40 | 3.55  $\pm$0.34 | 52.59  $\pm$4.81 |
| MISO pairwise dist. | 32 | 13.41  $\pm$0.88 | 11.22  $\pm$0.80 | 66.06  $\pm$7.48 | 19.20  $\pm$0.49 | 6.07  $\pm$0.45 | 71.90  $\pm$7.90 |
| MISO winner-takes-all | 32 | 13.36  $\pm$0.88 | 10.48  $\pm$0.77 | 30.17  $\pm$2.24 | 2.72  $\pm$0.21 | 0.83  $\pm$0.06 | 30.75  $\pm$2.15 |
| MISO mix | 32 | 12.74  $\pm$0.86 | 10.48  $\pm$0.77 | 33.95  $\pm$2.39 | 2.44  $\pm$0.20 | 0.79  $\pm$0.04 | 33.38  $\pm$2.21 |
| Oracle Proxy | 1 | 13.43  $\pm$0.88 | 11.01  $\pm$0.80 | 41.94  $\pm$4.31 | 6.88  $\pm$0.58 | 4.54  $\pm$0.71 | 26.52  $\pm$2.00 |

**Table 2: Results for the multiple optimizers setting. Mean cost of solutions found by multiple optimizers using different initial solutions across tasks and evaluation settings.**
|  |  | One-Off Optimization | Sequential Optimization |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Method | $K$ | Reacher | Cart-pole | Driving | Reacher | Cart-pole | Driving |
| Multiple Optimizers |  |  |  |  |  |  |  |
| Warm-start w. perturb | 32 | 13.41  $\pm$0.88 | 10.93  $\pm$0.79 | 155.53  $\pm$24.33 | 5.89  $\pm$0.50 | 6.68  $\pm$0.59 | 162.13  $\pm$34.10 |
| Regression w. perturb | 32 | 13.34  $\pm$0.88 | 11.12  $\pm$0.80 | 64.88  $\pm$6.84 | 3.53  $\pm$0.27 | 5.36  $\pm$0.48 | 62.07  $\pm$6.39 |
| Multi-output regression | 32 | 13.34  $\pm$0.88 | 11.21  $\pm$0.80 | 70.29  $\pm$9.13 | 3.31  $\pm$0.26 | 6.38  $\pm$0.43 | 70.71  $\pm$8.18 |
| Ensemble | 32 | 13.34  $\pm$0.88 | 10.65  $\pm$0.78 | 45.44  $\pm$4.64 | 3.08  $\pm$0.23 | 2.21  $\pm$0.20 | 49.08  $\pm$5.29 |
| MISO pairwise dist. | 32 | 13.34  $\pm$0.88 | 11.22  $\pm$0.80 | 67.62  $\pm$7.58 | 3.42  $\pm$0.27 | 6.09  $\pm$0.47 | 71.33  $\pm$8.13 |
| MISO winner-takes-all | 32 | 13.34  $\pm$0.88 | 10.29  $\pm$0.76 | 30.87  $\pm$2.30 | 2.21  $\pm$0.16 | 0.76  $\pm$0.05 | 30.48  $\pm$2.07 |
| MISO mix | 32 | 12.72  $\pm$0.86 | 10.29  $\pm$0.76 | 33.52  $\pm$2.35 | 1.56  $\pm$0.14 | 0.63  $\pm$0.02 | 34.85  $\pm$2.64 |

#### Evaluation settings.

We employ two evaluation modes.
(i) One-off, where the optimization task is treated as an isolated problem with the objective of finding the minimum of a given function. This mode serves as the default configuration for training neural networks, where data is replayed to the model, and the optimizer’s solution is recorded but not executed. Methods are assessed by the mean cost of the optimizer’s output over problem instances. (ii) Sequential, which involves solving a series of related optimization problems, executing each proposed solution, and starting the subsequent optimization from the resulting state. This setting simulates real-world conditions where the optimizer continuously interacts with a dynamic environment in a closed loop. We evaluate performance by taking the mean cost over problems in a sequence, and then the mean over sequences.

To account for the additional time required to predict initial solutions, we assumed that all models perform inference in under 0.85ms, which was the case for all methods on both CPU and GPU, except for the ensemble (see Appendix A.6). In the autonomous driving task, we then reduced the runtime allocated to the optimizer accordingly.

#### Implementation details.

To generate the training data, we first create a set of problem instances by sequentially executing the optimizer initialized with the default warm-start strategy. The problem instances are then fed again to an “oracle” version of the optimizer with a significantly increased runtime limit, and the resulting solutions are recorded. After training, evaluation is done on a separate unseen set of problem instances.
All experiments are conducted on an Intel Core i9-13900KF CPU and an NVIDIA RTX 4090 GPU. Further implementation details, including hyperparameters and training procedures, are in Appendix A.4 and Appendix A.3.

## 6 Results

Figure: Figure 4: Mean Cost of the autonomous driving environment with varying values of $K$. The shaded regions around each curve indicate the standard error of the mean.
Refer to caption: x7.png

Our main results for optimization with different initial solutions are reported in Table 1 and Table 2 for single optimizer and multiple optimizers settings, respectively. Figure 4 shows the effect of the number of predicted initial solutions. Figure 5 provides qualitative results. More detailed results, including inference times, are in the Appendix.

#### Single optimizer

. In the single-optimizer setting, Table 1, we first observe that even one learned initialization outperforms heuristic solutions (regression vs. warm-start), in almost all settings, and in particular in the most challenging autonomous driving task.
We then examine the impact of generating multiple initial solutions. Perturbations-based methods show some improvement over their single-initialization counterparts in most cases, and ensembles of independently learned models perform consistently better than single models. Finally, our proposed multi-output methods demonstrate substantial improvements over all baselines because they can learn to predict diverse multimodal initial solutions. Specifically, MISO winner-takes-all or MISO mix achieve the lowest mean costs across all tasks.
Considering the pairwise distance term alone proves insufficient to ensure adequate diversity, whereas incorporating it with MISO winner-takes-all often boosts performance, yet, its effectiveness varies, which underscores the challenge of selecting optimal hyperparameters.
As expected, improvements are consistently larger in the more important sequential optimization setting, where errors over time compound.

#### Multiple optimizers

.
When considering the multiple-optimizers setting, we observe the same trend. Learning-based methods outperform heuristic ones, and multi-output approaches yield further enhancements. As expected, the use of multiple optimizers leads to consistently better results compared to the single-optimizer setting due to increased exploration of the solution space.

#### Scaling with the number of initial solutions.

Figure 4 shows that our method scales effectively and consistently with the number of predicted initial solutions $K$, and outperforms other approaches across varying values of $K$. Importantly, as $K$ increases, the inference time for ensemble approaches grows, whereas MISO remains almost constant (see Appendix A.6). We further evaluate mode diversity in Appendix A.8, and find that, in line with our conclusions, all MISO outputs remain useful even when $K$ increases.

#### Qualitative results.

Figure 5 (left) depicts the optimizer’s output trajectories with different initial solutions for the autonomous driving task. In this scenario, the high-level planner abruptly alters the reference path, which could happen, e.g., because of a newly detected pedestrian. The change in reference path makes the previous solution (warm-start) a poor initialization, and the optimizer converges to a local minimum that minimizes control effort but is far from the desired path. Regression and model ensemble also fail to predict a good initial solution. In contrast, MISO winner-takes-all adapts to this sudden reference change and closely follows the reference path.
Figure 5 (right) depicts MISO’s initial solutions for the cart-pole task. The different outputs capture different modes of the solution space (moving upright, moving with left swing, moving with right swing), showing MISO’s ability to generate diverse and multimodal solutions.

#### Summary.

Overall, our methods significantly outperform the other baselines in both settings. The consistent superiority of the MISO mix and MISO winner-takes-all methods across different tasks and configurations underscores the advantages of using learning-based multi-output strategies for generating initial solutions. These findings demonstrate that promoting diversity among multiple initializations is crucial for improving optimization outcomes, especially when combined with multiple optimizers.

Figure: Figure 5: Left: Each method adaptation when the high-level planner abruptly modifies the reference path. Right: Multiple predicted trajectories of MISO winner-takes-all.
Refer to caption: x9.png

## 7 Conclusions and Future Work

We introduced Learning Multiple Initial Solutions (MISO), a novel framework for learning multiple diverse initial solutions that significantly enhance the reliability and efficiency of local optimization algorithms across various settings. Extensive experiments in optimal control demonstrated that our method consistently outperforms baseline approaches and scales efficiently with the number of initializations.

#### Limitations.

Our approach is not without limitations. First, to train a useful model, we rely on the coverage and quality of the training data, as the method does not directly interact with the optimizer or the underlying objective function. Second, the underlying assumption of our regression loss is that initial solutions closer to the global optimum increase the likelihood of successful optimization may not hold in complex optimization landscapes with intricate constraints.
Third, in highly complex optimization problems where each solution constitutes a high-dimensional and intricate structure, accurately learning initial solution candidates can become exceedingly challenging, potentially diminishing the effectiveness of our approach.

#### Future work.

There are several promising directions for future research.
To address the aforementioned limitations, one may simply incorporate the optimization objective into the model training loss, thus creating a direct link to the final optimization goal. Alternatively, using reinforcement learning (RL) to train MISO is a particularly exciting opportunity. By framing the problem in an RL context, e.g., where the reward is the negative cost of the optimizer’s final solution, models would be directly trained to maximize the probability of the optimizer finding the global optima and may learn to specialize to the specific optimizer. One challenge would be computational, as RL would require running the optimizer numerous times during training.

Other extensions of our approach include probabilistic modeling, e.g., Gaussian mixture models, variational autoencoders, or diffusion models; however, preventing mode collapse and promoting diversity would remain a challenge. Future work may explore alternative selection functions, such as risk measures or criteria based on stability, robustness, exploration, or other domain-specific metrics; as well as using a heterogeneous set of parallel optimizers.
Finally, we are excited about various possible applications in optimal control and beyond, where sequences of similar optimization problems need to be solved, for example, localization and mapping in robotics, financial optimization, traffic routing optimization, or even training neural networks with different initial weights, e.g., for meta-learning, or scene representation learning with Neural Radiance Fields or 3D Gaussian splatting.

#### Ethics statement.

Our work is concerned with a general class of optimization problems that do not raise particular ethical considerations.

## Appendix A Appendix

### A.1 Detailed Descriptions of Baseline Optimizers

This subsection provides detailed descriptions of the optimization algorithms used in our evaluations: First-order Box Differential Dynamic Programming (DDP), Model Predictive Path Integral (MPPI), and the Iterative Linear Quadratic Regulator (iLQR). These algorithms were selected due to their widespread use and effectiveness in solving optimal control problems across various domains.

#### First-order Box Differential Dynamic Programming (DDP).

Building on the work of Tassa et al. (), Amos et al. () introduced a simplified version of Box-DDP that utilizes first-order linearization instead of second-order derivatives. This approach, termed ”first-order Box-DDP,” reduces computational complexity while maintaining the ability to handle box constraints on both the state and control spaces.

#### Model Predictive Path Integral (MPPI).

MPPI (Williams et al., ) is a sampling-based model predictive control algorithm that iteratively refines control inputs using stochastic sampling. Starting from the current state and a prior solution, it generates a set of randomly perturbed control sequences, simulates their trajectories, and evaluates them using a cost function. The control inputs are then updated based on a weighted average, favoring lower-cost trajectories. We use the implementation from [https://github.com/UM-ARM-Lab/pytorch_mppi](https://github.com/UM-ARM-Lab/pytorch_mppi).

#### Iterative Linear Quadratic Regulator (iLQR).

iLQR (Li & Todorov, ) is a trajectory optimization algorithm that refines control sequences iteratively by linearizing system dynamics and approximating the cost function quadratically around a nominal trajectory. It alternates between a forward pass, simulating the system trajectory, and a backward pass, computing optimal control updates. We use the implementation provided in the nuPlan simulator.

### A.2 Detailed Task Descriptions and Hyperparameters

This subsection provides detailed descriptions of the tasks used in our experiments: cart-pole, reacher, and autonomous driving. For each task, we outline the system dynamics, control inputs, and the specific hyperparameters employed in our evaluations.

#### Cart-pole.

The cart-pole task (Barto et al., ) involves a cart-pole system tasked with swinging the pole upright while moving the cart to a randomly selected target position along the rail. The goal is to balance the pole vertically and simultaneously reach the target cart position. The system is characterized by the state vector $\bm{s}_{t}\in\mathbb{R}^{4}$, which includes the pole angle $\theta$, pole angular velocity $\dot{\theta}$, cart position $x$, and cart velocity $\dot{x}$. The control input is a single force applied to the cart, $\bm{u}_{t}\in\mathbb{R}$.

Hyperparameters. The mass of the cart is $m_{c}=1.0\,\text{kg}$, the mass of the pole is $m_{p}=0.3\,\text{kg}$, and the length of the pole is $l=0.5\,\text{m}$. Gravity is set to $g=-9.81\,\text{m/s}^{2}$. Control inputs are bounded by $u_{\min}=-5.5\,\text{N}$ and $u_{\max}=5.5\,\text{N}$, with a time step of $\Delta t=0.1\,\text{s}$ and $n_{\text{sub_steps}}=2$ physics sub-steps per control step. Each episode has a maximum length of $T_{\text{env}}=50$ steps.
Both optimizers use goal weights of $[0.1,0.01,1.0,0.01]$ for the state variables and a control weight of $0.0001$. The prediction horizon is set to $H=10$. For the online optimizer, we set $\texttt{lqr_iter}=2$ and $\texttt{max_linesearch_iter}=1$. In the oracle optimizer, we use $\texttt{lqr_iter}=10$ and $\texttt{max_linesearch_iter}=3$.
The initial state $\bm{s}_{0}\in\mathbb{R}^{4}$ is sampled as follows: $x_{0}\sim\mathcal{U}(-2,2)\,\text{m}$, $\dot{x}_{0}\sim\mathcal{U}(-1,1)\,\text{m/s}$, $\theta_{0}\sim\mathcal{U}(-\frac{\pi}{2},\frac{\pi}{2})\,\text{rad}$, and $\dot{\theta}_{0}\sim\mathcal{U}(-\frac{\pi}{4},\frac{\pi}{4})\,\text{rad/s}$.
The goal state is defined by a target cart position $x_{\text{goal}}\sim\mathcal{U}(-2,2)\,\text{m}$, while the rest of the state variables (pole angle and velocities) are set to zero, ensuring the goal is to bring the pole upright and bring the system to rest.

#### Reacher.

The Reacher task (Tassa et al., ) involves a two-link planar robot arm tasked with reaching a randomly positioned target. The goal is to move the end-effector to the target position in the plane. The system is characterized by the state vector $\bm{s}_{t}\in\mathbb{R}^{4}$, which includes the joint angles $\theta_{1},\theta_{2}$ and angular velocities $\dot{\theta}_{1},\dot{\theta}_{2}$. The control inputs, $\bm{u}_{t}\in\mathbb{R}^{2}$, are torques applied to each joint.

Hyperparameters. The simulation uses a time step of $\Delta t=0.02\,\text{s}$, with joint damping set to $0.01$ and motor gear ratios of $0.05$. The control inputs are constrained by $u_{\min}=[-1,-1]$ and $u_{\max}=[1,1]$. The wrist joint has a limited range of $[-160^{\circ},160^{\circ}]$. Each episode is limited to $T_{\text{env}}=250$ steps.
Both optimizers are set with a control noise covariance $\sigma^{2}=1\times 10^{-3}$, a temperature parameter $\lambda=1\times 10^{-4}$, and a prediction horizon $H=10$. The online optimizer uses $\texttt{num_samples}=3$, while the oracle optimizer uses $\texttt{num_samples}=50$.
The target position is generated by sampling $\theta_{\text{target}}\sim\mathcal{U}(0,2\pi)$ and $r_{\text{target}}\sim\mathcal{U}(0.05,0.20)\,\text{m}$, then set as $\text{pos}_{x}=r_{\text{target}}\cos(\theta_{\text{target}})$ and $\text{pos}_{y}=r_{\text{target}}\sin(\theta_{\text{target}})$.

#### Autonomous Driving.

The autonomous driving task, based on the nuPlan benchmark (Caesar et al., ), evaluates the performance of motion planning algorithms in complex urban environments. Our focus is on the control (tracking) layer, which is responsible for accurately following the planned trajectories. The task involves navigating a vehicle through a series of scenarios with varying traffic conditions, obstacles, and road layouts. The goal is to execute safe, efficient, and comfortable trajectories while adhering to traffic rules and avoiding collisions. The system is characterized by the state vector $\bm{s}_{t}\in\mathbb{R}^{5}$, which includes the vehicle’s position $(x,y)$, orientation $\phi$, velocity $v$, and steering angle $\delta$. The control inputs, $\bm{u}_{t}\in\mathbb{R}^{2}$, are acceleration $a$ and steering angle rate $\dot{\delta}$.
We use the state-of-the-art Predictive Driver Model (PDM) planner (Dauner et al., ) to generate the reference trajectories $\bm{r}_{t}$.

Hyperparameters. The prediction horizon is set to $H=40$ with a discretization time step of $\Delta t=0.2\,\text{s}$. The cost function is weighted with state cost diagonal entries $[1.0,1.0,10.0,0.0,0.0]$ for the position, heading, velocity, and steering angle, respectively, and input cost diagonal entries $[1.0,10.0]$ for acceleration and steering angle rate. The maximum acceleration is constrained to $3.0\,\text{m/s}^{2}$, the maximum steering angle is $60^{\circ}$, and the maximum steering angle rate is $0.5\,\text{rad/s}$. A minimum velocity threshold for linearization is set at $0.01\,\text{m/s}$.
The online optimizer is limited to a maximum solve time of $\texttt{max_solve_time}=5\,\text{ms}$, while the oracle optimizer allows for $\texttt{max_solve_time}=50\,\text{ms}$.
For a fair comparison, we keep the total runtime limit fixed, including both initialization and optimization. The total runtime limit for warm-start and perturbation methods is $5\,\text{ms}$. For learning-based methods, the inference time for our neural networks is between $0.6\,\text{ms}$ and $0.7\,\text{ms}$ on a GPU, and $0.7\,\text{ms}$ to $0.8\,\text{ms}$ on a CPU. For simplicity, we allocate $0.85\,\text{ms}$ for model inference and run the optimizer for the remaining $4.15\,\text{ms}$. We have not performed any inference optimization for our models (e.g., TensorRT).

### A.3 Network Architecture and Training Details

This section provides a brief overview of the network architecture, the data collection process, and the training procedures used in our experiments. We summarize the design of our base Transformer model, outline the methods used to generate and preprocess the training data, and detail the key training methodologies and hyperparameters employed.

#### Network Architecture.

The base model is a standard Transformer architecture with absolute positional embedding, a linear decoder layer, and output scaling. The core Transformer architecture remains standard, with task-specific configurations.
The input to the network is the concatenated sequence of the warm-start state trajectory error, $\bm{\tau}_{e}^{\mathrm{w.s.}}$, defined as the difference between the reference trajectory, $\bm{\psi}=\bm{\tau}_{r}$, (in the autonomous driving task) or the goal state, $\bm{\psi}=\bm{x}_{g}$ (in the cart-pole and reacher tasks), and the warm-start trajectory, $\bm{\tau}_{x}^{\mathrm{w.s.}}$. The warm-start control trajectory is $\bm{\tau}_{u}^{\mathrm{w.s.}}=\{\{\bm{u}_{t+k}^{\mathrm{cand}}\}_{k=1}^{H-2},
\bm{0}\}$. The network predicts $K$ control trajectories, $\{\bm{\hat{\tau}}_{u,k}^{\mathrm{init}}\}_{k=1}^{K}$, for the next optimization step.
Each environment’s configuration is described in Table 3.

**Table 3: Configuration of the Transformer model for each environment.**
| Parameter | Description | Reacher | Cart-pole | Driving |
| --- | --- | --- | --- | --- |
| n_layer | Number of layers | 4 | 4 | 4 |
| n_head | Number of heads | 2 | 2 | 2 |
| n_embd | Embedding dimension | 64 | 64 | 64 |
| dropout | Dropout rate | 0.1 | 0.1 | 0.1 |
| src_dim | Input dimension | 8 | 5 | 7 |
| src_len | Sequence length | 10 | 9 | 40 |
| out_dim | Output dimension | 2 | 1 | 2 |

#### Data Collection and Preprocessing.

In all experiments, the training data is generated by (1) unrolling an optimizer using a warm-start initialization policy and recording its inputs and outputs, and (2) replaying the same scenarios using an oracle optimizer—essentially the same optimizer with enhanced capabilities, such as more optimization steps or additional sampled trajectories—and logging its inputs and outputs. The resulting mapping may be seen as a filter, refining (near-)optimal initial solutions into optimal ones.

#### Training.

Prior to training, all features were standardized to ensure consistent input scaling. We used the AdamW optimizer and applied gradient norm clipping. The models were trained using standard settings without any complex modifications.
All hyperparameters are detailed in Table 4.

**Table 4: Training hyperparameters for each environment.**
| Parameter | Description | Reacher | Cart-pole | Driving |
| --- | --- | --- | --- | --- |
| epochs | Epochs | 125 | 125 | 125 |
| batch_size | Batch size | 1024 | 1024 | 1024 |
| lr | Learning rate | 0.001 | 0.0003 | 0.0001 |
| weight_decay | Weight decay | 0.0001 | 0.0001 | 0.0001 |
| grad_norm_clip | Gradient norm clipping | 2.0 | 2.0 | 2.0 |
| control_loss_weight | Control loss weight | 100.0 | 1.0 | 5.0 |
| state_loss_weight | State loss weight | 0 | 0.01 | 0.005 |
| pairwise_loss_weight | Pairwise loss weight | 0.1 | 0.01 | 0.1 |

### A.4 Descriptions of Baseline Methods

This section introduces the baseline methods used in our experiments in more detail and discusses their implementation specifics. These baselines serve as reference points to evaluate our proposed methods’ performance and understand the benefits and limitations of different initialization strategies in optimization algorithms.

#### Warm-start ( K 𝐾 K italic_K = 1).

A common technique involves shifting the previous solution forward by one time step and padding it with zeros. Assuming the system does not exhibit rapid changes in this interval, the previous solution should retain local, feasible information.

#### Oracle Proxy ( K 𝐾 K italic_K = 1).

This proxy serves two purposes: (1) estimating the gap between a real-time-constrained optimizer and an unrestricted one and (2) providing a proxy for a mapping worth learning. For each optimization algorithm, a suitable heuristic is defined. In DDP, the oracle is allowed more iterations to converge; in MPPI, it has a larger sample budget; and in iLQR, it is given more time to perform optimization iterations.

#### Regression ( K 𝐾 K italic_K = 1).

This approach involves training a neural network to approximate the oracle’s mapping. Unlike the oracle heuristic, which is impractical for real-time use, the trained neural network requires only a single forward pass.

#### Warm-start with Perturbations ( K 𝐾 K italic_K >1).

We utilize the warm-start technique as another baseline by duplicating the proposed initial solution $K$ times and adding Gaussian noise. While this introduces some form of dispersion, the resulting initial solutions are neither guaranteed to be feasible nor to ensure any level of optimality.

#### Regression with Perturbations ( K 𝐾 K italic_K >1).

Similar to the warm-start with perturbation, after predicting an initial solution using the neural network, we duplicate it $K$ times and add perturbations.

#### Ensemble ( K 𝐾 K italic_K >1).

An ensemble of $K$ separate neural networks leverages the idea that networks initialized with different weights during training will often produce different predictions. The main drawbacks of this approach are (1) the need to train $K$ neural networks and (2) the requirement to run $K$ forward passes, which may be impractical for real-time deployment.

#### Multi-Output Regression ( K 𝐾 K italic_K >1).

A naive approach to predicting $K$ initializations from a single network involves calculating the loss for each prediction and summing the losses. However, since there is no explicit multimodality objective, these models are prone to mode collapse.

#### MISO Pairwise Distance ( K 𝐾 K italic_K >1).

One way to mitigate mode collapse is to introduce an additional term in the loss function, such as the pairwise distance between predictions. While this approach requires weight tuning and selecting an appropriate norm, a significant challenge lies in understanding how effectively this term promotes multimodality in practice.

#### MISO Winner-Takes-All ( K 𝐾 K italic_K >1).

This approach updates only the best-performing mode based on the loss of each prediction. Although no explicit dispersion objective is included, multimodality is indirectly encouraged by maintaining multiple active modes while refining only the best one and not penalizing the others.

#### MISO Mix ( K 𝐾 K italic_K >1).

Lastly, we combine the pairwise distance term with the Winner-Takes-All approach. While this allows for greater refinement, it adds complexity due to additional hyperparameters and the need to apply further operations—such as clamping distances—to ensure the loss won’t diverge.

### A.5 Illustrative Example

This example provides a simplified scenario to illustrate the behavior of local optimization algorithms in a controlled, low-dimensional setting with non-convex and multimodal objective function.
The system dynamics are defined by the linear equation $x_{t+1}=x_{t}+u_{t}$, where $x_{t}\in\mathbb{R}$ represents the state and $u_{t}\in[-1,1]$ is the constrained control input at time step $t$. The initial state is $x_{0}=0$, and the optimization horizon is set to $H=5$.
The objective is to find the optimal control trajectory $\bm{\tau}_{u}^{\star}=\{u_{t}\}_{t=0}^{H-1}$ that minimizes the cumulative cost function $\bm{\tau}_{u}^{\star}=\arg\min_{\bm{\tau}_{u}}\sum_{k=0}^{H-1}c(x_{t+k})$, where the resulting state trajectory $\bm{\tau}_{x}=\{x_{t}\}_{t=0}^{H}$ is obtained by unrolling $\bm{\tau}_{u}$ from $x_{0}$. The cost function, $c(x)=(x^{2}+0.05)(x+1.5)^{2}(x-2)^{2}$, is non-convex and multimodal, featuring two global minima at $x^{\star}_{1}=-1.5$ and $x^{\star}_{2}=2$.

**Table 5: Comparison of different methods for predicting the optimal control trajectories**
| Method | $x_{H+1}$ | $\bm{\hat{\tau}}_{u}$ |
| --- | --- | --- |
| Ensemble | 0.03 | -0.02, -0.03, -0.00, -0.01, -0.01 |
|  | 0.02 | -0.01, -0.02, -0.01, -0.01, -0.01 |
| MISO pairwise dist. | -0.24 | -0.12, -0.05, -0.04, -0.02, -0.01 |
|  | 0.24 | 0.08, -0.05, -0.04, -0.03, -0.03 |
| MISO winner-takes-all | -1.49 | -0.88, -0.65, -0.00, -0.02, -0.02 |
|  | 2.01 | 0.99, -0.94, -0.05, -0.01, -0.01 |
| MISO mix | -1.52 | -0.90, -0.68, -0.00, -0.03, -0.03 |
|  | -2.05 | 0.99, -0.95, -0.07, -0.02, -0.02 |
| Optimal | -1.50 | -1.00, -0.50, -0.00, -0.00, -0.00 |
|  | 2.00 | 1.00, -1.00, -0.00, -0.00, -0.00 |

The results from Table LABEL:tab:toy-task provide a comparison of different methods for predicting optimal control trajectories. The Ensemble method, which combines multiple single-output predictions, yields the least accurate results, with final states close to zero. Due to the dispersion term, MISO pairwise distance is a bit further from zero but still far from either optimum. On the other hand, MISO winner-takes-all and MISO mix successfully predict both optimal sequences with high fidelity and thus are able to reach either global optimum.

Overall, the results suggest that methods specifically tailored for capturing multimodality, such as the winner-takes-all and mixed strategies, are more effective than their single-output regression counterparts, particularly in non-convex environments.

### A.6 Inference Time: Ensemble vs. Multi-Output Models

Figure: Figure 6: Mean inference time for varying values of $K$ for the ensemble and multi-output models on CPU and CUDA
Refer to caption: x10.png

This experiment benchmarks the execution time of two model architectures: an ensemble of $K$ single-output models and a single multi-output model producing $K$ outputs. Both models are based on the Transformer architecture used in the autonomous driving environment.

The experiments were conducted on an Intel Core i9-13900KF CPU and an NVIDIA RTX 4090 GPU, measuring the mean inference time over 1000 runs across five random seeds. GPU operations were synchronized before timing to ensure accurate measurements. The results, shown in Fig. 6, display the mean inference time in milliseconds as $K$ increases for both the ensemble and multi-output models on CPU and GPU.

The multi-output model exhibits minimal sensitivity to the increase in $K$ on both the CPU and GPU, indicating that this architecture scales efficiently, maintaining a low overhead even as the number of outputs grows. In contrast, the ensemble model’s inference time increases significantly with $K$, suggesting that managing multiple models introduces overhead that scales poorly as $K$ grows.

In applications with strict runtime constraints, such as the autonomous driving environment, the ensemble approach becomes impractical as $K$ increases. Conversely, the multi-output model remains a viable option, even at larger values of $K$, making it the preferred choice for time-sensitive scenarios.

### A.7 State Loss

An additional challenge in learning control policies is addressing *compounding errors*—small inaccuracies in the predicted control trajectory $\bm{\tau}_{u}$ that, when unrolled, cause significant deviations in the state trajectory $\bm{\tau}_{x}$. Even if most elements $\bm{\tau}_{u}$ are accurate, errors in the initial steps can cause the state $\bm{\tau}_{x}$ to drift, leading to further divergence as the system evolves.

To mitigate compounding errors, we introduce a regression loss not only over the control trajectory $\bm{\tau}_{u}$ but also over the resulting state trajectory $\bm{\tau}_{x}$. In a supervised learning setting, this requires a model of the system dynamics, which can either be known or learned. As we show in Table 6, incorporating state trajectory loss helps mitigate these types of error accumulation and improve long-horizon trajectory accuracy. More specifically, we see that (1) as the prediction horizon increases, from $H=9$ (Cart-pole) to $H=40$ (Driving), so does the difference between using and not using state loss, (2) The gap between One-Off and Sequential also increases thus we do not generalize as well, (3) For the single-output regression model, the difference is even greater.

**Table 6: Mean Cost Comparison for One-Off and Sequential Optimization (With and Without State Loss)**
|  |  | One-Off Optimization | Sequential Optimization |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | $K$ | Cart-pole | Driving | Cart-pole | Driving |  |  |  |  |
|  |  | No SL | With SL | No SL | With SL | No SL | With SL | No SL | With SL |
| Single Opt. |  |  |  |  |  |  |  |  |  |
| Regression | 1 | 11.88  $\pm$0.79 | 11.19  $\pm$0.80 | 440.97  $\pm$74.92 | 74.23  $\pm$7.69 | 11.93  $\pm$0.29 | 6.18  $\pm$0.47 | 590.50  $\pm$89.06 | 70.62  $\pm$7.38 |
| MISO WTA | 32 | 10.62  $\pm$0.77 | 10.48  $\pm$0.77 | 128.32  $\pm$22.55 | 30.17  $\pm$2.24 | 1.86  $\pm$0.20 | 0.83  $\pm$0.06 | 151.4  $\pm$20.70 | 30.75  $\pm$2.15 |
| MISO mix | 32 | 10.63  $\pm$0.77 | 10.48  $\pm$0.77 | 153.37  $\pm$21.8 | 33.95  $\pm$2.39 | 2.28  $\pm$0.26 | 0.79  $\pm$0.04 | 212.70  $\pm$28.02 | 33.38  $\pm$2.21 |
| Multiple Opt. |  |  |  |  |  |  |  |  |  |
| MISO WTA | 32 | 10.25  $\pm$0.75 | 10.29  $\pm$0.76 | 144.51  $\pm$19.51 | 30.87  $\pm$2.30 | 0.97  $\pm$0.06 | 0.76  $\pm$0.05 | 158.71  $\pm$20.94 | 30.48  $\pm$2.07 |
| MISO mix | 32 | 10.24  $\pm$0.75 | 10.29  $\pm$0.76 | 196.09  $\pm$31.43 | 33.52  $\pm$2.35 | 1.14  $\pm$0.12 | 0.63  $\pm$0.02 | 214.56  $\pm$28.02 | 34.85  $\pm$2.64 |

Table 6 shows a comparison between using and not using state loss (SL) across One-Off and Sequential Optimization settings. While both strategies benefit from including the state loss, the improvement is more profound in the Sequential Optimization setting. This is because it involves executing each solution and starting the next optimization from the resulting state, causing the compounding errors to accumulate. The One-Off Optimization setting also benefits from state loss, but the impact is less apparent and thus was not marked in the table.

### A.8 Mode frequency

Figure 7 presents a heatmap illustrating the percentage of selections for each specific mode (output) in MISO winner-takes-all by the selection function, which in this context chooses the output with the lowest resulting cost. Although a few modes appear to dominate, all other modes remain active, i.e., with a frequency greater than zero, indicating that there are problem instances where these less frequent modes identify the optimal action, thereby contributing to the overall performance improvements. Additionally, Fig. 8 compares the distribution of selections between MISO winner-takes-all and MISO mix with the expected, approximately uniform distribution of the Ensemble method.

Figure: Figure 7: Heatmap of MISO winner-takes-all outputs argmin frequency
Refer to caption: x11.png

Figure: Figure 8: Argmin frequency of various methods for $K$ =32
Refer to caption: x13.png

### A.9 Mean Cost Sequential Optimization

Figure: Figure 9: Mean Cost of the cart-pole and reacher environments with varying values of $K$. Subfigures (a) and (b) show the results for cart-pole using Single and Multiple Optimizers, respectively, while (c) and (d) display the same for the reacher environment. The shaded regions around each curve represent the standard error of the mean.
Refer to caption: x15.png

Figure 9 shows the mean cost of each method with varying values of $K$, for both cart-pole and reacher environments. Both showcase that our method scales effectively and consistently with the number of predicted initial solutions and outperforms other approaches across varying values of $K$.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - As mentioned by the authors, finding optimal solutions for higher dimensional problems might be challenging. In that setting, the optimizers are more likely to provide only suboptimal solutions within a time budget.  How could the proposed loss be extended to leverage the potentially suboptimal generated by the optimizers in a more complex setting?

- Is there any specific reason why a specific optimizer wash was chosen for each task? The paper mentions a trivial generalization to a setting with a heterogeneous set of optimization methods. But no experiments are reported. Would a heterogeneous set of optimization methods increase the chances of finding multimodal solutions and thus help with scalability to more complex settings? 

- What is the total time required to get a  "best solution"? (Fig 1). The inference time of the network might be fast but if the predicted initial solutions are not close to a local optimum, the optimizers might require multiple iterations to converge. Reporting the average number of additional iterations of the optimizers or the time to converge after warm-starting can potentially further highlight the quality of the learned initial solutions.
soundness: 2
strengths: - The problem setting is relevant for the community. The proposed method outperforms the presented baselines on 3 different tasks and the trained neural network architecture achieves fast inference rates, which are suitable for closed-loop control.

- The paper is well-written overall and presents multiple ablations for the different loss functions that were introduced. The experiments also indicate that the trained model captures the multimodality of the solution space, to some degree, depending on the hyperparameter K that denotes the fixed number of initial solutions that are predicted.
summary: The paper introduces a method for learning a set of candidate initial solutions to warm-start optimal control problems. It proposes a series of objectives that encourage learning multimodal solutions, using a transformer architecture as a backbone to predict K control trajectories. The proposed method is evaluated on 3 different sequential optimization tasks and yields performance improvements over the presented baselines.
weaknesses: - Using ensembles of models is not a very strong baseline for multimodality. Diffusion Policies [1] or action chunking transformers [2] might be stronger baselines. Even if they do not have such a fast inference time as the proposed method, it would further strengthen the paper to position the method with respect to such baselines. 
- The method is only evaluated on 3 low-dimensional problems and it is unclear how its performance will scale or degrade in more complex settings. 

[1] Cheng Chi et al, Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.
[2] Tony Zhao et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: At first sight, the second approach presented by the authors seems to always be at least as good as the first one. The only limitation seems to be that it is more computationally expensive because multiple optimizers are used. 

Why would any function other than J be used to specify \Lambda? It feels to me like J is simply poorly chosen if \Lambda is specified using any other criterion.

The name for the winner-takes-all is somewhat confusing. Intuitively, the name indicates that the best solution is used to compute the loss, not the worst.

In lines 299/300, what do the authors mean by "training loss over either control, state, or state-control sequences"? Does this mean that they use a distance metric for L_reg that takes the state into account? I would be helpful to have an explicit mathematical formulation in the appendix.

Is the difference between one-off and sequential evaluation the same as open-loop and closed-loop control? How are the initial conditions selected for the one-off evaluation?

As I understood it, the environment parameters do not vary between evaluations in the reacher and cart-pole settings. It would interesting to see how the approach performs, e.g., with varying constraints, obstacles and environment parameters (weight, arm length, etc.)
soundness: 3
strengths: The paper is easy to read and well-motivated. The presented approach is relevant, interesting and seemingly novel.
summary: The authors present an approach for computing multiple initializations for a downstream optimization task. The authors employ a winner-takes-all and pairwise distance loss to ensure multimodality of the initializations. This way, the produced initializations achieve better coverage and are more representative of a landscape with multiple optima.
weaknesses: The proposed approach does not have any theoretical guarantees.

It is unclear if the experimental section considers varying constraints and environment parameters outside the autonomous driving example. The paper would also benefit from comparing to more well-known control examples, e.g., from Mujoco.

The choice of \Lambda beyond the optimization problem is somewhat unclear and lacks motivation.

The second optimization pipeline is merely an extension of the first, though it is presented as one of the major contributions.

---

confidence: 5
contribution: 2
initial_rating: 6
presentation: 3
questions: See weakness
soundness: 4
strengths: 1. The paper is very well written and easy to follow. 

2. The figures, especially Figure 1 and 2, help clearly illustrate the proposed approach vs. prior methods. 

3. The problem is well motivated and could be general for many tasks.
summary: The paper considers the problem of improving optimization solvers. The paper proposes to learn multiple initial solutions for the optimizers to improve its final performance. The authors argue the learning of initial solutions should consider both the distance to ground-truth optimal parameters and the dispersion among the multiple initial solutions to ensure coverage of the optimization landscape. Experiments show the proposed method improves the optimization performance with three different optimizers on three simulated tasks. Although the paper is sound and well-written, I find the method missing important details, its novelty questionable and the experiment domains relatively simple.
weaknesses: 1. Important details about the method is missing: Section 4 focuses on introducing the learning objective, i.e., the loss function, However, it is unclear to me what function approximator was used to output the multiple initial guesses for the optimization problems. Is it a neural network with randomness that will have a different output in each forward pass? Is it a neural network outputting all K guesses in one forward pass and the output is split into K guesses? Or other design? This is arguably the most important part of the method but is missing. 

1.1. The $\Lambda$ function is another important part of the method, but was very briefly described. According to Line 191 (“A reasonable choice”), the $\Lambda$ function is just chosen as the argmin of objective function? It is unclear what exactly is the choice of $\Lambda$ in the experiments. 

1.2. The distance function in Line 223 only makes sense if the norm of the different dimensions of the parameter x make sense. For example, if one dimension of x ranges from [0, 1] and another dimension ranges from [0, 10^5], the norm is not a good measure of difference. It is unclear whether this assumption is satisfied in the experimented problems. 

2. The claims in Line 259-262 may not be true. It depends on how the ensemble of models are learned. If the models themselves are multimodal such as latent models or energy-based models, they could represent multimodal behavior. 

3. Missing information in the experiments: 

3.1. What are the variations \psi for the experiment domains? This is completely missing and how do the authors make sure the training setup and evaluation setup do not overlap? Otherwise, the initial solution learning is just memorizing the best solutions for the tasks. 

3.2. How much data is used to train in the experiments? 

3.3. I believe the two evaluation modes in Line 359-367 are just open-loop and closed-loop. Should the authors use these well-accepted naming conventions instead? 

3.4. Because Cartpole and Reacher are relatively simple domains, it is surprising to see just naïve optimizers cannot solve these problems very well. Did the authors constrain the optimization steps to certain budget such that optimal solution was not found in time? 

4. One arXiv paper has done a similar approach on a more challenging racing task [1]. I understand it is an arXiv paper recently and the authors might have missed it, but it seems the paper has even accomplished what this paper proposes in future work (reinforcement learning tuning for the initial guess). Could the authors compare and state how the proposed method is still unique? 

[1] Li, Z., Chen, L., Paleja, R., Nageshrao, S., & Gombolay, M. (2024). Faster Model Predictive Control via Self-Supervised Initialization Learning. arXiv preprint arXiv:2408.03394.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: For line 223, why is the pairwise distance loss defined as a function of $x^*$? To encourage diversity, shouldn’t the pairwise loss be maximized instead?
Could you elaborate what does problem instance $\psi$ stand for?
soundness: 2
strengths: The paper introduces a framework that leverages a neural network to generate initial conditions for general optimization problems. This framework enhances optimization performance by selecting an initial condition that is generated by a neural network trained to encourage diversity while remaining close to the global optimum.
summary: The paper introduces the Learning Multiple Initial Solutions framework, a neural network-based method for generating diverse initial solutions to improve local optimizer performance in time-sensitive tasks, such as robotics and autonomous driving. MISO supports both single-optimizer and multiple-optimizers configurations, allowing flexibility in selecting or parallelizing initial solutions.
weaknesses: While the empirical results indicate that the proposed framework may enhance optimization performance, the overall concept could be considered somewhat straightforward. A more substantial theoretical analysis could add depth to the work, as there is no clear indication that the neural network-generated initial states will consistently yield improved results. Consequently, the framework's contribution may seem limited in its novelty and theoretical rigor.

Meta-review: This paper presented a novel framework for predicting multiple initial solutions for sequential black-box optimization problems.
The authors introduced two strategies to predict initial solutions called single-optimizer and multiple-optimizers (not necessarily running in parallel), for the chosen of the best one-step optimizer.
As a neural network driving approach, the proposed method does not have any theoretical analysis, which is ok. The authors evaluated three distinct sequential optimization tasks in simulation, without comparison to any decision-transformer line of algorithms.
The improvement of either theoretical or empirical performance are welcome.

Decision: Reject