SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Arman Zharmagambetov<sup>1</sup> Brandon Amos<sup>1</sup> Aaron Ferber<sup>2</sup> Taoan Huang<sup>2</sup> Bistra Dilkina<sup>2</sup> Yuandong Tian<sup>1</sup>
$^{1}$ FAIR at Meta  $^{2}$ University of Southern California  $\{\text { armanz, bda, yuandong } \} @$ meta.com  $\{\text { aferber, taoanhua, dilkina } \} @$ usc.edu
Recent works in learning-integrated optimization have shown promise in settings where the optimization problem is only partially observed or where general-purpose optimizers perform poorly without expert tuning. By learning an optimizer  $\mathbf{g}$  to tackle these challenging problems with  $f$  as the objective, the optimization process can be substantially accelerated by leveraging past experience. The optimizer can be trained with supervision from known optimal solutions or implicitly by optimizing the compound function  $f\circ \mathbf{g}$ . The implicit approach may not require optimal solutions as labels and is capable of handling problem uncertainty; however, it is slow to train and deploy due to frequent calls to optimizer  $\mathbf{g}$  during both training and testing. The training is further challenged by sparse gradients of  $\mathbf{g}$ , especially for combinatorial solvers. To address these challenges, we propose using a smooth and learnable Landscape Surrogate  $\mathcal{M}$  as a replacement for  $f\circ \mathbf{g}$ . This surrogate, learnable by neural networks, can be computed faster than the solver  $\mathbf{g}$ , provides dense and smooth gradients during training, can generalize to unseen optimization problems, and is efficiently learned via alternating optimization. We test our approach on both synthetic problems, including shortest path and multidimensional knapsack, and real-world problems such as portfolio optimization, achieving comparable or superior objective values compared to state-of-the-art baselines while reducing the number of calls to  $\mathbf{g}$ . Notably, our approach outperforms existing methods for computationally expensive high-dimensional problems.
Mathematical optimization problems in various settings have been widely studied, and numerous methods exist to solve them [25, 32]. Although the literature on this topic is immense, real-world applications consider settings that are nontrivial or extremely costly to solve. The issue often stems from uncertainty in the objective or in the problem definition. For example, combinatorial problems involving nonlinear objectives are generally hard to address, even if there are efficient methods that can handle special cases (e.g.,  $k$ -means). One possible approach could be learning so-called linear surrogate costs [16] that guide an efficient linear solver towards high quality solutions for the original hard nonlinear problem. This automatically finds a surrogate mixed integer linear program (MILP), for which relatively efficient solvers exist [19]. Another example is the smart predict+optimize framework (a.k.a. decision-focused learning) [13, 41] where some problem parameters are unknown at test time and must be inferred from the observed input using a model (e.g., neural nets).
Despite having completely different settings and purposes, what is common among learning surrogate costs, smart predict+optimize, and other integrations of learning and optimization, is the need to learn a certain target mapping to estimate the parameters of a latent optimization problem. This makes the optimization problem well-defined, easy to address, or both. In this work, we draw general
Figure 1: Overview of our proposed framework LANCER. We replace the non-convex and often non-differentiable function  $f \circ \mathbf{g}$  with landscape surrogate  $\mathcal{M}$  and use it to learn the target mapping  $\mathbf{c}_{\theta}$ . The current output of  $\mathbf{c}_{\theta}$  is then used to evaluate  $f$  and to refine  $\mathcal{M}$ . This procedure is repeated in alternating optimization fashion.
connections between different problem families and combine them into a unified framework. The core idea (section 3) is to formulate the learning problem via constructing a compound function  $f \circ \mathbf{g}$  that includes a parametric solver  $\mathbf{g}$  and the original objective  $f$ . To the best of our knowledge, this paper is the first to propose a generic optimization formulation (section 3) for these types of problems.
Minimizing this new compound function  $f \circ g$  via gradient descent is a nontrivial task as it requires differentiation through the argmin operator. Although various methods have been proposed to tackle this issue [3, 2], they have several limitations. First, they are not directly applicable to combinatorial optimization problems, which have 0 gradient almost everywhere, and thus require various computationally expensive approximations [34, 41, 15, 39]. Second, even if the decision variables are continuous, the solution space (i.e., argmin) may be discontinuous. Some papers [12, 17] discuss the fully continuous domain but typically involve computing the Jacobian matrix, which leads to scalability issues. Furthermore, in some cases, an explicit expression for the objective may not be given, and we may only have black-box access to the objective function, preventing straightforward end-to-end backpropagation.
These limitations motivate Landscape Surrogate losses (LANCER), a unified model for solving coupled learning and optimization problems. LANCER accurately approximates the behavior of the compound function  $f \circ \mathbf{g}$ , allowing us to use it to learn our target parametric mapping (see fig. 1). Intuitively, LANCER must be differentiable and smooth (e.g., neural nets) to enable exact and efficient gradient computation. Furthermore, we propose an efficient alternating optimization algorithm that jointly trains LANCER and the parameters of the target mapping. Our motivation is that training LANCER in this manner better distills task-specific knowledge, resulting in improved overall performance. Experimental evaluations (section 5) confirm this hypothesis and demonstrate the scalability of our proposed method.
The implementation of LANCER can be found at https://github.com/facebookresearch/LANCER.
Smart Predict+Optimize framework considers settings where we want to train a target mapping which predicts latent components to an optimization problem to improve downstream performance. A straightforward and naive approach in  $\mathrm{P + O}$  is to build the target machine learning model in a two-stage fashion: train the model on ground truth problem parameters using some standard measure of accuracy (e.g., mean squared error) and then use its prediction to solve the optimization problem. However, this approach is prone to produce highly suboptimal models [6, 41] since the learning problem does not capture the task-specific objectives. Instead, "smart" Predict+Optimize (SPO) [13] proposed to minimize the decision regret, i.e., the error induced by the optimization solution based on the estimated problem parameters coming from the machine learning model. A variety of methods exist for learning such models in the continuous, and often convex, optimization setting. These approaches usually involve backpropagating through the solver [11, 12]. In some isolated and simple scenarios, an optimal solution exists [22], or the learning problem can be formulated via efficient surrogate losses [13]. Extending the SPO framework for combinatorial problems is challenging, and current methods rely on identifying heuristic gradients often via continuous, primal, or dual relaxations [33, 41, 34, 31, 39, 15, 28]. Alternative approaches leverage specific structures of the optimization problem [9, 21, 40], such as being solvable via dynamic programming or graph
partitioning. Further work focuses on SPO for individual problems Shah et al. [37], which collects perturbed input instances and learns a separate locally convex loss for each instance. In contrast, we define the loss over the domain rather than per instance, allowing for generalization to unseen instances, and our bilevel optimization formulation enables more efficient training. Moreover, compared to prior works, our method generically applies to a wider variety of problem settings, losses, and problem formulations. See Table 1 for a direct comparison of the requirements and capabilities for different approaches.
Mixed integer nonlinear programming (MINLP) LANCER can be considered as a solver for constrained combinatorial problems with nonlinear and nonconvex objectives (MINLP), a challenging class of optimization problems [4]. Apart from SurCo [16], which we will discuss in detail in section 3, specialized solvers exist that deal with various MINLP formulations [7, 1, 19]. These specialized solvers generally require an analytical form for the objective and cannot handle black-box objectives. Additionally, these methods are not designed to handle general-purpose large-scale problems without making certain approximations or using specialized modeling techniques, such as linearizing the objective function or applying domain-specific heuristics. Lastly, some previous work directly predicts solutions to economic dispatch problems [8], uses reinforcement learning to build solutions to linear combinatorial problems [23], or uses reinforcement learning for ride hailing problems [43]. These approaches are designed for specific application domains or are tailored to linear optimization settings where the reward for a single decision variable is its objective coefficient, which is not trivially applicable in nonlinear settings.
**Optimization as a layer** The optimization-as-a-layer family of methods considers the composition of functions where one (or more) of the functions is defined as the solution to a mathematical optimization problem [3, 2, 17, 18]. Since both  $\mathrm{P} + \mathrm{O}$  and  $\mathrm{SurCo}$  can be formulated as a nesting of a target mapping with the argmin operator (i.e., optimization solver), one can leverage approaches from this literature. The core idea here is based on using the implicit function theorem to find necessary gradients and backpropagate through the solver. Similar concepts exist for combinatorial optimization [15, 39, 28, 31, 29], where gradient non-existence is tackled through improved primal or dual relaxations.
3 A Unified Training Procedure
In this work, we focus on solving the following optimization problems:
$$
\min  _ {\mathbf {x}} f (\mathbf {x}; \mathbf {z}) \quad \text {s . t .} \quad \mathbf {x} \in \Omega \tag {1}
$$
where  $f$  is the function to be optimized (linear or nonlinear),  $\mathbf{x} \in \Omega$  are the decision variables that must lie in the feasible region, typically specified by (non)linear (in)equalities and possibly integer constraints, and  $\mathbf{z} \in \mathcal{Z}$  is the problem description (or problem features). For example, if  $f$  is to find a shortest path in a graph, then  $\mathbf{x}$  is the path to be optimized, and  $\mathbf{z}$  represents the pairwise distances (or features used to estimate them) in the formulation.
Ideally, we would like to have an optimizer that can (1) deal with the complexity of the loss function landscape (e.g., highly nonlinear objective  $f$ , complicated and possibly combinatorial domain  $\Omega$ ), (2) leverage past experience in solving similar problems, and (3) can deal with a partial information setting, in which only an observable problem description  $\mathbf{y}$  can be seen but not the true problem description  $\mathbf{z}$  when the decision is made at test time.
To design such an optimizer, we consider the following setting: assume that for the training instances, we have access to the full problem descriptions  $\{\mathbf{z}_i\} \subseteq \mathcal{Z}$ , as well as the observable descriptions  $\{\mathbf{y}_i\} \subseteq \mathcal{Y}$ , while for the test instance, we only know its observable description  $\mathbf{y}_{\mathrm{test}}$ , but not its full description  $\mathbf{z}_{\mathrm{test}}$ . Note that such a setting naturally incorporates optimization under uncertainty, in which a decision need to be made without full information, while the full information can be obtained in hindsight (e.g., portfolio optimization). Given this setting, we propose the following general training procedure on a training set  $\mathcal{D}_{\mathrm{train}} := \{(\mathbf{y}_i, \mathbf{z}_i)\}_{i=1}^N$  to learn a good optimizer:
$$
\min  _ {\boldsymbol {\theta}} \mathcal {L} (Y, Z) := \sum_ {i = 1} ^ {N} f \left(\mathbf {g} _ {\boldsymbol {\theta}} \left(\mathbf {y} _ {i}\right); \mathbf {z} _ {i}\right) \tag {2}
$$
Here  $\mathbf{g}_{\theta}:\mathcal{V}\mapsto \Omega$  is a learnable solver that returns a high quality solution for objective  $f$  directly from the observable problem description  $\mathbf{y}_i$ .  $\pmb{\theta}$  are the learnable solver's parameters. Once  $\mathbf{g}_{\theta}$  is learned, we can solve new problem instances with observable description  $\mathbf{y}_{\mathrm{test}}$  by either calling  $\mathbf{x}_{\mathrm{test}} = \mathbf{g}_{\theta}(\mathbf{y}_{\mathrm{test}})$  to get a reasonable solution, or continue to optimize Eqn. 1 using  $\mathbf{x} = \mathbf{x}_{\mathrm{test}}$  as an initial solution.
Theoretically, if problem description  $\mathbf{z}$  is fully observable (i.e.,  $\mathbf{y} = \mathbf{z}$ ), the optimization oracle  $\arg \min_{\mathbf{x} \in \Omega} f(\mathbf{x}; \mathbf{z})$  solves Eqn. 2. However, solving may be computationally intractable even with full information as in nonlinear combinatorial optimization.
Our proposed training procedure is general and covers many previous works that rely on either fully or partially observed problem information.
Smart Predict+Optimize  $(\mathbf{P} + \mathbf{O})$  In this setting,  $f$  belongs to a specific function family (e.g., linear or quadratic programs). The full problem description  $\mathbf{z}$  includes objective coefficients, but we only have access to noisy versions of them in  $\mathbf{y}$ . Then the goal in  $\mathrm{P} + \mathrm{O}$  is to identify a mapping  $\mathbf{c}_{\theta}$  (e.g. a neural net) so that a downstream solver outputs a high quality solution:  $\mathbf{g}_{\theta}(\mathbf{y}) = \arg \min_{\mathbf{x} \in \Omega} f(\mathbf{x}; \mathbf{c}_{\theta}(\mathbf{y}))$ . Here  $\arg \min_{\mathbf{x} \in \Omega} f$  can often be solved with standard approaches, and the main challenge is to estimate the problem description accurately (w.r.t. eq. (2)). Note that other  $\mathrm{P} + \mathrm{O}$  formulations can be encompassed within our framework in eq. (2). For instance, the regret-based formulation described in SPO [13] can be represented as  $\max_{\hat{\mathbf{z}} \in \mathbf{g}_{\theta}(\mathbf{y})} f(\hat{\mathbf{z}}; \mathbf{z}) - f^{*}$  where  $f^{*}$  is the optimal loss that is independent of  $\theta$ .
Learning surrogate costs for MINLP When  $f$  is a general nonlinear objective (but  $\mathbf{y} = \mathbf{z}$  is fully observed), computing  $\arg \min_{\mathbf{x} \in \Omega} f$  also becomes non-trivial, especially if  $\mathbf{x}$  is in combinatorial spaces. Such problems are commonly referred as mixed integer nonlinear programming (MINLP). To leverage the power of linear combinatorial solvers, SurCo [16] sets the learnable solver to be  $\mathbf{g}_{\theta}(\mathbf{y}) = \arg \min_{\mathbf{x} \in \Omega} \mathbf{x}^{\top} \mathbf{c}_{\theta}(\mathbf{y})$ , which is a linear solver and does not include the nonlinear function  $f$  at all. Intuitively, this models the complexity of  $f$  by the learned surrogate cost  $c_{\theta}$ , which is parameterized by a neural network. Surprisingly, this works quite well in practice [16].
4 LANCER: Learning Landscape Surrogate Losses
Variations of training objective Eqn. 2 have been proposed to learn  $\theta$  in one way or another. This includes derivative-based approaches discussed in section 2 as well as domain-specific methods that learn  $\theta$  efficiently and avoid backpropagating through the solver, e.g., SPO+ [13].
While these are valid approaches, at each step of the training process, we need to call a solver to evaluate  $\mathbf{g}_{\theta}$ , which can be computationally expensive. Furthermore,  $\mathbf{g}_{\theta}$  is learned via gradient descent of Eqn. 2, which involves backpropagating through the solver. One issue of this procedure is that the gradient is non-zero only at certain locations (i.e., when changes in the coefficients lead to changes in the optimal solution), which makes the gradient-based optimization difficult.
One question arises: can we model the composite function  $f \circ \mathbf{g}_{\theta}$  jointly? The intuition here is that while  $\mathbf{g}_{\theta}$  can be hard to compute,  $f \circ \mathbf{g}_{\theta}$  can be smooth to model, since  $f$  can be smooth around the solution provided by  $\mathbf{g}_{\theta}$ . If we model  $f \circ \mathbf{g}_{\theta}$  locally by a landscape surrogate model  $\mathcal{M}$ , and optimize directly on the local landscape of  $\mathcal{M}$ , then the target mapping  $\mathbf{c}_{\theta}$  can be trained without running expensive solvers:
$$
\min  _ {\boldsymbol {\theta}} \mathcal {M} (Y, Z) := \sum_ {i = 1} ^ {N} \mathcal {M} \left(\mathbf {c} _ {\boldsymbol {\theta}} \left(\mathbf {y} _ {i}\right); \mathbf {z} _ {i}\right). \tag {3}
$$
Note that  $\mathcal{M}$  directly depends on  $\mathbf{c}_{\theta}$  (not on  $\mathbf{g}_{\theta}$ ). Obviously,  $\mathcal{M}$  cannot be any arbitrary function. Rather it should satisfy certain conditions: 1) capture a task-specific loss  $f \circ \mathbf{g}_{\theta}$  (not just  $f$  and not the solver  $\mathbf{g}$  alone, but jointly); 2) be differentiable and smooth. Differentiability allows us to train our target model  $\mathbf{c}_{\theta}$  in end-to-end fashion (assuming  $\mathbf{c}$  is itself differentiable). The primary advantage is that we can avoid backpropagating through the solver or even through  $f$  (e.g., multiarmed bandits). Moreover,  $\mathcal{M}_{\mathbf{w}}$  is typically high dimensional (e.g., a neural net) and potentially can make the learning problem for  $\mathbf{c}_{\theta}$  much easier. The question is how to obtain such a model  $\mathcal{M}$ ? One
way is to parameterize it and formulate the learning problem:
$$
\min  _ {\mathbf {w}} \sum_ {i = 1} ^ {N} \left\| \mathcal {M} _ {\mathbf {w}} \left(\mathbf {c} _ {\boldsymbol {\theta} ^ {*}} \left(\mathbf {y} _ {i}\right), \mathbf {z} _ {i}\right) - f \left(\mathbf {g} _ {\boldsymbol {\theta} ^ {*}} \left(\mathbf {y} _ {i}\right); \mathbf {z} _ {i}\right) \right\| \tag {4}
$$
$$
\begin{array}{l} \text {s . t .} \quad \boldsymbol {\theta} ^ {*} \in \operatorname {a r g m i n} _ {\boldsymbol {\theta}} \sum_ {i = 1} ^ {N} \mathcal {M} _ {\mathbf {w}} (\mathbf {c} _ {\boldsymbol {\theta}} (\mathbf {y} _ {i}), \mathbf {z} _ {i}). \end{array}
$$
The main motivation for this bi-level optimization formulation is that the surrogate model  $\mathcal{M}_{\mathbf{w}}$  is not concerned to be accurate for all possible  $\mathbf{c}_{\theta}$ ; rather, we are interested in finding the configuration of  $\mathcal{M}_{\mathbf{w}}$  that yields the closest approximation to  $f \circ \mathbf{g}$  near  $\theta^{*}$ . In other words,  $\mathcal{M}_{\mathbf{w}}$  serves as a surrogate loss that approximates  $\mathcal{L}$  at a certain landscape:  $\mathcal{M}_{\mathbf{w}}(Y,Z;\pmb{\theta}^{*}) \sim \mathcal{L}(Y,Z;\pmb{\theta}^{*})$ . By adopting the bi-level optimization framework, we can focus on learning accurate  $\mathcal{M}_{\mathbf{w}}$  for such  $\theta^{*}$ .
It is important to note that evaluating  $f \circ \mathbf{g}$  depends on both  $\mathbf{c}_{\theta}(\mathbf{y}_i)$  and  $\mathbf{z}_i$  (see eq. (2)). Therefore, to construct an accurate surrogate model  $\mathcal{M}_{\mathbf{w}}$  that can effectively approximate  $f \circ \mathbf{g}$ , it is essential for  $\mathcal{M}_{\mathbf{w}}$  to take into consideration the influence of both inputs.
To solve eq. 4, one could apply established methods from the bi-level optimization literature, such as [17, 42]. However, majority of them still rely on  $\nabla_{\theta}\mathcal{L}$  (or even  $\nabla_{\theta}^{2}$ ), which involves differentiating through the solver. To overcome this issue, we propose a simple and generic Algorithm 1, which is based on alternating optimization (high-level idea is depicted in fig. 1). The core idea is to simultaneously learn both mappings  $(\mathcal{M}_{\mathbf{w}}$  and  $\mathbf{c}_{\theta})$  to explore different solution spaces. By improving our target model  $\mathbf{c}_{\theta}$ , we obtain better estimates of the surrogate loss around the solution, and a better estimator  $\mathcal{M}_{\mathbf{w}}$  leads to better optimization of the desired loss  $\mathcal{L}$ . The use of alternating optimization helps both mappings reach a common goal. Furthermore, similarly flavored alternating optimization techniques were found to be successful in other problems with non-differentiable nature [44, 45].
In practice, the "inner" optimization (lines 10 and 12) does not have to be done "perfectly". In our approach, we perform a fixed (smaller) number of updates; that is, we do not precisely solve the minimization for both  $\mathcal{M}_{\mathbf{w}}$  and  $\mathbf{c}_{\theta}$ . For example, most experiments use 10-20 updates per  $t$ . One potential reason is that we do not want the models to overfit to the data; instead, we increase the total number of outer loop  $T$ . We discuss this further in the experimental setup.
Algorithm 1 Pseudocode for simultaneously learning LANCER and target model  $\mathbf{c}_{\theta}$ . Note that the algorithm may vary slightly based on setting (e.g.,  $\mathrm{P + O}$  and variations of SurCo), see Appendix B.
1: Input:  $\mathcal{D}_{\mathrm{train}}\gets \{\mathbf{y}_i,\mathbf{z}_i\}_{i = 1}^N$  , solver g, objective  $f$  , target model  $\mathbf{c}_{\theta}$    
2: Initialize  $\mathbf{c}_{\theta}$  (e.g. random, warm start);   
3: for  $t = 1\dots T$  do   
4: w-step (fix  $\pmb{\theta}$  and optimize over w):   
5: for  $(\mathbf{y}_i,\mathbf{z}_i)\in \mathcal{D}_{\mathrm{train}}$  do   
6: evaluate  $\hat{\mathbf{c}}_i = \mathbf{c}_{\theta}(\mathbf{y}_i)$  .   
7: evaluate  $\hat{f}_i = f(\mathbf{g}(\hat{\mathbf{c}}_i);\mathbf{z}_i)$  .   
8: add  $(\hat{\mathbf{c}}_i,\mathbf{z}_i,\hat{f}_i)$  to  $\mathcal{D}$  .   
9: end for   
10: solve minw  $\sum_{i\in \mathcal{D}}\left\| \mathcal{M}_{\mathbf{w}}(\hat{\mathbf{c}}_i,\mathbf{z}_i) - \hat{f}_i\right\|$  via supervised learning;   
11:  $\pmb{\theta}$  step (fix w and optimize over  $\pmb{\theta}$  ..   
12: solve min  $\pmb{\theta}$ $\sum_{i\in \mathcal{D}_{\mathrm{train}}}\mathcal{M}_{\mathbf{w}}(\mathbf{c}_{\pmb{\theta}}(\mathbf{y}_i),\mathbf{z}_i)$  via supervised learning.   
13: end for
Note that the Algorithm 1 avoids backpropagating through the solver or even through  $f$ . The only requirement is evaluating the function  $f$  at the solution of  $\mathbf{g}$ , which can be achieved by blackbox solver access. As a result, this approach eliminates the complexity and computational expense associated with computing derivatives of combinatorial solvers, making it a more efficient and practical solution as shown in Table 1.
It is worth noting that our framework shares similarities with actor-critic reinforcement learning [24]. In this analogy, we can think of  $\mathbf{c}_{\theta}$  as an actor responsible for making certain decisions, like predicting coefficients. On the other hand,  $\mathcal{M}_{\mathbf{w}}$  plays the role of a critic which evaluates the actor's decisions by estimating the objective value  $f$ , and provides feedback to the actor.
Table 1: Conceptual comparison of methods from related literature. Capabilities on the left are present or not for Methods on the top.  $\pm$  is given when this capability depends empirically on the problem at hand.
Feature\Method	LANCER	SurCo [16]	LODLs [37]	SPO+ [13]	DiffOpt [15,2,34]	Exact [1,36]
On-the-fly opt	+	+	-	-	-	+
Nonlinear f	+	+	+	-	+	+
Blackbox f	+	-	+	-	-	-
\( \partial f \) not required	+	-	+	+	-	+
\( \partial \mathbf{g}_{\theta} \) not required	+	-	+	+	-	+
Generalization	+	+	-	+	+	-
Few fast solver calls	±	±	-	±	±	-