SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Similar to many other approaches designed for solving problems with bi-level optimizations, including actor-critic algorithms, we currently lack theoretical guarantees regarding the convergence of LANCER with respect to either  $\mathcal{M}_{\mathbf{w}}$  or  $\mathbf{c}_{\theta}$ . This lack of guarantees can also impact the stability of the algorithm. However, this is primarily a matter that can be addressed through empirical investigation and depends on various factors, which is typical in the process of model selection. In Appendix E, we present ablation studies that we conducted to evaluate the stability of our approach across various hyperparameters and neural network architectures. These, along with our main experimental results in section 5, demonstrate that LANCER generally exhibits robustness in many cases.
4.1 Reusing landscape surrogate model  $\mathcal{M}_{\mathrm{w}}$
Once Algorithm 1 finishes, we usually discard  $\mathcal{M}_{\mathbf{w}}$  as it is an intermediate result of the algorithm, and we only retain  $\mathbf{c}_{\theta}$  (and solver  $\mathbf{g}$ ) for model deployment. However, we have found through empirical exploration that the learned surrogate loss  $\mathcal{M}_{\mathbf{w}}$  can be reused for a range of problems, increasing the versatility of the approach. This is particularly advantageous for SurCo setting, where we handle one instance at a time. In this scenario, we utilize the trained  $\mathcal{M}_{\mathbf{w}}$  for unseen test instances by executing only the  $\theta$ -step of Algorithm 1. The main advantage of this extension is that it eliminates the need for access to the solver  $\mathbf{g}$ , leading to significant deployment runtime improvements.
4.2 Reusing past evaluations of  $f \circ \mathbf{g}_{\theta}$
In LANCER, the learning process of  $\mathcal{M}_{\mathbf{w}}$  is solely reliant on  $\mathcal{D}$  and is independent of the current state of  $\mathbf{c}_{\theta}$ . Put simply, to effectively learn  $\mathcal{M}_{\mathbf{w}}$ , we only need the inputs and outputs of  $f \circ \mathbf{g}_{\theta}$ , namely  $\mathbf{c}_{\theta}(\mathbf{y}_i)$ ,  $Z$ , and the corresponding objective value  $\hat{\mathbf{f}}$ . Interestingly, we can cache the predicted descriptions themselves,  $\mathbf{c}_{\theta}(\mathbf{y}_i)$ , without the need for the model  $\theta$  or problem information. This caching mechanism allows us to reuse the data  $(\mathbf{c}_{\theta}(\mathbf{y}_i), \mathbf{z}, \hat{\mathbf{f}})$  from previous iterations  $(1 \dots T - 1)$  as-is. By adopting this practice, we enhance and diversify the available training data for  $\mathcal{M}_{\mathbf{w}}$ , which proves particularly advantageous for neural networks. This concept bears resemblance to the concept of a replay buffer [27] commonly found in the literature on Reinforcement Learning.
We validate our approach (LANCER) in two settings: smart predict+optimize and learning surrogate costs for MINLP. For each setting, we study a range of problems, including linear, nonlinear, combinatorial, and others, encompassing both synthetic and real-world scenarios. Overall, LANCER exhibits superior or comparable objective values while maintaining efficient runtime. Additionally, we perform ablation studies, such as re-using  $\mathcal{M}$ .
5.1.1 Combinatorial optimization with linear objective
The shortest path (SP) and multidimensional knapsack (MKS) are both classic problems in combinatorial optimization with broad practical applications. In this setting, we consider a scenario where problem parameters  $\mathbf{z}$ , such as graph edge weights and item prices, cannot be directly observed
Figure 2: Normalized test regret (lower is better) for different  $\mathrm{P + O}$  methods: 2-stage, SPO+ [13], DBB [34], LODLs [37] and ours (LANCER). Overlaid dark green bars (right) indicate that the method warm started from the solution of 2stg. DBB performs considerably worse on the right benchmark and is cut off on the  $y$ -axis.
during test time, and instead need to be estimated from  $\mathbf{y}$  via learnable mapping  $\mathbf{z} = \mathbf{c}_{\theta}(\mathbf{y})$ . That is, we consider smart  $\mathrm{P + O}$  setting.
Setup We use standard linear program (LP) formulation for SP and mixed integer linear program (MILP) formulation for MKS. Observed features  $\mathbf{y}$  and the corresponding ground truth problem descriptions  $\mathbf{z}$  for SP are generated using the same procedure as in [38]: grid size of  $5 \times 5$ , feature dimension of  $\mathbf{y} \in \mathbb{R}^5$  (obtained using a random linear mapping from  $\mathbf{z}$ ), 1000 instances for both train and test. For MKS, we increased the knapsack dimension to 5 and capacity to 45, and we set the number of items to 100. Moreover, we use randomly initialized MLP (1 ReLu hidden layer) to generate features of dimension  $\mathbf{y} \in \mathbb{R}^{256}$ . As for the baselines, apart from the naive 2-stage approach, we have SPO+ [13] and DBB [34], both implemented in PyEPO [38] library. Furthermore, we added LODLs, a novel method from Shah et al. [37]. We explored as best as we could all important hyperparameters for all methods on a fixed cross-validation set. Regarding the LANCER, we utilize MLP with 2 tanh hidden layers of size 200 for surrogate model  $\mathcal{M}$ . We set  $T = 10$  and the number of updates for  $\mathcal{M}$  and  $\mathbf{c}_{\theta}$  is at most 10. We use SCIP [1] to solve LP for the shortest path and MILP for knapsack. Further details can be found in Appendix C.1.1.
Results The results are summarized in fig. 2. We report the normalized regret as described in [38]. The findings indicate that LANCER and SPO+ consistently outperform the two-stage baseline, particularly when considering the warm start. As SPO+ is specifically designed for linear programs, it provides informative gradients, making it a robust baseline. Even in MKS, where theorems proposed in [13] are no longer applicable, SPO+ performs decently with minimal tuning effort. The DBB approach, however, demonstrates unsatisfactory default performance but can yield favorable outcomes with proper initialization and tuning (see the right plot). Interestingly, the other  $\mathrm{P + O}$  baselines, initialized randomly, were unable to outperform a naive 2stg in both benchmarks.
LANCER achieves superior performance in both tasks, with a noticeable advantage in MKS. This may be attributed to the high dimension of the MKS problem and the large feature space  $(\mathbf{y})$ . One possible explanation is that the sparse gradients of the derivative-based method make the learning problem harder, whereas LANCER models the landscape of  $f \circ g$ , providing informative gradients for  $\mathbf{c}_{\theta}$ .
5.1.2 Combinatorial optimization with nonlinear objective
In this section, we apply LANCER for solving mixed integer nonlinear programs (MINLP). Specifically, we transform a combinatorial problem with a nonlinear objective into an instance of MILP via learning linear surrogate costs as described in Ferber et al. [16]. Note that in this setting, we assume that the full problem description  $\mathbf{y} = \mathbf{z}$  is given and fully observable (in contrast to the  $\mathrm{P + O}$  setting).
We begin by examining on-the-fly optimization, where each problem is treated independently. In this scenario, the cost vector  $\mathbf{c}_{\theta}(\mathbf{y})$  simplifies to a constant value  $\mathbf{c}$ . SurCo is then responsible for directly training the cost vector  $\mathbf{c}$  of the linear surrogate. As we lack a distribution of problems to train  $\mathcal{M}$ , specific adaptations to Algorithm 1 are necessary, which are outlined in detail in Appendix B. We refer to this version as LANCER-zero to be consistent with SurCo-zero.
Grid 5x5
Grid 15x15
Figure 3: Results on stochastic shortest path using different grid sizes: 5x5 (left) and 15x15 (right). We report avg objective values (higher is better) on three settings described in [16]. For grid size of 15x15, SCIP [1] was unable to finish within the 30 min time limit.
Setup Nonlinear shortest path problems arise when the objective is to maximize the probability of reaching a destination before a specified time in graphs with random edges [26, 14]. The problem formulation is similar to the standard linear programming (LP) formulation of the shortest path, as described in section 5.1.1, with a few adjustments: 1) the weight of each edge follows a normal distribution, i.e.,  $w_{e} \sim \mathcal{N}(\mu_{e}, \sigma_{e})$ ; 2) the objective is to maximize the probability that the sum of weights along the shortest path is below a threshold  $W$ , which can be expressed using the standard Gaussian cumulative distribution function (CDF),  $P\left(\sum_{e \in E} w_{e} \leq W\right) = \Phi\left(\frac{\left(W - \sum_{e \in E} \mu_{e}\right)}{\sqrt{\sum_{e \in E} \sigma_{e}}}\right)$  where  $E$  is the set of edges belonging to the shortest path. We use  $5 \times 5$  and  $15 \times 15$  grid graphs with 25 draws of edge weights. We set the threshold  $W$  to three different values corresponding to loose, normal, and tight deadlines. The remaining settings are adapted from Ferber et al. [16], and additional details can be found in Appendix C.1.2.
Results Fig. 3 illustrates the performance of different methods in both grid sizes. SCIP directly formulates the MINLP to maximize the CDF, resulting in an optimal solution. However, this approach is not scalable for larger problems and is limited to smaller instances like the  $5 \times 5$  grid. The heuristic method assigns each edge weight as  $w_{e} = \mu_{e} + \gamma \sigma_{e}$ , where  $\gamma$  is a user-defined hyperparameter, and employs standard shortest path algorithms (e.g., Bellman-Ford). As the results indicate, this heuristic approach produces highly suboptimal solutions. SurCo-zero and LANCER-zero demonstrate similar performance, with LANCER-zero being superior in almost all scenarios.
5.2 Real-world use case: quadratic and broader nonlinear portfolio selection
5.2.1 The quadratic programming (QP) formulation
In this study, we tackle the classical quadratic Markowitz [30] portfolio selection problem. We use real-world data from Quandl [35] and follow the setup described in Shah et al. [37]. The prediction task leverages each stock's historical data  $\mathbf{y}$  to forecast future prices  $\mathbf{z}$ , which are then utilized to solve the QP (i.e.,  $\mathrm{P} + \mathrm{O}$  setting). The predictor is the MLP with 1 hidden layer of size 500.
Setup We follow a similar setup described in [37], except for a fix in the objective's quadratic term that slightly affects the decision error's magnitude. More details can be found in Appendix C.2.1. We compare LANCER against
Table 2: Portfolio selection normalized test decision loss (lower is better).
Method	Test DL
Random	1
Optimal	0
2-Stage	0.57 ± 0.02
LODLs [37]	0.55 ± 0.02
MDFL [41]	0.52 ± 0.01
LANCER	0.53 ± 0.02
two-stage and LODLs. However,  $\mathrm{SPO + }$  is not applicable in this nonlinear setting, and DBB's performance is notably worse since it is designed for purely combinatorial problems. Additionally, we report the optimal solution (using the ground truth values of  $\mathbf{z}$ ) and the (Melding Decision Focused Learning) MDFL [41] method, which leverages the implicit function theorem to differentiate through the KKT conditions. For our LANCER implementation, we use an MLP with 2 hidden layers for  $\mathcal{M}$
and update each of the parametric mappings 5 times per iteration with a total of  $T = 8$  iterations. We use cvxpy [10] to solve the QP.
Results Table 2 summarizes our results. We report the normalized decision loss (i.e., normalized Eqn. (2)) on test data. Since the problem is smooth and exact gradients can be calculated, MDFL achieves the best performance closely followed by the LANCER. The remaining results are in agreement with [37]. While LANCER does not achieve the best overall performance, it does so using a significantly smaller number of calls to a solver, as we discuss in more detail in Section 5.3.
5.2.2 Combinatorial portfolio selection with third-order objective
The convex portfolio optimization problem discussed in the previous section 5.2.1 is unable to capture desirable properties such as logical constraints [5, 15], or higher-order loss functions [20] that integrate metrics like co-skewness to better model risk. We use the Quandl [35] data (see Appendix C.2.2 for details on setup) and similar to section 5.1.2, we assume that the full problem description  $\mathbf{z}$  is given at train/test time.
Results are shown in fig. 4. We first tried to solve the given MINLP exactly via SCIP. However, it fails to produce the optimal solution within a 1 hour time limit and we report the best incumbent feasible solution. MIQP (blue squares) and MILP (blue triangles) approximations overlook the co-skewness and non-linear terms, respectively. Comparing their performance, MIQP exhibits a  $2 \times$  lower loss than the MILP baseline but in a significantly longer
Figure 4: Objective (lower is better) and deployment runtime for combinatorial portfolio selection problem. For LANCER-zero and SurCo-zero, numbers at each point correspond to the number of iterations.
runtime. For LANCER and SurCo, we present results for two scenarios: learning the linear cost vector  $\mathbf{c}$  directly (zero) for each instance, and a parameterized version  $\mathbf{c}_{\theta}(\mathbf{z})$  (prior). The main distinction of "prior" is that no learning occurs during test time, as we directly map the problem descriptor  $\mathbf{z}$  to a linear cost vector and solve the MILP. Consequently, the deployment runtime is similar to that of the MILP approximation, but LANCER-prior produces slightly superior solutions. Remarkably, LANCER-zero achieves significantly better loss values, surpassing all other methods. Although it takes longer to run, the runtime remains manageable, and importantly, the solution quality improves with an increasing number of iterations.
According to our experimental findings, LANCER demonstrated the most substantial improvement in this specific setting. Encouraged by this success, we conducted a deeper exploration of the complexities of this problem, as detailed in Appendix D.
5.3 Computational efficiency
Comparing baseline methods, including LANCER, we find that querying solver  $\mathbf{g}_{\theta}$  is the primary computational bottleneck. To evaluate this aspect, we empirically analyze different algorithms on various benchmarks in the  $\mathrm{P + O}$  domain. The results, depicted in fig. 5, highlight that LODLs require sampling a relatively large number of points per training instance, leading to potentially time-consuming solver access. On the other hand, gradient-based methods like DBB, MDFL, and SPO+ typically solve the optimization problem 1-2 times per update but require more iterations to converge. In contrast, LANCER accesses the solver in the w-step, with the number of accesses proportional to the training set size and a small total number of alternating optimization iterations. Moreover, we leverage saved solutions from previous iterations, akin to a replay buffer, when fitting  $\mathcal{M}$ . These combined factors allow us to achieve favorable results with a small value of  $T$ .
Figure 5: Trade-off curves between black-box solver calls (MILP or QP) vs decision loss (or regret) on  $\mathrm{P} + \mathrm{O}$  problems. Point labels (e.g. 1,4,7) correspond to the epoch; except for LODLs, where they correspond to the number of samples per instance. Each algorithm uses a different number of BB calls per epoch.
Table 3: Results of reusing  $\mathcal{M}$  on stochastic shortest path problem from fig. 3 ( $15 \times 15$  grid). Here, "reused  $\mathcal{M}_{\mathbf{w}}$  has limited access to the solver  $\mathbf{g}$ , and thus is much faster while retaining solution quality.
Method	Loose Deadline	Normal Deadline	Tight Deadline
obj.	time (s)	obj.	time (s)	obj.	time (s)
LANCER-zero	0.556 ± 0.006	61.1 ± 3.2	0.497 ± 0.004	62.3 ± 2.9	0.434 ± 0.005	62.8 ± 2.7
LANCER-reused Mw	0.556 ± 0.007	2.9 ± 0.3	0.496 ± 0.004	2.7 ± 0.6	0.432 ± 0.004	2.5 ± 0.6
5.4 Reusing landscape surrogate  $\mathcal{M}_{\mathrm{w}}$
In this scenario, we introduce a dependency of  $\mathcal{M}_{\mathbf{w}}$  on both the predicted linear cost (c) and the problem descriptor (y), as described in section 4.1. This enables us to reuse  $\mathcal{M}_{\mathbf{w}}$  for different problem instances without retraining, and eliminate the dependency on the solver  $\mathbf{g}_{\theta}$ , giving LANCER a substantial runtime acceleration. To validate this hypothesis, we pretrain  $\mathcal{M}_{\mathbf{w}}$  using 200 instances of the stochastic shortest path on a  $15 \times 15$  grid by providing concatenated  $(\mathbf{c}, \mathbf{y})$  as input. We apply LANCER-zero to the same test set as before and present results in fig. 3, demonstrating comparable performance between these two approaches, with "reused  $\mathcal{M}_{\mathbf{w}}$  " being much faster.
This paper makes a dual contribution. Firstly, we derive a unified training procedure to address various coupled learning and optimization settings, including smart predict+optimize and surrogate learning. This is significant as it advances our understanding of learning-integrated optimization under partial information. Secondly, we propose an effective and powerful method called LANCER to tackle this training procedure. LANCER offers several advantages over existing literature, such as versatility, differentiability, and efficiency. Experimental results validate these advantages, leading to significant performance improvements, especially in high-dimensional spaces, both in problem description and feature space. One potential drawback is the complexity of tuning  $\mathcal{M}$ , requiring model selection and training. However, future research directions include addressing this drawback and exploring extensions of LANCER, such as applying it to fully black box  $f$  scenarios.
The research at the University of Southern California was supported by the National Science Foundation (NSF) under grant number 2112533. We also thank the anonymous reviewers for helpful feedback.
[1] T. Achterberg. SCIP: Solving constraint integer programs. Mathematical Programming Computation, 1(1):1-41, July 2009. ISSN 1867-2957. doi: 10.1007/s12532-008-0001-1.
[40] B. Wilder, E. Ewing, B. Dilkina, and M. Tambe. End to end learning and optimization on graphs. In Advances in Neural and Information Processing Systems, 2019.
[41] B. Wilder, B. Dilkina, and M. Tambe. Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In AAAI Conference on Artificial Intelligence (AAAI 2020), 2020.
[42] M. Ye, B. Liu, S. Wright, P. Stone, and Q. Liu. Bome! bilevel optimization made easy: A simple first-order approach. In Advances in Neural Information Processing System, 2022.
[43] E. Yuan, W. Chen, and P. Van Hentenryck. Reinforcement learning from optimization proxy for ride-hailing vehicle relocation. Journal of Artificial Intelligence Research, 75:985-1002, 2022.
[44] A. Zharmagambetov and M. Á. Carreira-Perpínan. Learning interpretable, tree-based projection mappings for nonlinear embeddings. In International Conference on Artificial Intelligence and Statistics, pages 9550–9570, 2022.
[45] A. Zharmagambetov and M. Á. Carreira-Perpínán. Semi-supervised learning with decision trees: Graph Laplacian tree alternating optimization. In Advances in Neural Information Processing System, 2022.
A Computational complexity
It is quite straightforward to estimate the runtime of our approach from Algorithm 1 in the main paper. Let us denote  $I_{\mathbf{g}}$  as the time to get the solution from solver  $\mathbf{g}(\mathbf{y}_i)$ . Also, let  $I_{\mathbf{w}}$  and  $I_{\theta}$  be the runtime of training the parametric loss  $\mathcal{M}_{\mathbf{w}}$  and the target model  $\mathbf{c}_{\theta}$ , respectively. Then, assuming evaluating  $f$  is negligible, one iteration of our algorithm naively takes  $\mathcal{O}(N \cdot I_{\mathbf{g}} + I_{\mathbf{w}} + I_{\theta})$ , so the total runtime is  $\mathcal{O}(T \cdot N \cdot I_{\mathbf{g}} + T \cdot I_{\mathbf{w}} + T \cdot I_{\theta})$ . In practice, we iterate at most 100 times ( $T < 100$ ). Therefore, the main bottleneck is  $\mathcal{O}(T \cdot N \cdot I_{\mathbf{g}})$ , which is mainly overtaken by an access to  $\mathbf{g}$ . Although we claim that leveraging  $\mathcal{M}$  to learn  $\theta$  is efficient, we admit that there is still a requirement to access the solver  $\mathbf{g}$ .
However, there are several accelerations that can be made:
1.  $\mathcal{O}(N\cdot I_{\mathbf{g}})$  is embarrassingly parallel computation since each request to  $\mathbf{g}$  is independent and, additionally, subsampling on  $\mathcal{D}_{\mathrm{train}}$  can be performed.
2. We can "warm start" the solver  $\mathbf{g}$  from solutions obtained in  $t - 1$ , which typically yields faster convergence.
3. Although we did not test this, but one can "early stop" the solver  $\mathbf{g}$  if it is too costly to solve optimally (e.g. large scale MILP). Our hypothesis is that it is enough to obtain a feasible solution  $\hat{\mathbf{x}}$  in certain neighborhood of  $\mathbf{x}^*$ . Since we are still able to evaluate  $f$  and  $(f(\hat{\mathbf{x}}),\hat{\mathbf{c}},\mathbf{z})$  is a "valid" tuple, we can use it to train  $\mathcal{M}_{\mathbf{w}}$ .
4. Moreover, we empirically found out that the supervised learning steps (lines 10 and 12) do not require "perfect" learning. That is, we perform several gradient updates over  $\mathbf{w}$  and  $\theta$ , which significantly reduces  $I_{\mathbf{w}}$  and  $I_{\theta}$ .
B Pseudocodes
Algorithms 2-4 below closely resemble the Algorithm 1 in the main paper. However, there are minor variations that depend on the problem setting, whether it involves learning linear surrogates for MINLP or smart Predict+Optimize setting.
Algorithm 2 Pseudocode for learning LANCER and target model  $\mathbf{c}_{\theta}$  for smart Predict+Optimize setting. Note:  $\mathbf{y}$  - input (always observed) features,  $\mathbf{z}$  - ground truth problem descriptions (available at train time only).
1: Input:  $\mathcal{D}_{\mathrm{train}}\gets \{\mathbf{y}_i,\mathbf{z}_i\}_{i = 1}^N$  , solver g, objective  $f$  (can be black-box), target model  $\mathbf{c}_{\theta}$  ,(optional) prediction loss penalty  $\lambda$    
2: Initialize  $\mathbf{c}_{\theta}$  from solving:  $\min_{\theta}\sum_{i\in \mathcal{D}_{\mathrm{train}}}\| \mathbf{c}_{\theta}(\mathbf{y}_i) - \mathbf{z}_i\|$    
3: Set  $\mathcal{D}\gets \{\}$    
4: for  $t = 1\dots T$  do   
5: w-step (fix  $\pmb{\theta}$  and optimize over w):   
6: for  $(\mathbf{y}_i,\mathbf{z}_i)\in \mathcal{D}_{\mathrm{train}}$  do   
7: evaluate  $\hat{\mathbf{c}}_i = \mathbf{c}_{\theta}(\mathbf{y}_i)$    
8: evaluate  $\hat{f}_i = f(\mathbf{g}(\hat{\mathbf{c}}_i);\mathbf{z}_i)$    
9: add  $(\hat{\mathbf{c}}_i,\mathbf{z}_i,\hat{f}_i)$  to  $\mathcal{D}$    
10: end for   
11: minimize: minw  $\sum_{j\in \mathcal{D}}\left\| \mathcal{M}_{\mathbf{w}}(\hat{\mathbf{c}}_j,\mathbf{z}_j) - \hat{f}_j\right\|$    
12:  $\pmb{\theta}$  step (fix w and optimize over  $\pmb{\theta}$  .   
13: minimize: mino  $\sum_{i\in \mathcal{D}_{\mathrm{train}}}\mathcal{M}_{\mathbf{w}}(\mathbf{c}_{\theta}(\mathbf{y}_i),\mathbf{z}_i) + \lambda \| \mathbf{c}_{\theta}(\mathbf{y}_i) - \mathbf{z}_i\|$    
14: end for
Algorithm 3 Pseudocode for learning linear surrogates with LANCER-zero (used in sections 5.1.2 and 5.2.2). Note that  $\mathbf{y} = \mathbf{z}$  in this setting and we have only one optimization problem instance.
1: Input: problem description y, solver g, objective f (can be black-box);  
2: Initialize c ∈ RL;  
3: Set D ← {};  
4: for t = 1...T do  
5: • w-step (fix c and optimize over w):  
6: randomly sample {c}iN around c;  
7: for i = 1...N do  
8: evaluate f_i = f(g(c_i); y);  
9: add (c_i, f_i) to D;  
10: end for  
11: minimize: minw ∑j∈D ||Mw(cj) - fj||;  
12: • θ-step (fix w and optimize over c):  
13: // θ = c in this setting as we solve for a single problem instance y  
14: minimize: minc Mw(c).  
15: end for