Title: Solver-Free Decision-Focused Learning for Linear Optimization Problems
Venue: NeurIPS 2025
Accepted: yes
Keywords: Decision-Focused Learning

Predict-then-Optimize

Contextual Optimization

Linear Programming
Abstract: Mathematical optimization is a fundamental tool for decision-making in a wide range of applications. However, in many real-world scenarios, the parameters of the optimization problem are not known a priori and must be predicted from contextual features. This gives rise to predict-then-optimize problems, where a machine learning model predicts problem parameters that are then used to make decisions via optimization. A growing body of work on decision-focused learning (DFL) addresses this setting by training models specifically to produce predictions that maximize downstream decision quality, rather than accuracy. While effective, DFL is computationally expensive, because it requires solving the optimization problem with the predicted parameters at each loss evaluation. In this work, we address this computational bottleneck for linear optimization problems, a common class of problems in both DFL literature and real-world applications. We propose a solver-free training method that exploits the geometric structure of linear optimization to enable efficient training with minimal degradation in solution quality. Our method is based on the insight that a solution is optimal if and only if it achieves an objective value that is at least as good as that of its adjacent vertices on the feasible polytope. Building on this, our method compares the estimated quality of the ground-truth optimal solution with that of its precomputed adjacent vertices, and uses this as loss function. Experiments demonstrate that our method significantly reduces computational cost while maintaining high decision quality.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe method tackles improved training times for so-called predict-then-optimise problem, i.e. linear optimisation problems (LP) where the cost-function is unknown but rather needs to be predicted (an which usually require multiple LPs to be solved during training). The key technical contribution lies in a novel loss, called LAVA (loss via adjacent vertex alignment), which essentially favours clearer decision making w.r.t. the predicted cost. To this end, an optimal solution (vector), and its adjacent solution vectors are taken and the cost vector is pushed in such a direction (through back-propagation) that the decision towards the optimal solution vector is as clear as possible (i.e. the cost vector should not \"lean\" towards one of the adjacent solution vectors). The performance of the the proposed loss is evaluated on multiple instances of three classes of LPs.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- the paper is very well written, the problem is well motivated and key concepts are explained well and backed up by illustrations\n- compared to other approaches, the proposed method provides the best trade-off between training time and performance\n- code is claimed to be released\n- the main technical contribution is mathematically well-grounded \n\n## Weaknesses\n- the contribution of the paper seems limited to me. While the idea of the loss is well-grounded, its applicability to general predict-then-optimise problems seems not necessarily given as the optimal solution vector generally is not available (see also limitations).\n- the evaluation of the loss is limited as only smaller toy examples are used. It would be interesting to see the performance on large scale LPs in which solving the LP during training would be intractable computationally\n\nQUESTIONS:\n- l. 35: would be great to give examples on how suboptimal predictions can affect decision making\n- l. 58 - 59 repeat of \"set of optimal solution\"\n- l. 146-147: would be great to add that the fact that z is optimal if it has lower objective value than all its neighbours arises from the fact that the feasible region according to Ax = b is a >> convex << region\n- l.203: explicitly mention that that you refer to line 10 in algorithm 1\n- what is the computational complexity of algorithm 1?\n- why is normalised regret used as the metric to evaluate the performance?\n- who would the performance be on constrained shortest path problems (i.e. shortest path problems with additional constraints stating that \"whenever this edge is taken, the other edge also has to be taken\")?\n\nLIMITATIONS:\n- generally, an \"optimal\" solution z^star might not be available, i.e. for a given dataset it might be unclear what z^star is (since the cost vector c first has to be predicted) and with that restricts the LAVA loss to the special case where this is clear which in turn poses limited applicability of the method\n- for larger optimisation problems (millions of variables) it might be computationally intractable to find adjacent vertices", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a solver-free decision-focused learning (DFL) method for linear optimization problems. Specifically, it considers the prediction-then-optimize problem setting, where we need to train a machine learning model to predict the unknown parameters of an optimization problem, and then solve the optimization problem with predicted parameters. \nDFL is typically computationally expensive as it requires solving the optimization problem with the predicted parameters at each loss evaluation. To address this challenge, this paper proposes a Loss via Adjacent Vertex Alignment (LAVA), which evaluates the predicted cost vector without explicitly solving the optimization problem. Based on the insight that a solution is optimal if and only if it achieves an objective value that is at least as good as that of its adjacent vertices on the feasible polytope, the proposed loss measures how the derived objective of the how well the predicted cost vector distinguishes a true optimal solution from its adjacent vertices.\nExperiments demonstrate that the proposed method can achieve a favorable trade-off between computational efficiency and predictive performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1.\tThe presentation is good, including the problem formulation, the motivation statement, and the method illustration.\n2.\tThe proposed method mitigates the need to solve the problem and is motivated by the theoretical property presented in Proposition 4.1. The idea makes sense and is easy to implement.\n\nWeaknesses\n1.\tAlthough the loss function is motivated by Proposition 4.1, it would be better if the authors could present more theoretical analysis on the impact of the loss function impact on the final objective. For example, is it possible to present a theorem that if the loss is fewer than some threshold then the final objective would have a quality guarantee?\n2.\tAccording to the experiments on shortest path, the degeneracy may hinder the application of the proposed method. It would be better to conduct experiments on more datasets with degeneracy and to further investigate the relationship between the degree of degeneracy and the performance.\n3.\tIn the experiments, only linear regression is considered as the predictor. I wonder whether stronger predictors would improve the prediction accuracy and thus improve the final objective performance. It would be better to investigate the influence of the prediction accuracy on the final performance.\n\nQUESTIONS:\n1.\tAlthough the proposed method may achieve a favorable trade-off between computational efficiency and performance, is it possible to sacrifice training efficiency for better performance, or vice versa? If not, how could we conclude that the proposed method is better than others that are also on the Pareto frontier?\n2.\tIt would be better to present some more real-world applications of the considered settings, especially the incomplete information setting.\n\nLIMITATIONS:\nThis paper focuses on linear optimization problems. However, it seems nontrivial to extend the proposed method to harder problems like mixed-integer linear programs, as it leverages the special properties of linear optimization. Moreover, it focuses on linear regression as the predictor. These limitations may hinder its broad application.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new loss called LAVA for decision-focused learning (DFL), which enables solver-free training. In DFL, the parameters of a statistical model are chosen to optimize performance on a downstream optimization task, typically a linear program (LP) or mixed-integer linear program (MILP) whose cost vector $c$ must be predicted. In this setting, the non-differentiability of the mapping from cost $c$ to (MI)LP solution $z^{\\star}(c)$ is the main challenge limiting gradient-based training. While most existing methods require calling the optimization solver or a relaxation of it, LAVA instead relies on the geometric structure of the feasible set (a polytope) to evaluate the quality of a cost vector prediction $\\hat{c}$. If the predicted $\\hat{c}$ favors one of the neighbor vertices of $z^{\\star}(c)$, then it is suboptimal and penalized as such. The resulting loss has nice theoretical properties (differentiability, convexity) and can be obtained very efficiently once the set of adjacent vertices for each optimal solution in the training set has been precomputed. Experiments on three benchmark problems against a variety of DFL algorithms demonstrate that LAVA reaches a new Pareto optimum between training time and solution quality.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n### Quality\n\n- The LAVA loss is very fast to compute, differentiable, convex, and grounded in an understanding of polytope geometry, which makes it a very promising addition to the DFL arsenal.\n- Initial experimental evaluation is very convincing.\n\n### Clarity\n\n- The exposition is remarkably clear and precise, as is the motivation.\n\n### Significance\n\n- This paper single-handedly changed my mind on solver-free approaches. My previous opinion was that they cannot work since they do not leverage the geometry of the problem, but LAVA proves that geometry awareness does not have to be mediated by solver calls. I strongly expect it to become a reference approach in this field.\n\n### Originality\n\n- The authors did genuinely engage with research on linear programming to precompute neighbors for degenerate vertices. This cross-fertilization between operations research and machine learning is more than welcome.\n- The comparison with CaVE is enlightening: while both methods focus on aligning the objective vector with the normal cone of the optimal vertex, LAVA gets rid of the costly projection step at the cost of additional precomputation.\n\n## Weaknesses\n\n### Clarity\n\n- Figure 4 is slightly hard to read, especially for colorblind readers like myself different linestyles would be helpful here.\n\n### Significance\n\n- My main worry about this method is whether it can truly scale to large instances, because all of the experiments are on very small problems. When facing larger instances, not only would degeneracy become much more of an issue: computational bottlenecks while solving the linear system $B^{-1} N$ might be encountered.\n\nQUESTIONS:\n- Do you have any idea how to generalize this approach to the integer case? The CaVE paper discusses this aspect with their optimal subcone, does any of their insights generalize to your method?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- This paper proposes a novel, solve-free decision-focused learning method that eliminates the need to solve optimization problems during the learning process.\n- Focusing on Linear Programs (LPs), the research leverages the property that if the feasible region is non-empty and bounded, at least one optimal solution exists at an extreme point of the polyhedron.\n- Based on this property, the paper introduces a new loss function, LAVA (Loss based on Adjacent Vertices Approximation), which is derived from the difference in objective function values with extreme points adjacent to an observed solution.\n- The proposed regret minimization approach offers the advantage of not requiring an optimization problem to be solved. Numerical experiments demonstrate that the proposed method enables fast learning, especially for non-degenerate problem instances.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n\t- Decision-focused learning, which involves training prediction models by considering the performance of optimal solutions, is a major topic in machine learning. The computational time involved in decision-focused learning is a widely recognized practical challenge. This paper proposes an efficient learning method that leverages the polyhedral structure of LPs, showing a significant reduction in computation time for small-scale problems. This result appears to be a valuable contribution within the decision-focused learning framework.\n\t- Beyond demonstrating the effectiveness of the proposed method, the experiments carefully address and discuss the computational bottleneck of calculating adjacent extreme points when problems are degenerate. The experimental results presented in this research are well-argued and convincing.\n\t- The paper itself is well-written and easy to read. \n- Weaknesses\n\t- The effect of introducing a margin in the proposed method isn't well understood. The paper states that introducing a margin stabilizes the training process, and Appendix B shows it reduces variance and improves the average behavior of regret. However, the reason why such a margin can contribute to stabilizing the learning process needs a more thorough discussion.\n\t- While it is inevitable, given the nature of the proposed method, it strongly relies on the properties of LPs. Consequently, its extensibility to more general Mixed-Integer Programs (MIPs) or convex optimization problems in a decision-focused context appears limited.\n\nQUESTIONS:\n- As for the first point in the weakness I raised, the rationale behind the margin parameter's contribution to stabilizing the learning process isn't entirely clear. Could the authors explain in more detail how this margin helps stabilize learning?\n- In Line 168, the authors state that LAVA is differentiable. Is this really true? LAVA is a point-wise maximum of linear functions for $\\hat{c}$, so I understand that while its subgradient can be calculated, it is not generally differentiable. Please clarify if I've misunderstood.\n- Can the proposed method be applied even when the observed solution is not an extreme point of the polyhedron? If an observed optimal solution lies on a facet and is not an extreme point, is it still possible to enumerate adjacent extreme points?\n- Figure 4 shows that the training time of the proposed method increases more quickly when the number of items changes from 200 to 400. Might the computational time advantage of the proposed method disappear when dealing with more than 400 items?\n- When training a model to predict the cost vector $c$ by minimizing LAVA, it appears that a model consistently predicting a zero vector for $c$ might be optimal. Such a model would render all feasible solutions optimal, which would be practically worthless in decision-focused learning. Is my understanding correct? Or does the proposed method naturally avoid such an undesirable model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}