Title: Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints
Venue: NeurIPS 2025
Accepted: yes
Keywords: decision-focused learning

predict-and-optimize

constrained optimization
Abstract: When some parameters of a constrained optimization problem (COP) are uncertain, this gives rise to a predict-then-optimize (PtO) problem, comprising two stages: the \textit{prediction} of the unknown parameters from contextual information and the subsequent \textit{optimization} using those predicted parameters. Decision-focused learning (DFL) implements the first stage by training a machine learning (ML) model to optimize the quality of the decisions made using the predicted parameters. When the predicted parameters occur in the constraints, they can lead to infeasible solutions. Therefore, it is important to simultaneously manage both feasibility and decision quality. We develop a DFL framework for predicting constraint parameters in a generic COP. While prior works typically assume that the underlying optimization problem is a linear program (LP) or integer LP (ILP), our approach makes no such assumption. We derive two novel loss functions based on maximum likelihood estimation (MLE): the first one penalizes infeasibility (by penalizing predicted parameters that lead to infeasible solutions), while the second one penalizes suboptimal decisions (by penalizing predicted parameters that make the true optimal solution infeasible). We introduce a single tunable parameter to form a weighted average of the two losses, allowing decision-makers to balance suboptimality and feasibility. We experimentally demonstrate that adjusting this parameter provides decision-makers control over this trade-off. Moreover, across several COP instances, we show that adjusting the tunable parameter allows a decision-maker to prioritize either suboptimality or feasibility, outperforming the performance of existing baselines in either objective.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGenerally, this paper proposed a trade-off learning approach for decision-focused learning/prediction. The penalization for infeasibility and suboptimality is balanced through a convex combination of them. The effectiveness of the proposed penalization is proven theoretically. Experimental results show the performance of the approach under different trade-off parameters. \n\nThe proposed method is well-formulated and empirically supported, and the authors clearly state that \"it is important to simultaneously manage both feasibility and decision quality.\" While the tunable trade-off parameter introduced in the paper provides a flexible way for decision-makers to control this balance, its practical utility could be further improved by offering guidance on how to align the parameter with real-world decision-making preferences. In particular, when constraint parameters are implicit or arise from black-box systems, it becomes nontrivial for decision-makers to calibrate the trade-off between feasibility and suboptimality without a structured mechanism or interpretive support. As acknowledged in the paper, decision-makers may wish to “control the trade-off between suboptimality and infeasibility” according to their preference, and designing data-driven or preference-informed ways to adapt this parameter could enhance the applicability of the approach in practical settings.\n\nSTRENGTHS AND WEAKNESSES:\nThe quality of this paper is generally high. The present is clear; perhaps more results should be discussed. The approach proposed in this paper doesn’t show high/strong significance compared with other existing works. From the point of view of the reviewer, the work in this paper is original. Specifically, the derivation of the two loss functions (one penalizes infeasibility and the other penalizes suboptimality) is principled and well-grounded in a maximum likelihood framework. The proposed method is general enough to accommodate nonlinear and integer programs, unlike many prior works restricted to LPs/ILPs. Experiments demonstrate that tuning the trade-off parameter $\\alpha$ can achieve a flexible balance between regret and feasibility, which may align with different decision-maker preferences.\n\nWhile the proposed formulation is sound and the empirical results are convincing, the paper could benefit from a deeper discussion on how decision-makers can select or adapt the trade-off parameter $\\alpha$ in practice. This is particularly important because α encodes a key behavioral preference — how to weigh feasibility versus optimality under uncertainty — but is currently treated as an external hyperparameter, selected by trial. More specifically, in practical applications, it is challenging for decision-makers to specify their preferences through the hyperparameter $\\alpha$ without clear guidance, which may limit the practical utility of the proposed method.\n\nQUESTIONS:\n1. The essence of the penalization is the amount of constraint violation and margin, after ReLU approximation. Considering the value range of the different constraint functions is different, a proper normalization approach should be proposed. \n2. For equality constraints, the higher buffer (constraint margin parameter) is likely to kill their influence on the loss function. Meanwhile, the decision makers’ concern about the violation direction may be different; this concern could possibly solve this problem.\n3. The selection of the convex combination trade-off parameter should be discussed. \n4. A little more related work should be discussed and cited.\n\nLIMITATIONS:\nYes, the biggest limitation of the proposed method is that it requires solving the programming problem during the training process of the prediction model.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is in the domain of Decision Focused Learning, where we have to predict parameters of some optimisation problem and subsequently solve it. Instead of minimising the error in predicted parameters, Decision Focused learning minimises the error in decision outcomes from the optimisation problem. However, in a constrained optimisation problem, if predicted parameters also include the constraints, then such a DFL setup may lead to infeasible solutions. This work proposes DFL in constrained optimisation problems by proposing two loss functions -  Infeasibility Penalty Loss (IPL) and Optimality-Preserving Loss (OPL). By adjusting the weighted combination of these two losses, the end user can decide the tradeoff between feasibility and optimality. The paper also showcases empirical results on two constrained optimisation problems - a multi-dimensional knapsack problem and a covering LP problem. Compared to baselines, the proposed method precisely allows controlling performance in feasibility or optimality dimensions based on the loss combination weight.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe paper is well written and easy to follow, and the problem formulation is comprehensive. Claims on the correctness of proposed loss functions are supported by proofs.\n\nThe problem of predicting constraint parameters in optimisation problems as a single-stage decision-focused learning problem is important. Unlike prior work, the proposed method in the paper can handle broad classes of constrained optimisation problems.\n\nLikelihood perspective to DFL, i.e., maximising likelihood of the predicted parameters leading to an optimal solution, is an interesting framing, especially because it encapsulates both the objective function parameters and the constraint parameters.\n\n\n\nWeaknesses:\n\nWhile the paper claims that the method is applicable for a general class of constrained optimisation problems (even outside linear programs and mixed integer programs), the two problem settings in the empirical results seem like somewhat small examples and are rather limited. Prior works in DFL, for instance, have often used real-world data from healthcare, finance, etc,  which model the relationship between real-world features and predicted parameters of optimisation problems. It would be useful to expand the set of experiments to such scenarios.\n\nIt would be useful to interpret the reasons why and in what scenarios the SFL or existing method prefer feasibility or optimality. For example, in Figure 1, we see that SFL chooses a solution that balances the tradeoff in two dimensions for the ‘predicting capacities in MDKP’ problem, but it favours minimising regret in the other two problems. No reasoning is provided for why such behaviour happens (and the interpretation of parameters which lead to the proposed method’s control of the tradeoff).\n\nQUESTIONS:\nQuestions and Comments:\n\n5 seeds seem to be a low number for the empirical experiments. It would be useful to run more seeded experiments.\n\nWhat would be the kinds of scenarios where a decision-maker would choose feasibility over optimality (or vice versa)? While the paper shows that the proposed technique allows precise control over the tradeoff between the two, it is not motivated why we want this control and further unclear how the decision maker will choose the mixing weight. A decision on these questions would be critical to evaluate the significance of the contributions of this paper.\n\nIn the Optimality-Preserving Loss formulation described in lines 214-216, when you replace a feasible point x with x*, it is a bit unclear how much the loss would be sensitive to the choice of the feasible point x?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new Decision-Focused Learning (DFL) approach that specifically addresses learning/predicting constraint parameters for (combinatorial) optimization problems. It should be noted that the vast majority of research on DFL deals with predicting objective function parameters, and only few publications deal with DFL for constraints. Predicting constraint parameters is challenging since in case of hard constraints wrong predictions can lead to infeasible solutions. The paper introduces a new loss function consisting of a convex combination of two (also new) loss functions that explicitly address infeasibility: The first loss function (IAL) aims at minimizing the probability that that the estimated parameters induce a constraint violation, the second loss function (OPL) aims at minimizing the probability that the (true) optimal solution is violated in the optimization problem parameterized by the predicted parameters. The relative weight of the two loss functions in the compound loss function can be adjusted according to the preferences of the decision maker.\nIn a set of computational experiments with the multidimensional knapsack problem and an alloy production problem, it is shown that by varying the weight parameter of the loss function, it is possible to navigate the trade-off between a small regret and a small probability of yielding infeasible solutions.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1.\tThe paper proposes an innovative approach for DFL with uncertain constraint parameters.\n2.\tThe new loss functions are very adequate and seem to work well\n3.\tThe idea of introducing a parameter for balancing the trade-off between small regrets and probability of infeasible is very good\n4.\tThe computational results show that the approach allows to trade optimality vs feasible using the introduced parameter, and for some settings of the trade-off weight, the approach seems to dominate alternative approaches\n\n### Weaknesses:\n1.\tFor me, an important concern with the paper is that it does not properly account the approach from Hu et al 2023, NeurIPS) (Reference 13 in the submission), both when describing the approach and also in the computational experiment.    While it is true that the approach is designed as a two-stage approach with so-called corrective actions / penalty functions, the paper from Hu clearly describe how to use it in a setting with hard constraints, the details can be found in the Appendix .. of Hu ; they use an indicator function combined with a penalty in order to make predictions inducing infeasibilities less likely. In that appendix, there is even a small set of computational results with the 0/1 knapsack problem in which they show that increasing the penalty increases the empirical probability of achieving feasible solutions (e.g. a probability of >= 99% in case of a penalty of 4. \nAs a result, I would have expected the authors to include results with the penalty function mentioned in the Appendix of Hu et al., and with much higher penalty values: In the present manuscript, you use small penalty values that were used by Hu et al in a setting with corrective actions, which is not appropriate.\n2.\tThe authors also write that existing approaches do not allow adjusting the prediction based on their subjective preference between optimality and feasibility. Given the previous remark, by adjusting the penalty weight when using the penalty function for hard constraints, the approach does allow that.\n3. The \"buffer\" parameter is introduced in the paper, but the paper gives no hint on how to choose it, although I feel it will have a considerable impact on the prediction.\n\nQUESTIONS:\n1.\tReferring to my comment above, please provide additional results using the approach detailed in Appendix A2 of the Hu et.al. 2023 NeurIPS paper, with varying penalty weights. \n2.\tGiven that also the approach in that paper uses an indicator function for infeasibility, could you please elaborate a bit more on the differences to your approach?\n3. How to choose the buffer parameter?\n\nLIMITATIONS:\nLimitations are addressed, with the exception mentioned above that some related work is not represprented and considered adequately.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-09"}