Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints
Authors: Jayanta Mandi
Marianne Defresne
Senne Berden
Tias Guns
Venue: NeurIPS 2025 poster ()
Keywords: decision-focused learning
predict-and-optimize
constrained optimization
Primary area: optimization
TL;DR: We propose a constraint-aware decision-focused learning framework  that enables parameter prediction in constraints, without making any assumptions about the underlying optimization problem.

Abstract: When some parameters of a constrained optimization problem (COP) are uncertain, this gives rise to a predict-then-optimize (PtO) problem, comprising two stages: the \textit{prediction} of the unknown parameters from contextual information and the subsequent \textit{optimization} using those predicted parameters. Decision-focused learning (DFL) implements the first stage by training a machine learning (ML) model to optimize the quality of the decisions made using the predicted parameters. When the predicted parameters occur in the constraints, they can lead to infeasible solutions. Therefore, it is important to simultaneously manage both feasibility and decision quality. We develop a DFL framework for predicting constraint parameters in a generic COP. While prior works typically assume that the underlying optimization problem is a linear program (LP) or integer LP (ILP), our approach makes no such assumption. We derive two novel loss functions based on maximum likelihood estimation (MLE): the first one penalizes infeasibility (by penalizing predicted parameters that lead to infeasible solutions), while the second one penalizes suboptimal decisions (by penalizing predicted parameters that make the true optimal solution infeasible). We introduce a single tunable parameter to form a weighted average of the two losses, allowing decision-makers to balance suboptimality and feasibility. We experimentally demonstrate that adjusting this parameter provides decision-makers control over this trade-off. Moreover, across several COP instances, we show that adjusting the tunable parameter allows a decision-maker to prioritize either suboptimality or feasibility, outperforming the performance of existing baselines in either objective.

Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints
Jayanta Mandi
Marianne Defresne"
Department of Computer Science
LAAS-CNRS, Université de Toulouse
KU Leuven, Leuven, Belgium jayanta. mandi
CNRS, INSA, Toulouse marianne.. bariiamne.efreeseell.e... defresne@laas.FFr
Sayyana.manniaa @wwleuven. be
Senne Berden
Tias Guns
Department of Computer Science KU Leuven, Leuven, Belgium senne. wwwdennkulleuuennnc. be
Department Computer Science KU Leuven, Leuven, Belgium tias. www.suunnuuuuuueee euven. be
Abstract
When some parameters of constrained optimization problem (COP) are uncer- tain, this gives rise to a predict-then-optimize (P1O) problem, comprising two stages: the prediction of the unknown parameters from contextual information and the subsequent optimization using those predicted parameters. Decision-focused learning (DFL) implements the first stage by training machine learning (ML) model to optimize the quality of the decisions made using the predicted param- eters. When the predicted parameters occur the constraints, they can lead infcasible solutions. Therefore, it is important to simultaneously manage both feasibility and decision quality. We develop a DFL framework for predicting con- straint parameters in generic COP. While prior works typically assume that the underlying optimization problem is linear program (LP) or integer LP (ILP), our approach makes no such assumption. We derive two novel loss functions based on maximum likelihood estimation (MLE): the first one penalizes infeasibility (by penalizing predicted parameters that lead to infeasible solutions). while the second one penalizes suboptimal decisions (by penalizing predicted parameters that make the true optimal solution infeasible). We introduce single tunable parameter form a weighted average of the two losses, allowing Decision-makees to balance suboptimality and feasibility.. We experimentally demonstrate that adjusting this parameter provides decision-makers control over this trade-off. Moreover, across several COP instances, decision-man mekisr to prioritize either suboptimality or feasibility, outperforming the performance of existing baselines in either objective.
show that adjusting the tunable parameter allows
Introduction
Many real-world optimization problems involve parameters that are unknown at decision time, such uncertain customer demand in manufacturing problems unknown traffic in delivery routing. This gives rise to predict-then then-optimize (PtO) problems (22). where a machine learning (ML. model first makes point predictions of the unknown parameters from contextual features, after which the resulting instantiated optimization problem is solved.
Affiliated KU Leuven during submission of this article
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In fredeeeion-focused learning (PFL), the predictive model is trained solely to maximize the accuracy of parameter predictions by minimizing standard ML losses. By doing so, PFL does not consider the effect of prediction errors on the solution to the optimization problem. For example, consider a vehicle routing problem, where one first needs to predict unknown customer demands, to then make routing decisions. In this setting, overestimations and underestimations can have differing effects. Overestimating demand may lead to a conservative suboptimal decision, while underestimating demand may lead to infeasible decision. However, these differing effects are not accounted for by PFL losses.
To address this shortcoming of PFL, focusid learning (DFL) directly trains the ML model to minimize a task spee--ee decision loss that reflects the quality of the solution produced by using the predicted parameters. With a slight abuse of notation, we will use the term 'predicted solution' to denote the solution produced by using the predicted parameters, although the MI. model does not directly predict the solution, it predicts parameters that are passed to an optimization solver at inference time.
Most existing DFL works 12 3 23 33 focus on cases where the predicted parameters appear only in the objective function. In such settings, the decision loss is typically the regret, which measures suboptimality of the predicted solution. Instead, we focus on predicting parameters that appear in the constraints of the optimization problem. In this setting too, suboptimal solutions may arise, because the feasible space defined by the predicted parameters may exclude the true optimal solution. However, focusing only on suboptimality insufficient. This is because the predicted solution may become infeasible with respect to the constraints instantiated by the true parameters. Therefore, the MI. model should also be trained to minimize the likelihood of such infeasible outcomes.
Focusing only on suboptimality or on infeasibility generally does not lead to a desirable balance between these two objectives. To address this, Hu et al. [14] model decision-making with uncertain constraint parameters as a two-stage process: first, solution is generated using the predicted parameters; then, if needed, corrective actions incurring additional costs are carried out to resolve potential infeasibility. Subsequent works [15 16 have extended this line of research.
One limitation of their approach is that the corrective action is often problem-specifice and restricted to specific form of optimization problem (e.g.. a linear program). Instead, we avoid the complexity of second stage altogether, and focus on predicting parameters that minimize both the infeasibility and the suboptimality of the first-stage solution itself. To this end, we first propose two novel loss functions: the Infeasibility Penalty Loss (IPL) and Optimality-Preserve
Loss (OPL). The idea behind IPL is that when solving with the predicted parameters leads to a solution that is infeasible according to the true parameters, such infeasibility should be (proportionally) penalized. On the other hand, the OPL loss will penalize predicted parameters that make the true optimal solution infeasible, to avoid obtaining a suboptimal solution when solving with the predicted parameters.
These two losses often contradict each other. The IPL term encourages conservative parameter predictions to guarantee fcasibility, even at the expense of producing extremely suboptimal solutions. In contrast, OPL. incentivizes loose parameter predictions to ensure the optimal solution is not cut away. Moreover, in practice, decision- maker may favor more feasibility over optimality vice versa. To support this subjective preference of one loss over the other, we propose to minimize a weighted average of the two losses. In this way, we introduce a tunable weighting mechanism that adjusts the total loss based on the decision-maker's preference. We refer to our approach as Odece, for "optimizing decisions through end-to-end constraint estimation".
We compare Odece with the PFL approach of minimizing the mean squared error (MSE), and with some existing DFL approaches. These approaches do not allow the decision- maker adjust the predicted solution based on their subjective preference between optimality and fcasibility. Moreover, the DFL approaches for constraint prediction are limited to linear programs (LPs) or integer or mixed-integer LPs (ILPs and MILPs), or they focus on predicting full constraint systems rather than individual parameters within the constraints. Our experimental results across various optimization problem instances show that, for a single value of the tunable parameter, Odece matches or out- performs existing baselines in terms of both suboptimality and feasibility. More importantly, they show that our approach offers a simple way for decision-makers to control the trade-off between suboptimality and infeasibility using a single parameter.
2

---

2 Related Literature
Differentiating through optimization problems. Differentiating through optimization problems has gained attention in ML, as it allows solvers to be embedded into gradient-based training. This is often done by ADMM 2 29 or by implicit differentiation [ of the optimality conditions. Implicit differentiation techniques have been developed for quadratic programs [2] and conic optimization [.. However, for combinatorial optimization problems like ILPs, the true gradient is zero almost every- where because small changes in the parameters usually do not affect the optimal solution. Hence, implicit differentiation techniques, which are applicable to smooth optimization problems, cannot be applied to such combinatorial problems. To address this, Paulus et al. (26) propose CombOptNet, technique for ILPs to compute aseudo-gradient based on the Euclidean distance between the solution and the constraints.
Predicting objective parameters. Most DFL works focus on predicting parameters in the objective function. Some of these works are based on differentiating through the problem after smoothing it, or perturbation [5] 10
either using regularization 13
18 3
25
28
Another approach is to define Regularization-bin -based methods
informative surrogates of the decision loss 4 12
19
20
23
31
require differentiable solvers, such as Cixpylayers
or OptNer 2 In contrast, perturbation and
surrogate- based approaches only need access to solutions. These solutions generally do not need to come from an exact solver and can be obtained by heuristics or alternative methods. For instance, surrogate-based techniques have been applied to planning problems solved via approximate automated planners 21
We refer readers to recent survey paper [2 for more comprehensive overview.
Predicting constraint parameters. Relatively few DFL papers have considered the prediction of parameters in the constraints. Most of these [14-16] model peeiiioon-making as two-stage process. In the first stage, a solution is obtained using the predicted parameters, which may be infeasible with respect to the true parameters. The second stage then applies a corrective action to convert the first-stage solution into a feasible one. Among these works, Hu et al. [15] also recommend a strategy for applying their upproach to settings where no corrective action is available at inference time. Their proposal is to assign a very high penalty factor to the corrective action during training. In contrast to this line of work, we do not introduce corrective actions even during training and focus on balancing decision quality and feasibility in single stage. Another approach [1] simultaneously learns the objective and the constraints, but the constraints are soft and the problem must be formulated as a discrete graphical model. The work by Nandwani et al. (24] aims to learn the full set of constraints from the features. They do this by learning set of hyperplanes that separate the optimal solution from other negative assignments. A key component of their technique is the generation of such negative assignments, which must include both infeasible as well as feasible but suboptimal ones. In contrast to their setting, we focus learning only the parameters within known constraint structure. Their method does not easily generalize to our setting, in which the learned parameters must remain compatible with the known constraints to make high-quality solutions. Moreover, both Hu et al. [15] and Nandwani et al. 24 restrict their methods to LPs and IIPs, while the framework proposed in this paper is more general and applicable to broader class of optimization problems. The prediction of constraints is also central in constraint acquisition, for optimization 3 or constraint satisfaction
6 32) problems, but the setting differs because the structure of the constraints is unknown and the uncertainty does not depend on any input features.
3 Preliminaries
Problem setup. We will consider a constrained optimization problem (COP) of the following form:
min f(z; æEn
(1a)
subject to (: p) S, i M)
(1b)
where defines the domain of the decision variable vector æ. Ω may be a subset of real numbers
or integers, depending on the type of optimization problem. The upper and lower bounds for each
decision variable are also specified by Ω. We will assume that the constraint functions g:( p)
are differentiable functions. We do not explicitly express equality constraints, since any constraint
3

---

of the form h [x: = 0 can be represented by two inequality constraints: h p) S 0 and
-zy p) - 0. To make the notation simpler. we denote the set of constraints by and the feasible set when the constraint parameters take the value p by F(p). More formally,
F(p) = {2 E Ω 9: (2; p) - O, E)
(2)
We use x² (q. p) to denote a (possibly non- non-unique) optimal solution the optimization problem for a given objective parameter q and constraint parameter p. When multiple optimal solutions exist, we define the set of all optimal solutions as W (q. p) = (a: [(x: q) S f(x': q); Va' E F(p)). Note that W (g. P) is a subset of the feasible region F(p). So, if F empty (i.e., no feasible solution exists), W also empty. In the special case of a unique optimal solution, W (q, P) is singleton. However, our implementation relies on solver that returns single solution. When multiple optimal solutions exist, the commercial solvers we use break ties using a predefined rule.
Example. Consider the Iulti-dimeesioonl 0-1 knapsack problem (MDKP) [27 of this problem to choose a subset with maximal value from given set of items, subject to M capacity constraints. Let p. be the capacity in dimension i. There are N items, the value of each item is gn. and P(n,s) is the weight of item n in dimension i. Each decision variable Vn can be either zero or one, hence Ω = 10, 1 This optimization problem can be modeled as an ILP as follows:
The objective
N
N
min
(-qn) such that In E (0, Vn [N],
Σ ==1
Oni S Vi [M]
(3)
Here, S consists of the M capacity constraints.
Predict-then-Optinie problems. In the PtO formulation for uncertainty in the constraints, some or all components of the parameter vector are unknown and must be estimated before solving the COP. For example, in the knapsack problem, the item weights P(n.a) may be unknown, but the capacity in each dimension P might be known. Additionally, we observe a set of values correlated with p. which we refer as features and denote by Φ. These features are used to make estimates of the unknown parameters. For the estimation task, a predictive model Ma, parameterized by 0, trained using training data of the form D {(@xx: Px: 9r. x' (9x. }r))]==> Formally, the goal is to learn a mapping from the feature space to the parameter space. In this work, we assume that the cost vector qx is instance-specifie but known and not used to learn the mapping. Note that the structure of the COP is fixed, i.c., the number of constraints and the domains of decision variables are known. Once trained, given a new feature vector & and known cost vector q, the model produces an estimated parameter vector P Mo(Φ) using Φb Since this is PtO task, Mo is ultimately evaluated based on the quality of the solution x' (g. P) induced by the predicted parameters.
A maximum likelihood perspective on DFL. We will develop framework based on maximum likelihood estimation (MLE) to predict constraint parameters. Our goal is to maximize the log- likelihood of the observed data D. In PFL., the parameters 0 of the predictive model would be estimated to increase the likelihood of the observations Pv: given the features Φ.. However, in DFI. training, the focus is on the likelihood of x (gxx Px) being the optimal solution, given q. and Φx- To put it another way, the MLE perspective of DFL consists of learning a 0 to predict the constraint parameter vector Px so that x= (gr, Px) is the most likely solution to the parameter Pr. Mo((...
4 Methodology
We start by defining a function UNSAT, (x. pxx) to indicate whether the i-th constraint violated by an assignment when the constraint parameter is set to the vector Px- It is defined as:
UNSAT P.)) = (1,
0}
Similarly, we define SAT, Px) to indicate that the i-th constraint is satisfied. Therefore, SATx(x. P.)) := UNSAT, ( p.)). Using these definitions, we introduce Feas(æ, Pa)), an indi- cator variable equal to if the solution x satisfies all the constraints.
Feas (x, Pw) II
SAT, (AT,(O, (æ, Px)

---

We use (x. to denote that æ is infeasible with respect to P.. Note that æ is an infeasible assignment if violates any one of the constraints. Hence, we can write:
Infeas( Px) = max UNSAT, (x.A,..(.Pp) xES
(5)
As we want to maximize the likelihood of obtaining feasible solution with optimal ob- jective value, we first model the probability of violating the constraints using the predicted parameters. Given predicted parameters Ma(Øx), we model the parametric probability Po UNSAT, (x, (x, Mo(ox)) Mo((x)) in the following manner:
Po UNSAT ( Me( Mo((..) 1+ exp(-g((( Mo(ow)))
(6)
Note that the above is a smooth version of the following non-smooth distribution:
f1, if go 2 0
Pe - (0,
(7)
otherwise
As Po (SAT, (x, (o. = Po (UNSATI (C we can write:
(Mo(o.)) = 1+ (xppo(((,
1
(8)
4.1 Loss Function to Penalize Infeasibillty-Inddue Predictions
We first propose a loss function that ensures that true infeasible assignments remain infeasible under the predicted parameters. Let Fag be an assignment that infeasible under the true parameters. As it is easy to determine which constraints are violated by Eneg, we develop an MLE framework to maximize the likelihood of violating those constraints under the predicted parameters. Given a Fnegs the log-likelihood of violating these constraints can be written in the following form:
Σ UnSAT; (Inegs Pxx) log Po (USST,, M Mo(Φx)) SES
Note that UNSAT; ner Pa)) can be viewed mask that equals when Fieeg violates constraint i under the true parameters Px. Using the definition of Pa (USAT, (æ, Mo((o.)))Mo(oo) from Eq. we can write the following:
log Po (UNSAT. Maleg Mo((.)) = log (1 + exp (-p (Xneg, (dx)))) (10)
We now propose loss function to maximize this likelihood (i.e., minimize the negative log- likelihood). We include a margin parameter T > to encourage the constraints to be violated with buffer. Since this loss penalizes cases where infeasible assignment &ney is considered feasible under the predicted parameters, we refer to it as the Infeasibility- Loss (IAL).
C IAT (wney Ma(px)) = Σ UNSATr (Eneg Pw)) softplus (T - ge ge(Wnngg Mo(px)))
(11)
iES
where we write log(1 + exp(-)) as softplus(.).
Intuition behind IAL. Note that softplus is a smooth approximation to the Relu function. If we replace the softplus function with Relu function, Eq. I1 takes the following form:
Σ UNSAT,(I Pxx) max (0, Y 9: Mo( SES
(12)
From this expression, we that for a truly violated constraint, i.e., of which UNSAT, ( (æney Px)
the loss becomes zero only if g, Ma > Y. In other words, the predicted constraint
must also be violated. LIAL ()(Φ..) is zero if this holds for all constraints violated by &neg-
5

---

Selecting neg. The IAL loss formulation is based on 2 Eney, candidate assignment which violates least one constraint with respect to the true parameter Pa. This leads the question of how to obtain such Fneg. We adopt the following approach: for each instance, after predicting Px.. we solve the COP using P.x and obtain solution x x' (qx, Pxx). this solution is infeasible under the true parameters Px.. we use it as neg. Otherwise, we set the loss to zero, since the predicted solution satisfies all the true constraints, which is the preferred outcome. We call this final loss function the Infeasibility Penalty Loss (IPL), as it penalizes the predicted parameters when the resulting predicted solution violates the true constraints.
C (pis Prs) (1 Feas (x* (qx: Pre), Px)) (a" Pc):
(13)
We can also define L where LIAL in Eq. 13 replaced with CIAL.
Lemma 1. For true parameter PN and a predicted parameter Pu. CIPL (prs Px) is zero if and only if xt (qxs Px), an optimal solution for Px. is feasible with respect to Pi.
Proof. LTP, is by construction zero when x² (n)) feasible with respect to Px- Because x' (x) is feasible with respect to Px. Feus (x* (Axr Pa), Pn.) is equal to one, which makes
L 0. Hence,
Feas (w" (gex Pa) Pk) =1 LIPL
To prove the other direction, let us assume, LIPL = O. but Feas (x' (gx-Pr), ;Pr) + 1. Hence, CIAL(x (gis px), Px) must be zero. Note that, CIAL( (4k Pm), Pu) can be zero only if gi(x" (qxs P=): Pk)) 2 > for all the violated constraints. However, as x" (gxx px) is the optimal solution with P... being the parameter, y:("" (Ans Px), Pxx must be less than zero for all the constraints and we arrived at contradiction. So,
LIP = 0 - Feas (x* (4x, Px), Pxx =1
4.2 Loss Function for Preserving Optimal Solutions
The IPL loss penalizes predicted parameters when the predicted solution becomes infeasible with respect to the true parameters. However, minimizing only L can lead to learning a feasible region that is much stricter than the true feasible region, any solution therein will be feasible under the true parameters. For example, when learning the capacity parameter in a knapsack problem, minimizing only CIPL might result in capacity values small that no items are selected. While such solutions are always feasible with respect to the true parameters, they are suboptimal and of no practical value. Therefore, we also need loss function to penalize cases where the learned parameters turn out to be too strict. To do this, we define a loss function that penalizes predicted parameters when they render known optimal solutions infeasible. Within the MLE framework, this corresponds maximizing the likelihood that the true solution is feasible.
Thus, formulate the probability of an assignment x being feasible point, given the parameter Mo( Note that æ is a feasible assignment only satisfies all the constraints. While the probabilities of satisfying constraints and j are not independent, they are conditionally independent 7 Chapter 8] given the constraint parameter vector M.( This is because the constraint parameter vector determines the feasibility of each constraint independently. Once M)((x)) known, the probability of satisfying constraint depends only on Me( (..)). and any additional information about constraint being satisfied does not affect that probability. Because of the conditional independence, we can express the conditional probability of the assignment, x being feasible given Mo(x)), by:
Po Feas (z, Mo(cw)) Me(
II Pe (SAT, (oo iE
(14)
we can model Pg (Feas (x, Mo Now using Eq.
Po (Feas (x, Mo(cw) Mo(ew)
II
1
(15)
I (x, Mo(
iES

---

Thus, we obtain the log- log-likelihood function of a being feasible, in the form of:
log (1 exp (ar (x, (o(H))))))) - Σ Σ
softplus (gi(0 Mo(G.))
(16)
iES
VES
We point out that in the training data, we know that x (gro Pxx) is feasible solution. Denoting æ; as the known feasible point, we can formulate loss function by minimizing the negative log-likelihood. In our implementation, we add margin parameter Y as we have done for L C Since this loss is designed to maximize the probability that the optimal solution remains feasible, we refer as the Optimality-Pee Optinality-Preeerving Loss (OPL). We express it in the following form:
C Pu, Mo(px)) =
Σ
softplus (T + 96 (x" (qxs Pw) Mo(
(17)
IES
Intuition behind OPL. By replacing the softplus in Eq. 17 with a Relu function, we can write
COPL Pra)
Σ max ((r Dr (a* (gx, Pie) pxx)),
(18)
where we write ()xx)) as P. for brevity. From this expression, it is easy to see that COPL attains its minimum value of when all constraint values are less than or equal -T, i... when all the constraints are satisfied with some margin.
Lemma 2. For
parameter Ps. and predicted parameter Pno. both LOPE (pxx Px) and CIPL (Pe; Pa) will be zero if and only if x² (qe:Pa), a solution of Pic. is an optimal solution with respect to Pre.
Proof. Let [OFE(px, (x) and C (px, Px)) be equal to zero. We have proved in Lemma that CIPL being zero implies that solution of 6x is feasible with respect to k. So, xx (gus 2x), solution of Åx. lies in the set F(px).
Because x= (qn. Px) is optimal solution,
f(x" (gn, Px)); qm) I f(a'; Val E F(pn)
As x (qx, Pm) E F(px), from the above argument, we can write
f(x" (gx, pw)): qe) S (ax (gis Px)) :p:)
(19)
Now, COPL(px. Pr)) being zero implies that x:' (qx: Pic), an optimal solution of the true parameter, Px. lies in F F(px). Note,
(yn, (w) S f(x'; Val E F(px)
As x: (g Pa)) E F(p,)
f(x" (qxs pw): S f(x" (axs px):
(20)
Hence, Eq. 19 and Eq. suggests
fax Pw):: qn) = f(x" (gx. (x):
As (qx. P.)) is feasible solution with respect to Ps and its objective value is equal to the optimal objective value, is an optimal solution.
Now, to prove the opposite direction, assume that x² (gx: Pic) is an optimal solution with respect to Px- This implies that x=* (gx: Pkx) is feasible solution with respect Px- Therefore, from Lemma CIPT (pi. (x.) 0. As xx (q Pa)) adheres all the constraints when P.. is the constraints parameter value,
Σ max (T + gi( (gx, Px) Pa)) ba))).)) =0
(21)
As x² (qr: Pax) is an optimal solution with respect to Pic, x* (grs Pk) is one of the x* (g...w). If
replace x' (gx, Px) with x" (gr, Px) in Eq, 21 COPL Pkx. P.)) 0 by definition (Eq. 18

---

4.3 Minimizing a Convex Combination of LOP,. and L
Lemma 2 suggests that by minimizing both L and C to zero, we can obtain a solution a'(ge Px),, which is optimal with respect to Psc. However, simultaneously minimizing _ and L presents a challenge. This is because _ and L represent two conflicting goals. Mini- mizing only LOPL encourages the model to predict loose constraints so that all true feasible solutions remain feasible, hence making even infeasible solutions appear feasible. Conversely, minimizing _ encourages stricter constraints, which may render even true optimal solutions infeasible. We thus propose to minimize a convex combination of these losses, in the following form:
a. I.C.PL (p.. + - a)) (pss PRcSPr)
(22)
E [0, 1] is a hyperparameter we call the infeasibilit---versiio coefficient. It reflects how much importance the decision-makee places on avoiding infeasible solutions (through IPL) versus maintain- ing the feasibility of known feasible ones (through OPL). The two extremes are = 1 and a = 0, prioritizes only
both of which represent unrealistic preferences. a means the feasibility, possibly ending up with trivial solutions (e.g., picking no items in a knapsack problem). near implies a focus on high-reward solutions, even if they are infeasible (e.g., selecting all items).
In this section, we have proposed a novel approach for 'optimizing decisions through end-to-end constraint estimation' (Odece), involving two novel loss functions: _ and L C We have proposed to consider a weighted average of the two losses to reflect the decision-maker subjective preference of infeasibility over suboptimality. Next, will experimentally investigate how Odece performs compared to other existing approaches for predicting constraint parameters.
5 Experimental Evaluation
We experiment with predicting constraint parameters in the following optimization problems:
Multi-dimeenional knapsack problem. The first is the 0-1 multi-ddmmnsional knapsack problem (MDKP), introduced in Section 3 In the experimental section, we consider MDKP instances with 50 items and constraints in three dimensions. We consider two settings: 1) predicting the item weight vectors while keeping the capacity constraints known, and 2) predicting the capacity vectors with known item weights.
Brass Alloy Production. Our third experimental setting involves the Brass alloy production problem. We reproduce this experimental setup from the work by Hu et al. [15]. It is a covering LP problem. We use the publicly available Brass alloy data from their repository The production of the Brass alloy requires two metals: Copper (Cu) and Zinc (Zn). A factory needs to purchase ores that contain these two metals. There are 10 potential suppliers, each offering ores at a different price per unit. The exact metal content in each supplier's 's ore is unknown and must be predicted. The goal is to buy a combination of ores from the 10 suppliers at the lowest possible cost, while ensuring that the total purchase contains at least 627.54 units of Cu and 369.72 units of Zn.
5.1 Dataset Description
Synthetic parameter generation of the MDKP. We adopt the synthetic data generation process described by Elmachtoub and Grigas [12]. This data generation approach introduces non-linear relationship between input features and the unknown parameters. A linear model is used to predict the parameters from features. The motivation behind using linear model is to demonstrate that DFL methods can produce high-quality solutions despite model misspecification. This approach has been
widely used as a benchmark in DFL studies [12 20
30 31].
Each knapsack instance contains 50 items, with all items correlated with 10 input features. Item values, which appear in the objective function, are sampled independently from Gumbel distribution with location 100 and scale 20. For each run, 1500 instances are generated, split into 900 for training, 100 for validation, and 500 for testing. We took extra caution to ensure that the synthetically-genraaa parameters produce non- non-trive trivial optimal solutions, avoiding cases where all or none of the items are

---

Predicting Weights MDKP
Predicting Capacities in MDKP
Predicting Quantities Brass Alley
0.32
648
Test Regret
0.4
Regret.
623
2.04
6:2-
6.3
020
12
64 Test Infeasibility
25
Co
02
64
64
20
65
Test infeasibility
Test Infeasibility
MSE
SFL 2sPtO
Odece (0.2)
Odece (0.4)
Odece (0.6)
Odece (0.8)
CombOptNet
Odece (0.3)
Odece (0.5)
Odece (0.7)
Figure 1: Infeasibility ratio and regret of feasible solutions Test instances (best viewed in colors).
selected in the knapsack. To achieve this, we clip the capacity in each dimension to be less than half the total weight of all items and ensure that individual item weights remain below the respective capacity values.
the dataset from Hu et al. [15. In this dataset, the Cu and Zn contents in each supplier's ore are predicted using separate sets of 4096 features each. Out of the available 500 instances, use 350, 50, and 100 for training, validation, and testing, respectively. We use a fully-connected neural network with one hidden layer of 512 neurons for prediction.
Brass Alloy Production. For this experiment,
The experiments were executed on an Intel 17-13800H (20 cores) CPU with 32GB RAM. Details on the problems and datasets used in the three experiments are provided in Appendix A The source code for reproducing our results is publicly available at: /uttps://githbb com/ JayMan91/ ReccolFfforroossrriitttselrr
5.2 Experimental Results
We denote the model as Odece(a)) (e.g., Odece(0.1)) to indicate Odece is trained with that specific value of We compare the proposed Odece against four competitors: MSE: PFL approach which trains the MI. model to minimize the MSE loss over the parameter predictions, ii) CombOptNet: the technique proposed by Paulus et al. (26] to compute the gradient of II.P parameters, iii) SFL: the solver-free learning proposed by Nandwani et al. 22 for learning parameters of an IIP, iv) 2sP1O: the two-stage predict+optimize approach proposed by Hu al. (15). We implement 2sPtO using a high penalty factor, as recommended in Appendix A.2 of their paper. We could not apply 2sPtO to predict the knapsack capacity, as it is designed to predict only the left-hand side parameters in linear constraints. More specifically. it can compute gradients with respect to in constraints of the form TTa x S c, but not with respect to c. For CombOptNet we compute the L1 loss of the predicted solution and the true solution and backpropagate the L1 loss through CombOptNet.
We evaluate performance using two metrics: the proportion of infeasible solutions and the normalized regret on the test data. Each technique is run five times with different random seeds. Figure rreport the average proportion of infeasible solutions (X-axis) and average normalized regret (y-axis) across these runs. Detailed results, including averages and standard deviations across the five runs, are provided in Appendix B We compute regret only over predicted solutions that are feasible under the true parameters. More Tormally, out of K test instances, let K' denote the number of instances where K
the solution xx (gi,, P.x) is feasible under Pxc- The proportion of infeasible solutions is then K The
normalized regret is defined follows:
K' f (x* (qe. : f(x; qx)
(23)
K
f( qn.)
Note that normalized regret can misleading in this setting. This is because extreme cases, where most predicted solutions are infeasible, regret is computed over only a small subset of instances.

---

Results. The first observation from Figure is that CombOptNet exhibits relatively poor perfor- mance across the three problems. For both MDKP weight prediction and the Brass Alloy problem. the proportion of infeasible solutions generated by CombOptNet is very high (> 85%). In MDKP capacity prediction, the proportion of infeasible solutions is slightly lower but still high (> 65%). The performance of SFL is similar to CombOptNet for MDKP weight prediction and the Brass Alloy problem. However, in the MDKP capacity prediction problem, SFL performs significantly better, with the proportion of infeasible solutions around 25%. We suspect this is because SFL relies more on è than 6 to separate the true optimal solution from the negative assignments, resulting in more informative gradients for and less informative gradients for b. (For example, in a 3-dimensional MDKP where the optimal solution is [0, 0. 1]. negative samples such as [1. 1, [], 1, 0j, [0. 1, 1] and [1, 0, 1] can be classified as negative based on the capacity, i.e., the right- side parameter, c.)
For MDKP weight prediction, 2sPtO performs very poorly (the proportion of infeasible solutions is greater than 99%). It is not applicable for MDKP capacity prediction. Figure 1 might suggest that 2sPtO achieves relatively lower proportion of infeasible solutions for the Brass Alloy problem, but this is misleading for two reasons. First, it results in extremely high normalized regret (> 850%). Note that we have to break the y-axis in Figure [ to accommodate 2sPtO's 's regret values. More importantly, its performance is highly unstable, which is not noticeable in Figure I where the average of five runs is plotted. Table 3 in the Appendix reveals that the standard deviation across the five runs is very high. In some runs, the proportion of infeasible solutions is near O, while in others it reaches se 99%. This indicates unstable learning.
Our closest competitor is MSE, but minimizing MSE does not let decision-maker adjust the trade-off between optimality and feasibility. MSE has infeasibility rates of 33%, 60%, and 53% across the three problems, respectively. For eecisionnmaker these infeasibility rates may be unacceptably high. Odece allows a decision-maker to attain lower infeasibility rate by varying a, at the cost of higher normalized regret. For instance, Odece(0.8) brings the infeasibility rates down to 6%, 4%, and 18%, across the three problems, respectively. We have also conducted statistical significance tests to determine whether the test regret and infeasibility of Odece are lower than those of MSE. Since each run was conducted using the same random seed for both models, we used a paired t-test with the alternative hypothesis that MSE has higher regret and higher infeasibility than Odece. The p-values from the paired t-tests for different values of a are reported in Table S and 6 in Appendix B The results of the statistical significance test reveal that for > 0.5, Odece produces statistically significantly lower proportion of infeasible solutions than MSE. On the other hand, for a < 0.3, the normalized regret on feasible instances generated by Odece is statistically significantly lower than that of MSE. This holds true across all three problems.
Conclusion
We proposed Odece, a novel DFL technique for predicting parameters appearing the constraints of COPs. Unlike existing DFL methods for constraint predictions - which either depend on problem- specific second-stage corrective actions or which are designed to predict the full set of constraints Odece directly optimizes for feasibility and solution quality in single stage. Our experiments showed that by tuning Odece's 's a parameter, infeasibility and suboptimality, allowing them to align solutions with their individual preferences.
makers can flexibly manage the trade-off between
One limitation of the current Odece implementation is that it solves the COP for each instance using the predicted parameters during training. To speed up training, future work could explore caching candidate assignments, done by Mulamba al. [23) for predicting objective parameters. Odece could also be extended to jointly predict both constraint and objective parameters. Finally, we plan to apply Odece to a wider range of optimization problems, such as combinatorial problems with non-linear objective and constraint functions, as well as larger, real-world applications.
Acknowledgments and Disclosure of Funding
This research received funding from the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Programme (Grant No. 101002802, CHAT-Opt), and from the Research Foundation Flanders (FWO) project G0G3220N. Senne Berden is fellow of the Research Foundation- Foundation-Flanders
IPQQ24NN)
10

---

References
[1] A. Agrawal, B. Amos, S. Barratt, S. Boyd, Diamond, and Z. Kolter. Differentiable convex optimization layers. Advances in neural information processing systems, 32, 2019.
[2] B. Amos and J. Z. Kolter. Optnet: Differentiable optimization layer in neural networks. In International Conference on Machine Learning, pages 136-145. PMLR, 2017.
[3] S. Berden, M. Kumar, S. Kolb, and T. Guns. Learning MAX-SAT models from examples using genetic algorithms and knowledge compilation. In 28th International Conference on Principles and Practice of Constraint Programming (CP 2022), pages 8-1. Schloss Dagstuhl-Leibin Zentrum für Informatik, 2022.
[4] S. Berden, A. i. Mahmutogullan, D. Tsouros, and T. Guns. Solver-free focisedn learning for linear optimization problems. arXiv preprint S.ki::5505,22222 2025.
[5] Q. Berthet, M. Blondel, O. Teboul, M. Cuturi, J.-P. Vert, and F. Bach. Learning with differen- tiable pertubed optimizers. Advances in neural information processing systems, 33:9508 9519, 2020.
[6] C. Bessiere, C. Carbonnel, and A. Himeur. Learning constraint networks over unknown con- straint languages. In IJCAI 2023-32nd International Joint Conference Artificial Intelligence, pages 1876-1883, International Joint Conferences on Artificial Intelligence Organization, 2023.
[7] C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
[8] M. Blondel. Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares-Lopez. Pedregosa, and J.-P. Vert. Efficient and modular implicit differentiation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 5230-5242. 5242. Curran Associates, Inc., 2022.
[9] A. Butler and R. H. Kwon. Efficient differentiable quadratic programming layers: ADMM approach. Comput. Optim. Appl., 84(2):449-47 2023.
[10] G. Dalle, L. Baty, L. Bouvier, and A. Parmentier. Learning with combinatorial optimization layers: probabilistic approach, 2022. URI hwt../tttsss//arrv.orr///aa////
13513
[11] M. Defresne, Barbe, and Schiex. Scalable Coupling of Deep Learning with Logical Rea- soning. In Thirty-second International Joint Conference on Artificial Intelligence, VICAI'2023: 2023.
[12] A. N. Elmachtoub and P. Grigas. Smart "predict, then optimize". Management Science, 68(1): 26, 2022.
[13] A. Ferber, B. Wilder, B. Dilkina, and M. Tambe. Mipaal: Mixed integer program layer. Proceedings of the AAAI Conference on Artificial Intelligence, 34(02):1504-1511, Apr. 2020.
[14] X. Hu, C. Lee, and J. H. Lee. Predict+ optimize for packing and covering lps with unknown parameters in constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3987-3995, 2023.
[15] X. Hu, C. Lee, and J. H. Lee. Two-stage predict+optimize for MILPs with unknown parameters in constraints, In seirth- Conference on Neural Information Processing Systems, 2023. URL
.nt. www.uniiddttnnppppppp
[16] X. Hu, C. H. Lee, and J. H. man Lee. Branch & learn with post-hoc correction for pre- dict+optimize with unknown parameters in constraints. In Integration of Al and OR Techniques in Constraint Programming. 2023.
[17] M. Kumar, S. Kolb, S. Teso, and L. De Raedt. Learning max-sat from contextual examples for combinatorial optimisation. Artificial Intelligence, 314:103794, 2023. ISSN 0004-3702. doi: http://ddoi.ogg///o.oooii.mmmmt2 ociaaoor/rriicee///////000022220
URL htt.:httss://// bteepp::/ww.ssiieneelreee. www.weeeieennediecc........ com/

---

[18] J. Mandi and T. Guns. Interior point solving for Ip-based prediction-opiimistton.. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neu- ral Information Processing Systems, volume 33, pages 7272-7282, 2020.
[19] J. Mandi, E. Demirović, P. Stuckey, and T. Guns. Smart meedictannd-opimizzz for hard combi- natorial optimization problems. Proceedings of the AAAI Conference on Artificial Intelligence, 34(02):1603-100 1610, Apr. 2020.
[20] J. Mandi, V. Bucarey, M. M. K. Tchomba, and T. Guns. Decision-focused learning: Through the lens of learning rank. In International conference on machine learning, pages 14935-14947. PMLR, 2022.
[21] J. Mandi, M. Foschini, D. Höller, S. Thiébaux, J. Hoffmaan,, and T. Guns. focusedn learning to predict action costs for planning. In Proceedings of the European Conference on Artificial Intelligence (ECAI-24), pages 4060-4067. IOS Press, 2024.
[22] J. Mandi, J. Kotary, S. Berden, M. Mulamba, V. Bucarey, T. Guns, and F. Fioretto. Decision- focused learning: Foundations, state of the art, benchmark and future opportunities. Journal of Artificial Intelligence Research, 80:1623-1701, 2024.
[23] M. Mulamba, Mandi, M. Diligenti, M. Lombardi, V. Bucarey, and Guns. Contrastive losses and solution caching for predict-and-optimizz. In Z.-H. Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCA1-21, pages 2833- 2840. International Joint Conferences on Artificial Intelligence Organization, 2021. doi: 10. 10.24963/ijcai 22249663/caii2222133999 2021/390. Main Track.
[24] Y. Nandwani, R. Ranjan, Mausam, and P. Singla. A solver-free framework for scalable learning in neural ILP architectures. In S. Koyejo, Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
[25] M. Niepert, P. Minervini, and L. Franceschi. Implicit mle: Backpropagating through discrete exponential family distributions. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 14567-14579. Curran Associates, Inc., 2021.
[26] A. Paulus, M. Rolinek, V. Musil, B. Amos, and G. Martius. Comboptnet: Fit the right np-hard problem by learning integer programming constraints. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8443-8453. PMLR, 18-24 Jul 2021.
[27] D. Pisinger. Where are the hard knapsack problems? Computers & Operations Research, 32(9): 2271-2284, 2005.
[28] M. V. Pogančić, A. Paulus, V. Musil, G. Martius, and M. Rolinek. Differentiation of blackbox combinatorial solvers. In International Conference on Learning Representations, 2020.
[29] H. Sun, Y. Shi, J. Wang, H. D. Tuan, H. V. Poor, and D. Tao. Alternating differentiation for optimization layers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
[30] B. Tang and E. B. Khalil. Pyepo: A pytorch- based end-to-end predict-then-optimie library for linear and integer programming, 2023.
[31] B. Tang and F. B. Khalil. Cave:
A cone-aligned approach for fast predict-then-optimiz with binary linear programs. B. Dilkina, editor, Integration of Constraint Programming, Artificial Intelligence, and Operations Research 21st International Conference, CPAIOR 2024, Uppsala, Sweden, May 28-31, 2024, Proceedings, Part II, volume 14743 of Lecture Notes in Computer Science, pages 193-210. Springer, 2024.
[32] D. Tsouros, S. Berden, and T. Guns. Learning to learn in interactive constraint acquisition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8154-8162, 2024.
12

---

[33] B. Wilder, B. Dilkina, and M. Tambe. Melding the data data-decisions pipeline: Decision-focused learning for combinatorial optimization. In The Thirty-Thin Third AAAI Conference on Artificial Intelligence, AAA/ 2019, pages 1658-1665. AAAI Press, 2019.
13

---

A Problem and Dataset Description
A.1
Multi-dimensional knapsack problem
The formulation of the optimization problem is as follows:
N
mHin
(-qn)
such that In E (0, Vn E [N], Pus Pi, Vi [M]
(24)
Here, On and Pas are the values and weights in dimension of item n. We synthetically generate 1500 MDKP instances for each run. Of these, 900 are used for training, 100 for validation, and 500 for testing. We represent the dataset as {(h,. Pic, Qx, x= (q px))] where Φx is the feature vector, Ps is the concatenated vector of weights and capacity, dr is the objective parameter, and x' agxe Pix) denotes the solution to instance k. In our experiment, Φx is of dimension 10. The feature vectors are sampled from multivariate Gaussian distribution with zero mean and unit variance, i.e.
ΦR
N(, I1o).
A.1.1
Data Generation for Unknown Weight Vectors
We generate our dataset similarly to Elmachtoub and Grigas [12] and Tang and Khalil [30]. While these works consider predicting parameters in the objective function, we focus on predicting param- eters in the constraints. We use the same data generation process synthetically create constraint parameters from features.
To generate the weight vector, first matrix B E IR MxNII0 is generated, which represents the true underlying model, unknown to the decision- Each entry the matrix B is sampled from (e)) independent Bernoulli distributions of probability 0.5. The weight Pni for s-th instance, is then generated according to the following formula:
[c) Pns
1 VIO
Bli +3
(Ø B[i,
Deg
+1
(n) Snd
(25)
3.5
The Deg is 'model misspecification parameter. This is because a linear model is used as a predictive model in the experiment and a higher value of Deg indicates the predictive model deviates more from the true underlying model and larger the prediction emros.. c(x) is a multiplicative noise term sampled randomly from the uniform distribution e(()) U[L w. 1 w). We report results for Deg 6 and w = 0.25.
We generated the capacity p; (rc) the the following ways:
N 10
r
Σ
Bii n,
(26)
The term inside the summation represents upper bound on the total weight along dimension i Therefore, setting the capacity vector equal to this summation would always allow all items to be selected in the true solution, resulting in trivial case. To avoid this, we add r [0, 1]. It tightens the capacity constraints, thereby preventing trivial solutions where all items are feasibly selected. In
experiments, set to 0.2. Due the multiplicative noise term (K) U w, 1 w). the capacity vectors for each instance are not exactly the same.
A.1.2 Data Generation for Unknown Capacity Vectors
We again follow an approach similar to Tang and Khalil (30] to construct this dataset. To generate the capacity vector, we first create a matrix, B E M generated, which represents the true underlying model. Each entry of the matrix B is sampled from independent Bernoulli distributions of probability 0.5 like before. The i-th dimensional capacity pi for w-th instance, then generated according to the following formula:
Deg
P
(0) b[]
+3
+1
(1))
(27)
3.5Deg
10
14

---

Table 1: Test Regret and Infeasibility of the Models on MDKP Weight Prediction.
Infeasibility Avg
Regret
Sd
Avg
Model
MSE
0.335
0.015
0.063
0.003
CombOptNet
0.996
0.003
0.000
0.000
SFI.
0.992
0.004
0.010
0.013
2sPtO
0.996
0.003
0.000
0.000
Odece(0.2)
0.952
0.020
0.017
0.002
Odece(0.33
0.845
0.019
0.024
0,002
Odece(0.4)
0,570
0.050
0.039
0.002
Odece(0.5)
0.134
0.023
0.081
0.004
Odece(0.6)
0.079
0.027
0.110
0.009
Odece(0.7)
0.066
0.021
0.126
0,011
Odece(0.8)
0.065
0.025
0.127
0.007
Table 2: Test Regret and Infeasibility of the Models on MDKP Capacity Prediction.
Infeasibility Avg
Regret Avg
Sd
Sd
Model
MSE
0.588
0.024
0.171
0.010
CombOptNet
0.647
0.092
0.202
0.024
SFI.
0.251
0.124
0.269
0.044
Odece(0.2)
0.791
0.077
0.123
0.022
Odece(0.3)
0.572
0.269
0.175
0.086
Odece(0.4)
0.153
0.184
0.330
0.097
Odece(0.5)
0.053
0.020
0.392
0,039
Odece(0.6)
0.049
0.024
0.412
0.058
Odece(0.7)
0.055
0.020
0.417
0.042
Odece(0.8)
0.038
0.008
0.428
0.042
However, directly using this capacity value would result in capacitics that are too low for most instances, leading solutions where items selected. To obviate such scenarios, scale the capacity vector by a factor r = 0.5 * N, where N is the number items. Further, to preserve the dependency between the feature vector and the capacity vector, multiply the feature vector by the same scaling factor r.
A.2 Brass alloy production problem
We reproduce this experimental setup from the work by Hu et al. [5). It actually a covering LP problem. The production of the Brass alloy requires two metals Copper (Cu) and Zinc (Zn). A factory needs to purchase ores that contain these two metals. These metals must be sourced from multiple suppliers, each offering metal ores at different prices per unit. The factory's 's goal to purchase combination of ores from these suppliers that minimizes the total cost while ensuring that the total quantity of each metal meets the production requirements. The optimization problem can be formally written as:
min
Jn
such that 2 0
Vn [N],
Σ Puitn 2 Pi. Vi E [M]
(28)
where the decision variable H, represents the amount ore to be purchased from supplier ni, and qr the cost per unit from that supplier. Pai denotes amount of metal i in one unit of ore from supplier
n. The constraints ensure that the total acquired amount of each metal across all suppliers sums up more than p.. the required quantity for production.
15

---

Table Test Regret and Infeasibility of the Models on Alloy Production Problem.
Infeasibility Avg
Regret Avg
Sd
Sd
Model
MSE
0.532
0.005
0.169
0.004
CombOptNet
0.896
0.233
0.181
0.002
SFI.
0.991
0.013
0.019
0.001
2sPtO
0,400
0.548
8.539
6.691
Odece(0.2)
0.790
0.064
0.081
0.033
Odece(0.33
0.729
0.078
0.123
0,025
Odece(0.4)
0.571
0.085
0.195
0.041
Odece(0.55
0.471
0.123
0.270
0.054
Odece(0.6)
0.324
0.042
0.365
0.045
Odece(0.7)
0.276
0.026
0.406
0.037
Odece(0.8)
0.174
0.029
0.486
0.037
In the Brass alloy problem instances, there are 10 customers and the factory needs least 627.54 units of Cu and 369.72 units of 7n. We consider the data for Brass alloy, which is publicly available the repository? + The dataset contains 500 instances. In each instance, there 4096-dimensional feature vector for predicting the values of each Pri. We use 350 instances for training, 50 for validation, and 100 for testing.
B Additional Experimental Results
In this section of the Appendix, we provide additional details on the experimental results that could be included in the main text due space limitations. Specifically, we report the average and standard deviation of test regret and infeasibility across five runs for each model. Table [] Table and Table present the average average and standard deviation of test regret and infeasibility across five runs for MDKP weight prediction, MDKP capacity prediction and Brass alloy production problem respectively.
p-value (infeasibility)
p-value (regret)
0.8
3 x 10-6
0,999
0.7
3 x 10-5
0.999
0.6
5 10-6
0,999
0.5
x 10-5
0,999
0.4
0.999
1.50 10-1
0.3
0.999
4 x 10-6
0.2
0.999
3 10-6
Table 4: P-values from paired t-tests comparing MSE and Odece for infeasibility and regret with different values of for MDKP weight predictions.
Test of statistical significance. For predicting capacity, we observe in Table Odece has signif- icantly lower infeasibility and higher regret till a = 0.4. We also observe the same for the alloy production problem in Table[6] We conducted statistical significance tests determine whether the test regret and infeasibility of Odece are lower than those of MSE. Since each run was conducted using the same random seed for both models, used paired t-test with the alternative hypothesis that MSE has higher regret and higher infeasibility than Odece. The p-values from the paired t-tests for different values of for MDKP weight predictions are reported in Table It tells that for 2 0.5, Odece has significantly lower infeasibility and higher regret than MSE. The opposite is true when goes below 0.4.
httep://ggthbbbbc//lllaaaattttap http:///sw...c.aaaaaaaaaoorrrtta Praaeo_Predictt-ptimmiee
16

---

p-value (infeasibility)
p-value (regret)
0.8
6 x 10-7
0.995
0.7
2 x 10-7
0.995
0.6
9 x 10-7
0.999
0.5
1 x 10-6
0,997
0.4
0.003
0.999
0.3
0,450
0,545
0.2
0.997
0.040
Table 5: P-values from paired t-tests comparing MSE and Odece for infeasibility and regret with different values of for MDKP capacity predictions.
p-value (infcasibility)
p-value (regret)
0.8
4 x 10-6
0,999
0.7
7 x 10-6
0,999
0.6
0.002
0.999
0.5
0.160
0.993
0.4
0.848
0.882
0.999
0.007
0.2
0.999
0.002
Table 6: P-values from paired I-tests comparing MSE and Odece for infeasibility and regret with different values of a for Alloy Production Problem.
17

---

C Details about Hyperparameter Configuration
Finally, in Table 7 we provide the hyperparameter configurations used for each model reproduce the results reported above.
Table 7: Hyperparameter configurations for each model and task.
Task
Hyperparameter
MSE
Odece
CombOptNet
SFL
2sPtO
Learning Rate
0.05
0.05
0.05
0.05
0.05
MDKP Weight Prediction
т (CombOptNet)
0.50
Temperature (SFL) damping (2sPtO) thr (2sPtO)
0.50
0.01
0.1
Learning Rate 7 (CombOptNett
0.005
0.005
0.005
0.01
MDKP Capacity Prediction
0.1
Temperature (SFL)
0.1
Learning Rate
0.001
0.001
0.005
0.001
0.05
Brass Alloy
T (CombOptNet)
0.5
Temperature (SFL) damping (2sPtO) thr (2sPtO)
0.5
0.01
0.1
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the main claims in the abstract and introduction accurately reflect the paper's contributions and scope. Yes, the main claims in the abstract and introduction accurately reflect the paper's contributions and scope. The paper motivated by the need to develop novel approach for predicting unknown parameters of an optimization problem using correlated features, while explicitly accounting for the subjective preferences of the decision maker regarding infeasibility and suboptimality. This motivation, along with the core contributions, is clearly stated in both the abstract and the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Yes, the paper provides brief summary of the limitations, which we aim to address future work.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
19

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Yes, we have provided proofs for the theorems proposed in the paper.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We are providing the code and data reproduce the main experimental results of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results.
20

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We will make the data and code publicly accessible upon acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
Codeeubmii ssiinn criionPooiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we have described the training, test, and validation split, as well as the ap- proach used for hyperparameter selection. For detailed information the hyperparameters, we refer the reader to the Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In the main text, we report only average of five runs for all the techniques. But we report standard deviation and conduct statistical significance in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: Yes]
Justification:
We have provided sufficient information on the computational resources used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://tppppppppp.......aaaaaaat cccppullc//thicccciieelllll
Answer: [Yes]
Justification:
We do not resort to crowdsourcing or contract work for our research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work is on fundamental research and we do not find any direct path to any negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
22

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper presents a fundamental research and does not release any model or data that can be easily misused.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, we cite the source of the data used in the paper
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
23

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, batasets.papersstt paperswi thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will be making our implementation publicly available along with docu- mentation to support reproducibility.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: human subjects are involved in this research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No human subjects are involved in this research.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
25


Reviews:
[{"review_id": "8xI4oyafO5", "reviewer": "Reviewer_VPp3", "summary": "This paper proposes a new Decision-Focused Learning (DFL) approach that specifically addresses learning/predicting constraint parameters for (combinatorial) optimization problems. It should be noted that the vast majority of research on DFL deals with predicting objective function parameters, and only few publications deal with DFL for constraints. Predicting constraint parameters is challenging since in case of hard constraints wrong predictions can lead to infeasible solutions. The paper introduces a new loss function consisting of a convex combination of two (also new) loss functions that explicitly address infeasibility: The first loss function (IAL) aims at minimizing the probability that that the estimated parameters induce a constraint violation, the second loss function (OPL) aims at minimizing the probability that the (true) optimal solution is violated in the optimization problem parameterized by the predicted parameters. The relative weight of the two loss functions in the compound loss function can be adjusted according to the preferences of the decision maker.\nIn a set of computational experiments with the multidimensional knapsack problem and an alloy production problem, it is shown that by varying the weight parameter of the loss function, it is possible to navigate the trade-off between a small regret and a small probability of yielding infeasible solutions.", "questions": "1.\tReferring to my comment above, please provide additional results using the approach detailed in Appendix A2 of the Hu et.al. 2023 NeurIPS paper, with varying penalty weights. \n2.\tGiven that also the approach in that paper uses an indicator function for infeasibility, could you please elaborate a bit more on the differences to your approach?\n3. How to choose the buffer parameter?", "limitations": "Limitations are addressed, with the exception mentioned above that some related work is not represprented and considered adequately.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1.\tThe paper proposes an innovative approach for DFL with uncertain constraint parameters.\n2.\tThe new loss functions are very adequate and seem to work well\n3.\tThe idea of introducing a parameter for balancing the trade-off between small regrets and probability of infeasible is very good\n4.\tThe computational results show that the approach allows to trade optimality vs feasible using the introduced parameter, and for some settings of the trade-off weight, the approach seems to dominate alternative approaches\n\n### Weaknesses:\n1.\tFor me, an important concern with the paper is that it does not properly account the approach from Hu et al 2023, NeurIPS) (Reference 13 in the submission), both when describing the approach and also in the computational experiment.    While it is true that the approach is designed as a two-stage approach with so-called corrective actions / penalty functions, the paper from Hu clearly describe how to use it in a setting with hard constraints, the details can be found in the Appendix .. of Hu ; they use an indicator function combined with a penalty in order to make predictions inducing infeasibilities less likely. In that appendix, there is even a small set of computational results with the 0/1 knapsack problem in which they show that increasing the penalty increases the empirical probability of achieving feasible solutions (e.g. a probability of >= 99% in case of a penalty of 4. \nAs a result, I would have expected the authors to include results with the penalty function mentioned in the Appendix of Hu et al., and with much higher penalty values: In the present manuscript, you use small penalty values that were used by Hu et al in a setting with corrective actions, which is not appropriate.\n2.\tThe authors also write that existing approaches do not allow adjusting the prediction based on their subjective preference between optimality and feasibility. Given the previous remark, by adjusting the penalty weight when using the penalty function for hard constraints, the approach does allow that.\n3. The \"buffer\" parameter is introduced in the paper, but the paper gives no hint on how to choose it, although I feel it will have a considerable impact on the prediction.", "was_revised": true, "final_justification": "My main concern with the paper was that I found the authors did not perform a thorough comparison against the Hu et al. paper. This issue was addressed in the review, so I am willing to increase my score.", "rebuttal": "Thank you for your review of our work and clarification questions, which we address below: \n\n**Comparison against Hu et al 2023, NeurIPS:**\n-  We first clarify how our use of an indicator function is very different from the approach by Hu et al:   \n    - The approach proposed by Hu et al. consists of two stages. Let the first-stage solution be $x_1^\\star$ and the second-stage solution be $x_2^\\star$. To apply their approach in a setting where performing a recourse action is not possible, they indeed mention in the appendix, that they impose a high penalty value $P \\to \\infty$ if $x_2^\\star$ differs from $x_1^\\star$. So, the indicator function is used to denote whether the *second-stage* solution $x_2^\\star$ is different from the first-stage solution be $x_1^\\star$, resulting in the loss function: $P * \\mathbb{1} [x_1^\\star \\neq x_2^\\star] $. .  \n    - On the other hand, we use an indicator function in the IAL loss to denote whether the (first/only stage) assignment is infeasible, i.e., whether it violates any constraints. More specifically, we use an indicator function to denote whether an assignment $x$, violates one specific constraint, e.g., for the $i^\\text{th}$ constraint: $g_i \\leq 0$, i.e., $\\mathbb{1} [g_i(x) >0] $.\n    - Moreover, in the approach of Hu et al., the indicator function $P * \\mathbb{1} [x_1^\\star \\neq x_2^\\star] $ is used in the objective function of the second-stage optimization problem. Instead, we only use an indicator function over individual constraints in one of the two losses, after we solve the original optimization problem. So, our approach is very different indeed. \n\n- Secondly, Hu et al. indeed have a short discussion in the appendix on how to apply their method in applications where recourse actions are not possible. As per your suggestion, we ran their approach for different values of penalty $P$ on the Brass Alloy problem, as shown below. We do not observe that the infeasibility ratio monotonically goes down with increasing $P$, as they did for the KP problem. On the other hand, by increasing $\\alpha$, we show that our approach can bring down infeasibility rate ($0.14$ at $\\alpha=0.8$ in Table 3 in the Appendix), which is lower than any of the values achieved with the method of Hu et al. \n\n| Penalty (P)| Infeasibility  |\n|---------|-------------------|\n| 0.25     | 0.6 $\\pm$ 0.55             |\n| 0.5     | 0.76 $\\pm$ 0.43               |\n| 4.0     | 0.20 $\\pm$ 0.45           |\n| 8.0    | 0.6  $\\pm$ 0.55               |\n\nWe also started the same experiment on the MDKP problems, but due to the large running time of the Hu et al. method, these have not finished yet. \n- **Regarding the buffer parameter:** We have set the value of the buffer to $\\gamma=2$ based on early experiments. It can also be treated as a hyperparameter, where its value would be set based on the result on a validation set, which could even better result, at the computational cost of hyperparameter tuning."}, {"review_id": "1yxNFIfWso", "reviewer": "Reviewer_pSY2", "summary": "The paper is in the domain of Decision Focused Learning, where we have to predict parameters of some optimisation problem and subsequently solve it. Instead of minimising the error in predicted parameters, Decision Focused learning minimises the error in decision outcomes from the optimisation problem. However, in a constrained optimisation problem, if predicted parameters also include the constraints, then such a DFL setup may lead to infeasible solutions. This work proposes DFL in constrained optimisation problems by proposing two loss functions -  Infeasibility Penalty Loss (IPL) and Optimality-Preserving Loss (OPL). By adjusting the weighted combination of these two losses, the end user can decide the tradeoff between feasibility and optimality. The paper also showcases empirical results on two constrained optimisation problems - a multi-dimensional knapsack problem and a covering LP problem. Compared to baselines, the proposed method precisely allows controlling performance in feasibility or optimality dimensions based on the loss combination weight.", "questions": "Questions and Comments:\n\n5 seeds seem to be a low number for the empirical experiments. It would be useful to run more seeded experiments.\n\nWhat would be the kinds of scenarios where a decision-maker would choose feasibility over optimality (or vice versa)? While the paper shows that the proposed technique allows precise control over the tradeoff between the two, it is not motivated why we want this control and further unclear how the decision maker will choose the mixing weight. A decision on these questions would be critical to evaluate the significance of the contributions of this paper.\n\nIn the Optimality-Preserving Loss formulation described in lines 214-216, when you replace a feasible point x with x*, it is a bit unclear how much the loss would be sensitive to the choice of the feasible point x?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper is well written and easy to follow, and the problem formulation is comprehensive. Claims on the correctness of proposed loss functions are supported by proofs.\n\nThe problem of predicting constraint parameters in optimisation problems as a single-stage decision-focused learning problem is important. Unlike prior work, the proposed method in the paper can handle broad classes of constrained optimisation problems.\n\nLikelihood perspective to DFL, i.e., maximising likelihood of the predicted parameters leading to an optimal solution, is an interesting framing, especially because it encapsulates both the objective function parameters and the constraint parameters.\n\n\n\nWeaknesses:\n\nWhile the paper claims that the method is applicable for a general class of constrained optimisation problems (even outside linear programs and mixed integer programs), the two problem settings in the empirical results seem like somewhat small examples and are rather limited. Prior works in DFL, for instance, have often used real-world data from healthcare, finance, etc,  which model the relationship between real-world features and predicted parameters of optimisation problems. It would be useful to expand the set of experiments to such scenarios.\n\nIt would be useful to interpret the reasons why and in what scenarios the SFL or existing method prefer feasibility or optimality. For example, in Figure 1, we see that SFL chooses a solution that balances the tradeoff in two dimensions for the ‘predicting capacities in MDKP’ problem, but it favours minimising regret in the other two problems. No reasoning is provided for why such behaviour happens (and the interpretation of parameters which lead to the proposed method’s control of the tradeoff).", "was_revised": true, "final_justification": "The authors have described that the dataset and problem setting in the paper have been used to benchmark the Decision Focused Learning technique in multiple prior works as well. They have described the connections between controlling the optimality-feasability tradeoff and Chance-Constrained Optimisation. They have also explained the behaviour of solutions from SFL and given an intuition on why its solutions lead to poorer performance, and particularly why SFL tends to choose a specific style of solutions. Lastly, the authors have pointed to the statistical significance tests they have conducted to show that ODECE performs significantly differently from the baseline methods. These points adequately answer the queries I had, and it would be useful to highlight them in the main paper to strengthen the contributions of this work.", "rebuttal": "Thank you for your review and analysis and the positive assessment that the theoretical claims of our proposed loss functions are well supported by formal proofs. \n- **Regarding the scope of the experimental setup:** We want to clarify that the Brass Alloy Production problem originates from a real use case and is based on real-world data. Moreover, this use case was previously used by Hu et al. in their NeurIPS paper. The Multi-Dimensional Knapsack Problem (MDKP) is adapted from the knapsack experiment in the PyEPO benchmark, which is widely used to evaluate DFL methodologies (e.g., recently by Zharmagambetov et al., NeurIPS (2023); Lin, Delage, and Chan, NeurIPS (2024); Huang and Gupta, NeurIPS (2024)). We used the same underlying relationship between features and optimization parameters as used in these papers, with the only difference being that the predicted parameters appear in the constraints in our work. While more experiments are always of interest, we argue that our current scope is in line with that of existing work and shows the promise of the approach.  \n\n\n- **Choosing feasibility over optimality:** We explain in the text that achieving both optimality and feasibility involves a trade-off, as enforcing strict feasibility can lead to overly suboptimal solutions. To manage this, our approach introduces a tunable parameter $\\alpha$ that balances the two loss components: IPL (penalizing infeasibility) and OPL (penalizing suboptimality). This formulation allows users to navigate the trade-off according to their preferences. The idea is similar in spirit to chance-constrained optimization (CCO), where no recourse actions are applied and where constraints are instead required to hold up to a specified probability to provide reliability under uncertainty. However, CCO typically assumes that the uncertain parameters follow a known distribution. In contrast, our approach makes point prediction of the constraint parameters without assuming they come from certain distributions. In CCO, the probability level is often chosen based on domain-specific reliability requirements (e.g., acceptable tolerances in production lines). Likewise, $\\alpha$ could also be chosen based on domain-specific reliability requirements. \n\n- **Performance of SFL:** We observed that the SFL technique tends to produce low-magnitude gradients for the constraint parameter $a$ (the LHS of constraints of the form $ax \\leq b$) and relatively higher gradients value for the RHS parameter $b$. We suspect this behavior is due to the way negative samples are generated. For example, in the MDKP problem, if the optimal solution is [0,0,1] in a 3-dimensional MDKP, the negative samples include [1,1,1],[1,1,0], [0,1,1] and [1,0,1]. These negative samples can be classified as negative based on capacity, the RHS parameter.\nBased on this, we believe that the SFL approach tends to place more importance on the RHS parameter, and the LHS parameter is less frequently used in discriminating between positive and negative samples. This in our view leads to poor performance when SFL is used to estimate only the LHS parameter.\n\n\n- **Experiment with five seeds:** We ran all experiments with five different random seeds and reported both the average and standard deviation in the Appendix. Additionally, we performed statistical significance tests to show that ODECE performs significantly differently from the baseline methods. This provides substantial evidence that our conclusions are valid.  \n\n- **Choosing Feasible Point in OPL Loss Formulation:** The performance of the proposed loss indeed depends on the choice of the feasible point used in Eq. (15). We use the true optimal point (with respect to the true parameters) as the feasible point for two main reasons: \n    1.  It is already available in the training set, and hence does not require solving an additional satisfaction problems; this could be expensive depending on the constraints.  \n    2. It enables Lemma 2, which guarantees that zero IPL and OPL imply optimality under true parameters—something we couldn’t prove with arbitrary feasible points."}, {"review_id": "XYIfzuxbCg", "reviewer": "Reviewer_fGw6", "summary": "Generally, this paper proposed a trade-off learning approach for decision-focused learning/prediction. The penalization for infeasibility and suboptimality is balanced through a convex combination of them. The effectiveness of the proposed penalization is proven theoretically. Experimental results show the performance of the approach under different trade-off parameters. \n\nThe proposed method is well-formulated and empirically supported, and the authors clearly state that \"it is important to simultaneously manage both feasibility and decision quality.\" While the tunable trade-off parameter introduced in the paper provides a flexible way for decision-makers to control this balance, its practical utility could be further improved by offering guidance on how to align the parameter with real-world decision-making preferences. In particular, when constraint parameters are implicit or arise from black-box systems, it becomes nontrivial for decision-makers to calibrate the trade-off between feasibility and suboptimality without a structured mechanism or interpretive support. As acknowledged in the paper, decision-makers may wish to “control the trade-off between suboptimality and infeasibility” according to their preference, and designing data-driven or preference-informed ways to adapt this parameter could enhance the applicability of the approach in practical settings.", "questions": "1. The essence of the penalization is the amount of constraint violation and margin, after ReLU approximation. Considering the value range of the different constraint functions is different, a proper normalization approach should be proposed. \n2. For equality constraints, the higher buffer (constraint margin parameter) is likely to kill their influence on the loss function. Meanwhile, the decision makers’ concern about the violation direction may be different; this concern could possibly solve this problem.\n3. The selection of the convex combination trade-off parameter should be discussed. \n4. A little more related work should be discussed and cited.", "limitations": "Yes, the biggest limitation of the proposed method is that it requires solving the programming problem during the training process of the prediction model.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The quality of this paper is generally high. The present is clear; perhaps more results should be discussed. The approach proposed in this paper doesn’t show high/strong significance compared with other existing works. From the point of view of the reviewer, the work in this paper is original. Specifically, the derivation of the two loss functions (one penalizes infeasibility and the other penalizes suboptimality) is principled and well-grounded in a maximum likelihood framework. The proposed method is general enough to accommodate nonlinear and integer programs, unlike many prior works restricted to LPs/ILPs. Experiments demonstrate that tuning the trade-off parameter $\\alpha$ can achieve a flexible balance between regret and feasibility, which may align with different decision-maker preferences.\n\nWhile the proposed formulation is sound and the empirical results are convincing, the paper could benefit from a deeper discussion on how decision-makers can select or adapt the trade-off parameter $\\alpha$ in practice. This is particularly important because α encodes a key behavioral preference — how to weigh feasibility versus optimality under uncertainty — but is currently treated as an external hyperparameter, selected by trial. More specifically, in practical applications, it is challenging for decision-makers to specify their preferences through the hyperparameter $\\alpha$ without clear guidance, which may limit the practical utility of the proposed method.", "was_revised": true, "final_justification": "I would keep my current score, because it is challenging for decision-makers to specify their preferences through the hyperparameter $\\alpha$ without clear guidance. Generally, this paper proposed a trade-off learning approach for decision-focused learning/prediction. I'm fine if the paper is accepted.", "rebuttal": "Thank you for your suggestion to expand the discussion on related work.  We will provide a more extensive review of DFL and contextual stochastic optimization by citing the recent survey papers by Mandi et al., JAIR (2024) and Sadana et al., EJOR (2025) and the references therein. The paper closest to ours is the work by Hu et al., who also focus on making point predictions of constraint parameters. However, their approach assumes the existence of a recourse action in case of infeasibility.  We refer to our response to reviewer VPp3, where we highlight how their approach differs from ours. Note that we also show that by increasing $\\alpha$, our approach can deliver lower infeasibility than theirs. \n\n- **Normalizing Constraint Violation:** We experimented with normalizing constraint violations $\\max\\big( \\big( \\Upsilon + g_i \\left( x^\\star , \\hat{\\rho} \\right) \\big) , 0 \\big)$ by dividing by absolute value of the true constraint values $|g_i \\left( x^\\star, \\rho \\right)|$. This did not work because in our setting, the absolute magnitude of violation is what directly impacts feasibility. Normalization led to disproportionately smaller penalties for constraints with higher true values. Moreover, constraints with higher values of $|g_i \\left( x^\\star, \\rho \\right)|$ are more likely to result in violations with the predicted constraint $\\hat{\\rho}$. As a result, normalizing by the absolute value of the true constraint reduced the penalty for such constraints, increasing the overall rate of infeasibility. \n\n- **Buffer with equality constraints:** We formulated the problem definition only with inequality constraints. Any equality constraints can be split into two inequality constraints, but in that case the buffer will point in the opposite direction. We have not experimented with equality constraints yet. One possible solution is to keep the buffer zero for equality constraints.\n\n- **Setting $\\alpha$:** Our motivation was to leave the selection of the tunable parameter, $\\alpha$, on the decision-maker. $\\alpha$ controls the trade-off between feasibility and optimality. As mentioned in response to Reviewer pSY2, the choice of parameter can be set empirically, for example based on domain-specific reliability requirements, e.g. tolerance in production lines."}]

Author rebuttal:

Meta-review: This paper presents a decision-focused learning (DFL) framework for predicting constraint parameters in constrained optimization problems (in contrast to the majority of DFL research, which deals with predicting objective function parameters). As constraint parameter prediction may lead to issues regarding optimization feasibility, the submission introduces a mixed loss function penalizing both infeasibility and solution optimality, with a weighting between them that can be chosen by a decision-maker. Experiments on multi-dimensional knapsack and brass alloy production problems show compelling performance compared to baselines, as well as the ability to indeed trade off between feasibility and optimality via choice of the weighting parameter.

Strengths: Reviewers felt the proposed approach is innovative -- e.g., one reviewer noted that the likelihood perspective to DFL is an interesting framing, and a number commended the conceptual focus on constraints and on broad classes of problems. The computational results are informative in demonstrating the efficacy of the approach in terms of overall performance and trading off between feasibility and optimality. The writing and presentation are clear.

Weaknesses: Reviewers felt it to be a strong assumption that the decision-maker would know the precise desired tradeoff between feasibility and optimality -- this merits additional discussion in the submission. They also felt the number of experimental settings was more limited than it could have been, and had questions regarding the extent to which this framework is more broadly applicable in practice.

Other major discussion points: 
* Reviewer VPp3 initially pointed out the lack of comparison against Hu et al (2023) - however, this was resolved in the reviewer-author discussion via clarifications and an additional experiment by the authors. The authors should include these additional experiments (for both multi-dimensional knapsack and brass alloy) in the revised paper.
* Reviewer pSY2 initially pointed out a lack of explanation regarding when/why the SFL baseline method chooses to prioritize feasibility vs. optimality in different settings, as well as the connections between controlling the optimality-feasibility tradeoff and chance-constrained optimization. The authors addressed this in the discussion, and indicated they would update the paper to address these points.
* Reviewer pSY2 initially also pointed out that the authors run experiments over 5 seeds, but the authors clarified that they have run statistical significance tests to show their method's improvement is indeed significant.

Overall, this seems to be an interesting and technically solid contribution, and while there is room for improvement, the strengths are sufficient to warrant recommendation of acceptance.
Decision: Accept (poster)