Decision-Focused Learning without Decision-Making: Learning Locally Optimized Decision Losses
Authors: Sanket Shah
Kai Wang
Bryan Wilder
Andrew Perrault
Milind Tambe
Venue: NeurIPS 2022 Accept (2022)
Keywords: Decision-Focused Learning
Prediction
Optimization
Machine Learning

Abstract: Decision-Focused Learning (DFL) is a paradigm for tailoring a predictive model to a downstream optimization task that uses its predictions in order to perform better \textit{on that specific task}. The main technical challenge associated with DFL is that it requires being able to differentiate through the optimization problem, which is difficult due to discontinuous solutions and other challenges. Past work has largely gotten around this this issue by \textit{handcrafting} task-specific surrogates to the original optimization problem that provide informative gradients when differentiated through. However, the need to handcraft surrogates for each new task limits the usability of DFL. In addition, there are often no guarantees about the convexity of the resulting surrogates and, as a result, training a predictive model using them can lead to inferior local optima. In this paper, we do away with surrogates altogether and instead \textit{learn} loss functions that capture task-specific information. To the best of our knowledge, ours is the first approach that entirely replaces the optimization component of decision-focused learning with a loss that is automatically learned. Our approach (a) only requires access to a black-box oracle that can solve the optimization problem and is thus \textit{generalizable}, and (b) can be \textit{convex by construction} and so can be easily optimized over. We evaluate our approach on three resource allocation problems from the literature and find that our approach outperforms learning without taking into account task-structure in all three domains, and even hand-crafted surrogates from the literature.

Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses
Sanket Shah Harvard University sanketshahhg harvard. edu
Kai Wang Harvard University
Bryan Wilder
Carnegie Mellon University
kaiwangog, harvard edu
ovilder@andrew cmu edu
Andrew Perrault
Milind Tambe
The Ohio State University perrault. 170osu. edu
Harvard University
tainndttambbhaararrrd.
Abstract
Decision-Focused Learning (DFL) is paradigm for tailoring predictive model a downstream optimization task that uses its predictions in order to perform bet- ter on that specific task. The main technical challenge associated with DFL is that it requires being able to differentiate through the optimization problem, which difficult due to discontinuous solutions and other challenges. Past work has largely gotten around this this issue by handcrafting task-specific surrogates the orig- inal optimization problem that provide informative gradients when differentiated through. However, the need to handcraft surrogates for each new task limits the usability of DFL. In addition, there are often guarantees about the convexity of the resulting surrogates and, as result, training a predictive model using them can lead inferior local optima. In this paper, do away with surrogates altogether and instead learn loss functions that capture task-specific information. To the best of our knowledge, ours is the first approach that entirely replaces the optimization component decision-focusee learning with a loss that is automatically learned. Our approach (a) only requires access to black-box oracle that solve the opti- mization problem and is thus generalizable, and (b) can be convex by construction and so can be easily optimized over. We evaluate our approach on three resource allocation problems from the literature and find that our approach outperforms learning without taking into account task-structure in all three domains, and even hand-crafted surrogates from the literature.
Introduction
Predict- tedlictthen-oppimmize 7 g is framework for using machine learning to perform makingg As the name suggests, it proceeds two stages- first, predictive model takes as input features and makes some predictions using them, then second, these predictions are used to parameterize an optimization problem that outputs a decision. A large number real-world applications involve both prediction and optimization components and can be framed as predict-then-optimize problems- for e.g., recommender systems in which missing user-item ratings need to be predicted [3], portfolio optimization in which future performance needs to be predicted [77 or strategic decision-making which the adversary behavior needs to be predicted [4]
In addition to wide applicability, this framework also formalizes the relationship between prediction and decision-making This is important because such predictive models are typically learned inde-
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

pendently of the downstream optimization task in Machine Learning-based decision- makinng sys- tems, and recent work in the predic-thhen-ppinng setting E. [6 25
26 has shown that it is possible to achieve better task-specific performance by tailoring the predictive model to the downstream task. This is often done by differentiating through the entire prediction and optimiza- tion pipeline end-to-end, leading to family of approaches that we will refer to as decision-focused learning (DFL) 125. Optimizing directly for the quality of decisions induced by the predictive model in this end-to-end manner yields loss function we call the decision loss.
However, training with the decision loss be challenging because the solutions to optimization problems are often discontinuous in the predictions (see Section 2.T). This results in an uninfor- mative loss function with zero or undefined gradients, neither of which are useful for learning predictive model. To address this, DFL approaches often leverage handcrafted surrogate optimiza- tion tasks that provides more useful gradients. These surrogate problems may be constructed by adding regularization to the objective B
relaxing the original problem B T6 25 9
25 27.. or
even using entirely an different optimization problem that shares the same decision space 126
Designing good surrogates is an art, requiring manual effort, insight into the optimization problem of interest. In addition, there are no guarantees that the surrogates induce convex decision losses, leading to local optima that further complicate training. Instead, we propose a fundamentally dif- ferent approach: to learn decision loss directly for a given task, circumventing surrogate problem design entirely. Our framework represents the loss as a function in a particular parametric family and selects parameters which provide informative loss for the optimization task. We call the resulting loss a locally optimized decision loss (LODL).
Our starting point is the observation that good decision loss should satisfy properties: it should (i) be faithful to the original task, i.e., the decision quality is consistent with the original problem; (ii) provide informative gradients everywhere (i.c., defined and non-zero); and (iii) be convex in predic- tion space avoid local minima. These demands are in tension-the the first requirement prevents the loss function from being modified too much to achieve the other two. It is not obvious apriori that any tractable parametric family should be able to simultaneously satisfy all three properties for the complex structure induced by many optimization tasks. We resolve this tension by separately model- ing the loss function locally for the neighborhood around each individual training example. Faithful representation of the decision loss is easier to accomplish locally in each individual neighborhood than globally across instances, allowing us to introduce convex parametric families of loss functions which capture structural intuitions about properties important for optimization. To fit the parame- ters, we sample points in the neighborhood of the true labels, evaluate the decision loss associated with these sampled points, and then train loss function to mimic the decision loss.
We evaluate LODLs on three resource allocation domains from the literature surprisingly, we find that LODLs outperform handcrafted surrogates in two out of the three. In our analysis, discover linear correlation between the agreement of the learned LODL with the decision loss and the decision loss of a predictive model learned using said LODL. Our approach motivates new line of research on decision-focused learning.
112
25
24
Perhaps
2 Background
In predict-then-optimizz: a predictive model Mg first takes as input features æ and produces predic- tions ÿ = Ma (x) These predictions are then used to parameterize optimization problem that is solved to yield decisions 2" z²())
( arg min f(z: 1)
s.1. gx(2) & 0, for E (1,..., m}
Note that, unlike typical machine learning problems, the dimensionality of dim D is likely to be large as it consists of all predictions needed to parameterize the optimization problem. Given the large dimensionality, and the similarity in the role of all the predictions, the predictive model typically predicts individual components of g. i.e. 9 = [ύ1.... uDl = [Mo (M(()))..... Mo ()))).
Predictions are evaluated with respect to the decision loss (DL) of the decision that they induce, i.e., the value under the objective function of the optimization under the ground truth parameters y:
DL(0. f(z" (0), y)
2

---

2 S.Saag
Prodictions
Intermediate
True Labels
Learn
DFL
Predictions
Features
Model
Optimizer
Decision Loss
True Labels
Learn
Leam
Siop
Sampled Predictions
Optimizer
Decisions
Decision Loss
True Labels
Sampler
Learn
MSE
LODI
LODL
Step
Prodictions
Features
Model
Learned LODL.
True Labels
Figure 1: Schematic highlighting how the predictive model Mo learned in different approaches. LODL does not require backpropagating through the optimization problem.
Thus, for dataset [(x1, 81), ..., (EN, BN)], we aim to learn model Mg that generates predictions ...... Dv] that minimize the decision loss DL:
N Σ n=1
0" = arg min
(M ym.
N
0
This is in contrast with standard supervised learning approaches which the quality of a prediction measured by somewhat arbitrary intermediate loss (c.g., mean squared error) that does not contain information about the downstream decision-making task. In this paper, we refer to models that use an intermediate loss as 2-stage and those that directly optimize for DL decision-focused learning (DFL). Figure outlines how predictive model is learned using these different approaches.
2.1 Motivating Example
Consider an arg min optimization where the goal is to predict the disutility of 2 agents (A, B), "Pick A". and
e.g., y = (0, 1). Now, if these parameters predicted perfectly, the decision the decision loss DL is the true disutility of agent A, i.e., DL = 0.
On the other hand, consider the set of predictions Ghad = (1 + c,, 0 + EB) for 0 < <, < 0.5. Any prediction this set will yield the decision 'Pick B" and decision loss DL. of Given that the decision loss is constant in this region, the gradients are all zero, i.e., V DLD y) gEggoot 0. As
result, a predictive model makes such prediction, cannot improve its predictions by gradient descent. Therefore, although D T. is what we want to minimize, we don't want to fit it perfectly.
Preatcted
Predicted Diautiity for
Predicted Disubily for
Provlcted Disudtl for
Mowooe SAwwwi for
0
(a) True Decision Loss
(b) Handcrafted Surrogate
LODL
Figure 2: Graphs plotting the values of different loss functions (Blue = 1, Red = 0) as function of
the predictions of different disutilities for and
If instead of minimizing DL directly, we minimized surrogate loss SL = ÚA ÔB. gradient descent would lead to the model to predict -o0, oo) regardless of the initialization (see Figure 2b. While isn't the frue set of disutilities, it does lead to the optimal decision, i.e., "Pick A" and is thus effective from trepoii-tee--ottmmiee perspective.
The challenge is then coming up with such surrogates. Although it is easy for the problem above, it becomes more complicated as the number and types of variables and constraints grows. Below, we propose an approach to automatically learn good surrogates. Figure 2c shows what our method looks like for the example above.

---

3
Related Work
A great deal of recent work on decision-focused learning and related topics aims to incorporate in- formation about downstream optimization problem into the training of a machine learning models. Some optimization problems (especially strongly convex ones) can be directly differentiated through 1 For others. particularly discrete optimization problems with ill-defined gradients, a variety of approaches have been proposed. Most of these construct surrogate loss functions via smoothing the optimization problem [8
23 [T]. Alternatively, Elmachtoub and Grigas S propose a closed-form convex surrogate loss with desirable theoretical properties: however, this loss applies only when the optimization problem has a linear objective function. Similarly, Mulamba et al. (20) provide a contrastive learning-basce surrogate which does not require differentiation through opti- mization, but which is also developed specifically for linear objectives. When the predictive model is itself a linear function, Guler et al. |T propose an approach to search directly for the best model.
25
D 26
Perhaps the most related work to ours is by Berthet et al. [4 They differentiate through linear opti- mization problems during training by adding randomized perturbations to smooth the induced loss function. Specifically, they randomly perturb the predictions ú with random noise and solve for 2" =" +e), which can be interpreted as replacing the decision loss with its averaged value in neigh- borhood around V. where the averaging smooths the function and ensures differentiability. There are two key differences between our approach and theirs. First, they apply random perturbations to optimization in the training loop in order to produce smoother surrogate loss. By contrast, we use random perturbations learn a loss function prior to training; during training optimization is removed entirely. This allows us to use the same random samples to inform each training iteration instead of drawing new samples per iteration. Second, their approach applies only to linear objective functions while ours applies to arbitrary optimization problems.
4 Locally Optimized Decision Losses (LODL)
In this paper, we do away with the need for custom task-speeific relaxations of the optimization problem 2= =")) by instead translating task-specific information from the decision loss DL into a loss function LODL (9. y) that (i) approximates the behavior of DL, and (ii) is convex by construction.
Our broad strategy to do this is to learn the function LODLS 2 learning. Specifically, we proceed in 3 steps:
DL using supervised machine
First, we learn a separate LODI, for every ((xr, y)) pair in the dataset to make our learning problem easier. Second, we note that there's chicken-and-ege problem associated with learning L.DDIs to train the LODI we need inputs of the form (g. y). but to produce ÿ we need predictive model trained said LODL. To resolve this, we make the assumption that our predictive model Mg will get us sufficiently close to the true labels y. This means:
We simplify the learning problem in two ways (Section 4.1
LODL ④ y) LODLS, (01) ..... LODLON (iN)), and G. x y
2. Given these simplifications, we propose convex-hy-constructin parametric forms for LODLo, (Section 4.2). We subtract a constant DL y:) from the target ensure that the function to be learned has a minima at ÿ y; and that the result can be modeled well by convex function:
constant
LODL (0.) ~ DL v.) - (L y.) = DL ve LODLS, (0.) + DL 31
We propose different strategies for sampling 9 (Section 4.3 and also describe the equation
used to learn the optimal LODL parameters 6* (Equation
4.1 Local Loss Functions
We introduce a separate set of parameters &n. for each (iii,y1).... (In, yr). ...,, (x yn)] the
training set. We take this step because learning global approximation to DL(ū. y) (for arbitrary Φ) hard; it requires learning closed-form approximation to the general optimization problem 2 (g) which may not always exist. Introducing separate parameters per-instance gives two key advantages.

---

1. Learning for a specific Yn: Instead of learning a (dim()))+dmm()) function LODL (ū. y) to imitate DL(.. y), we instead learn N different Rdim()) functions LODI (i) that imitate DLÍÿ. yn) for each n E N. Doing this significantly reduces the dimensionality of the learning problem-this is especially relevant when N is not very large in comparison to dim(y) (as in our experiments). It also cireumvents the need to enforce invariance properties on the global loss. For example, many optimization problems are invariant to permutations of the ordering of dimensions in y, making it difficult to measure the quality of ÿ's 's across different instances. In a local loss, the ordering of the dimensions is fixed and so this issue is no longer relevant.
2. Learning for only In R yn: In addition to the simplification above, we don't try to learn a faithful approximation Vÿ E (Rddim()) we instead limit ourselves to learning an approximation of (g yn) only when ğ is in the neighborhood of yn- We assume that our predictive model will always get us in the neighborhood of the true labels, and the utility of LODL is in helping distinguish between these points.
The combination of these two choices makes the learned LODLo. a "local" surrogate for DL, rather than a global one.
4.2 Representing the LODL
The key design choice in instantiating our framework is choice of the parametric family used to represent the LODL. Optimization problems can induce a complex loss landscape which is not easily summarized in a closed-form function with concise parameterization. Accordingly, we design a set of families which capture phenomena particularly important for common families of predict-then- optimize problems.
Parametric families for the local losses Having made the decision to allow separate parameters for each training instance, the second design choice is how to represent each local loss, i.c., the specific parametric family to use. Our choice must be sufficiently expressive to capture the local dynamics of the optimization problem while remaining sufficiently efficient to be replicated across the N training instances. We propose that the structure of the loss function should capture the underlying rationale for why dacision-focused learning provides advantage over 2-stage in the first place; this is the key behavior which will underpin improved decision quality. We identify three key phenomena which motivate the design of the family of loss functions:
1. Relative importance of different dimensions: Typically, the different dimensions of a pre- diction problem are given equal weight, e.g., the MSE weights errors in each coordinate of y equally. However, there may be some dimensions along which DL is more sensitive to local per- turbations. For example, a knapsack problem may be especially sensitive to errors the value of items which are on the cusp of being chosen. In such cases, DFL can capture the relative i importance of accurately predicting different dimensions.
2. Cost of correlation: Given the possibly large dimensionality of y. in practice, the predictive model Ma does not typically predict y directly. Instead, the the structure of the optimization is exploited to make multiple predictions (",...., 99] that are then combined to create ÿ. For example, in a knapsack problem, we might train a model which separately predicts the value of each item (i.e., predicts each entry of ÿ separately) using features specific to that item, instead of jointly predicting the entire set of values using the features of all of the items. However, Cameron al. show that ignoring the correlation between different sub-y scale predictions (as in 2- stage) can result in poor optimization performance, and that DFL can improve by propagating information about the interactions between entries of y the predictive model.
3. Directionality of predictions: In the arg min(ý). (1, (2) example from the introduction, the pre- diction أ 2, D22 produces the same decision ÿ1 2000, ý22 1. On the other hand, the prediction 인1 = 0.5, 82 = leads to a different decision. As result, over-predicting and under-predicting often have different associated costs for some optimization problems. Predic- tive models trained with DFL can take into account this behavior while those trained by typical symmetric 2-Stage losses like MSE cannot.
Given these insights we propose three corresponding families of loss functions for LODL. each of which convex by design and has global minima at the true label you a desirable property because DL also has its minima at ğ y.
5

---

1. WeightedMSSE: To take into account the relative importance of different dimensions, we pro- pose weighted version MSE:
dim(ly)
WeightedMSE(()-
(úr v1)?
1=1
R dimm(y) (for convexity).
where 'weights' uy are the parameters of the LODL, i.e., 2. Quadratic: To take into account the effect of correlation of different dimensions each other, we propose learning a quadratic function that has terms of the form (ū: yn) y5):
Quadratic( = y) H y),
where Ø- H is a learned low-rank symmetric Positive semidefinite (PSD) matrix. This family of functions is convex long as H is PSD, which we enforce by parameterizing
L where L is low-rank triangular matrix L of dimension dim( x k and le is the desired rank. This loss function family has an appealing interpretation because learning LODL is similar to estimating the partial derivative of DL with respect to first input ÿn- Specifically, consider the first three terms of the Taylor expansion of DL with respect to Gn at (Yn: Un))
9 DL + yn) = DL(yn, + Vg. (L Un) E + ET V22 DL yn) +
constant
0 (yn. V... miinim
Hossian
DL yn) + (9.. y,,) H yn)
Quadratic LODLS can be seen as a 2nd Taylor-serics approximation of DL at (yn, yn) where the learned I approximates the Hessian of DL. Note that WeightedMSE is special case of this Quadratic loss when H dwag(w).
WeiehtedWWig and DireeteddQuadratic:: To take into account the fact that overpre- have different consequences, propose modifications to the
dicting and underpredicting two loss function families above. For WeightedMSFF, we redefine the weight vector w as below, and learn both w.+ and w... Similarly for Quadratic, define 4 copies of the parameter T. based on the directionality the predictions.
L
if 0:
y: 2 0 and ú, - yj 2 0
13
W44 if Ú yo 2 0
L
if 0: 2 0 and Ú, < 0
Lij
W-, otherwise
if Uz ys 0 and Ú, Yj 0
Lig
otherwise
4.3 Learning LODL
Given families proposed in Section 4.1 our goal is to learn some Φ for every E N such that LODI T.p.. (#.) DL( yn) for yn "close" to Yu- We propose supervised approach to learning 6 which proceeds in two steps (Figure Dh: (1) we build a dataset mapping 9n - DL( yn) in the region of yn. and then (2) we use this dataset to estimate Φ; by minimizing the mean squared error to the true decision loss:
6o arg min E (LODLS. (wt)) Dl())) K
1
On
This framework has the key advantage of reducing the design of good surrogate tasks (a complex problem requiring in-depth knowledge of each optimization problem) to supervised learning (for which many methods available). Indeed, future advances, e.g. in representation learning, can simply be plugged into our framework.
The major remaining step is to specify the construction of the dataset for supervised learning of d; and calculate DL yn)) for each. In this paper, we consider three sampling strategies:
We propose to construct this dataset by sampling a set of points [y!,.... y the vieinity of y
All-Perturbed: Add zero-mean Gaussian noise to the true label yn: y Vn c yn NN(0 T),
where is normalization factor and I is dim(y) x dim(ly identity matrix.

---

2. I-Perturbed or 2-Perturbed: Estimating the behavior of DL + (.)) for small e can alternatively be interpreted as estimating (zg,. (z8 )dimml()) ( (9n. Yv)) (i.c., the dim(y,))* 1 partial derivative of DL w.n.t. its first input (n.) at (Yn. yn) because all the dimensions of Dn are being varied simultaneously. While computing )d) is computationally challenging, can be estimated using the simpler 1 or 2nd partial derivatives. This corresponds to perturbing only two dimensions time.
4.4 Time Complexity of Learning LODLs
The amount of time taken by each of the methods using gradient descent is:
2-Stage ( TM), where TM is the amount of time taken to run one forward and back- wards pass through the model Mg for one optimization instance, N is the number of optimization instances, and is the number of time-steps Ma is trained for.
DFL = N- (TM To + T))), where To is the time taken to solve the forward pass of one optimization instance and To is the time taken to compute the backward pass.
LODL = ((K N To + N . (T K TIoDI) + T N Tas), where K is the number of samples needed train the LODL,, and TRODE, is the amount of time taken to one forward and backwards pass through the LODI. The three terms correspond to (i) generating samples, (ii) training N LODLs, and (iti) training M, using the trained LODLs.
In practice, we find that (TH > To) >> (IM LODL). As a result, the difference in complexity of DFL and LODL is roughly ( N. To) vs. O(K N To + N T K .TLOD)). Further, the calculation above assumes that LODLs are trained in the same way as Mg. However, in practice, they can often be learned much faster, sometimes even in closed form (e.g., WeightedMSE and DirectedWeigheaMMSE)) leading to an effective runtime of O O(K N To). Then, the difference between DFL and LODL boils down to T vs. K, i.e. the number of time-steps needed truin Ma vs. the number of samples needed to truin the LODL.
To + N K TLODL) Ro
While our approach can be more computationally expensive, typically isn't for two reasons:
We need only sample candidate predictions once, to then train any number of LODLs (c.g., WeightedMMS., DirertedQuaddratic without ever having to call an optimization or- acle. Once the LODLs have been learned, you can train any number of predictive models Ma based on said LODLs in contrast DFL, which requires calling the oracle to train each model. DFL is thus more expensive when training a large number of models (e.g.. for hyperparame- ter/architecture search, trading-oft performance vs. inference time vs. interpretability, etc.). In the future, we imagine that datasets could be shipped with not only features and labels, but also LODLs associated with downstream tasks!
Amortization:
Parallelizability:
The sample generation process for LODL is completely parallelizable, result- ing in an ((TT) lower-bound wall-clock complexity for our approach. In contrast, the calls to the optimization oracle in DFL are interleaved with the training of Mo and, as result, cannot be parallelized with respect to T, resulting in an Q(T -o) wall-clock complexity.
We demonstrate this empirically in Section 5.3
5 Experiments
To validate the efficacy of our approach, we run experiments on three resource allocation tasks from the literature. We use the term decision quality DQ (higher is better) instead of DL because these are all maximization problems.
Linear Model This domain involves learning a linear model when the underlying mapping be- tween features and predictions cubic. Such problems are common in the explainable Al litera- ture (21 IO [2] where predictive models must be interpretable.
Predict: Given a feature x, ~ 00, 1). use a linear model to predict the utility ÿ of resource n, where the true utility is yn. 10x, 6.5x., Combining predictions yields ÿ [#ı,.... .., UN).
Optimize: Choose the B 1 out of N 50 resources with highest utility: =" =²(0) arg topk(()
7

---

Table 1: The decision quality achieved by each approach consistently performs well.
higher is better. DirectedQuadratic
Normalized DQ On Test Data
I Loss Function
Linear Model
Web Advertising
Portfolio Optimization
Random Optimal
0
I
2-Stage (MSE)
-0.95 0.00
0.48 = 0.15
0.32 = 0,02
DFL
0.83 = 0.38
0.85 = 0.10
0.35 = 0.02
NN
0.96 0.00
0.81 0.14
-0.11 0.08
WeightedMSE
-0.93 0.06
0.58 0.15
0.31 0.02
SireetedWeightedMMS.
0.96 0.00
0.53 0.14
0.02
Quadratic
-0.75 0.38
0.93 0.04
0.07
DurectedQuadratic
0.96 0.00
0.91 0.04
0.33 0.01
Because the arg max operation is piecewise constant, DFL requires a surrogate- we use the soft Top-K proposed by Xie et al. (27). Although this surrogate is convex in the decision variables, it not convex the predictions.
Surrogate:
Web Advertising This is submodular optimization task taken from Wilder al. 25 The aim is to determine on which websites to advertise given features about different websites.
Predict: Given features Z.m associated with some website m, predict the click-through rates (CTRs) for fixed set of N = 10 users M
websites Im 1m.1,.... Dm,N) To ob-
tain the features Im for each website m, true CTRs ym from the Yahoo! Webscope Dataset [28 are scrambled by multiplying with random N N matrix A, i.e., 3m Aym.
Optimize: Given the matrix of CTRs, determine on which B 2 (budget) websites advertise such that the expected number of users that click on the ad at least once is maximized, i.c.,
z" z"( arg max -1=o( 11
)1;)), where all the E (0.
Surrogate: Instead of requiring that 2, E {0, 1}. the multi-linear relaxation from Wilder et al. 15 allows fractional values. The DL induced by the relaxation is non-convex.
Portfolio Optimization This is a Quadratic Programming domain 24 in which the aim is to choose a distribution over N stocks that maximizes the expected profit minus quadratic risk penalty. We choose this domain as stress test-it is highly favorable for DFL because the opti- mization problem naturally provides informative gradients and thus requires no surrogate.
Predict: Given historical data n, about stock n, predict the future stock price Un. We use histori- cal data from 2004 to 2017 for set of = 50 stocks from the QuandIWIKI dataset [22]
Optimize: Given a historical correlation matrix Q between pairs of stocks, choose a distribution over stocks that maximizes I y A " Qğ, where À 0.1 is the risk aversion constant.
More experimental setup details are provided in Appendix
5.1 Results
We train either linear model (for the Linear Model domain) 2-layer fully-connected neural network with 500 hidden units (for the other domains) using LODLs and compare it to:
Random: The predictions are sampled uniformly from [0, 1jdim(v),
Optimal: The predictions are equal the true labels y.
3. 2-Stage: The model is trained the standard MSE loss ( ΣN ll 13).
4. DFL: Decusion learning using the specified surrogate.
5. NN: To determine how important convexity for LODL we define LODLa NN+(O) in
which NN, is 4-layer fully-connected Neural Network (NN) with 100 hidden units.
Table | shows the main results. We find that, in all domains, training predictive models with LODL outperforms training them with a task-independent 2-stage loss. Surprisingly, also outperforms DFL., which has the benefit of handcrafted surrogates, in two of the three. This is strong evidence favor of our hypothesis that we can automate away the need for handcrafting surrogates. We first analyze the results in terms of the domains:

---

Figure 3: Figure (a) shows that the time taken to train single model Mo reduces with the number of cores used. Figure (b) shows that this time is further reduced when the cost of learning LODLs is amortized across different predictive models Mg.
(a) Parallelizability
(b) Amortization
Training (secondo)
5000
2-Slage
Training Time (saronda)
2-Saae
4000
Weigh HeiMME
DFL
Amortzee
4000
DEL
WeiginedM55
Quardettic
3000
Quadnati
1000
1000
10
20 30
Number
Number of Models Amortized Over
Linear Model: In this domain, the directionality of predictions is very important. As we de- scribe in Section 2.1 predicting values higher than the true utilities for the B resources with highest utility, does not change the decision. However, that of the B 1hh best resource, the decision quality is affected. As a result, we see that the "Directed" methods perform significantly better than their competition.
their predicted utility is lower than
2. Web Advertising: this domain, Wilder et al. [25] suggest that the decision quality is linked to being able predict the quantity Σ, Vii (the sum of CTRs across all users for given website i). However, because the input features for every Di the same E; the errors can be correlated and so the sum can be biased. As a result, the ability to penalize the correlations between two predictions ūæ and Ūyı is important to being able to perform well on this task which results in the Quadratic methods outperforming the others.
3. Portfolio Optimization: Given that this stress-test domain was built to be favorable to DFL, it outperforms all other approaches with statistical significance. While the directed LODL methods do not outperform DFL, they nonetheless significantly outperform 2-stage
0.05.
We now analyze the results in terms of the methods:
Our DudratiiQuadrattc LODL consistently does well: In addition to consistently high ex- pected values (always better than 2-stage), the associated variance is lower as well.
2. Lack of convexity can cause inconsistent results: While NN does well in the first two domains, it fails catastrophically the Portfolio Optimization domain.
3. DFL has large variance in performance: In both the "Web Advertising" and "Linear Model" domain, DFL has higher variation than the best performing LODL.s. We posit that this is also because of the lack of convexity of the surrogates that DFL uses in these two domains.
5.2 Ablations
We study the impact of the sampling strategy and number of samples on the performance of the LODL methods in the Web Advertising domain in Table 3 We find:
1. Sampling strategy (Table 3a):
The best sampling strutegy is loss samily-specific. Specifically, NN and BirectedWWighheedMSt perform best with the "2-Perturbed" strategy, while the remain- ing LODLs perform best with the "All-Perturbed" strategy.
2. Number of samples (Table 3b): : All models perform better with more samples. In addition, the variance reduces as the number of samples increases (especially for Quadratic), suggesting that better approximations of DL lead to more consistent outcomes.
5.3
Computational Cost of Learning with LODLs
We measure the time taken to learn predictive models Me with LODLs in the Web Advertisement domain. We train each LODL for 100 gradient descent steps using 5000 samples and train the predictive model for 500 steps (the same as the setup as Table Dp. We find:
1. Learning LODLs is parallelizable: Figure 3a shows that the cost of training predictive model using LODL. decreases near-linearly in the number of cores used. With more than 20 cores, training with LODLs can be cheaper than training with for this domain.

---

2. If LODLS can be reused, their (already low) overhead quickly diminishes: From Figure 3b we see that even for a modest amount of amortization (over 5-10 predictive models), the training time using LODLs converges to that of two-stage (shown using parallelization with 40 cores).
5.4 Correlation between the 'quality' LODL and decision quality
Recall that LODL losses are fit using a Gaussian sampling strategy centered around the true labels. It is natural ask how well this proxy loss correlates with the decision quality on test data. We do this by measuring the mean absolute error (MAE) of LODL relative to the ground truth decision loss for points in the Gaussian neighborhood around the true labels.
This Gaussian neighborhood is only an approximation of the true distribution of interest-the dis- tribution of predictions generated by the predictive model that is trained using the LODI loss. We can measure the MAE on this distribution, which we call the empirical neighborkood.
Table 2: A table showing the relationship between the quality of the learned loss for different classes of LODLs, and the performance of model trained on said loss. The Empirical Neighborhood MAE is linearly correlated with DQ while the Gaussian Neighborhood MAE is not.
MAE in Gaussian
MAE in Empirical Neighborhood
Normalized DQ
1.0
Web Adverliing
Approach
Neighborhood 10 3)
on Test Data
Nomalssd test DD
10
NN WeightedMSE
0.94 + 0.06
2,22 1.73
0,80 =
0.5
1.04 + 0.00
4.48 + 1.71
0.58 + 0.15
Meighte/WiggteeMMSEE
0.92 + 0.00
5.58 1.64
0.50 100::0.1. 0.13
0.0
Quadratic
0.96 + 0.00
0.86 0.52
0.92 + 0.05
DirectedQuadratic
1.06 + 0.00
1.91 0.79
0.85 - 0.08
0.00
0.02
0.04
0.06 0.08
MAE Emprrcal Naighooumood
Table 2 shows the results for the Budget Allocation domain, while the remaining graphs are in Appendix B33 All methods are able to approximate the DL comparably well in the Gaussian neigh- borhood, but this does not correlate well with decision quality. In contrast, the error on the empirical neighborhood is tightly linearly correlated with decision quality. Furthermore, if we extrapolate the line of best fit to where the MAE is i.e., when there is no discrepancy LODL and DL, we find that the trend predicts the normalized DQ would be 1.
Discussion and Conclusion
Our work proposes a conceptual shift from hand-crafting surrogate losses for decision problems to automatically learning them, and demonstrates experimentally that the LODL paradigm enables us to learn high-quality models without such manual effort. Nevertheless, our current instantiation of this framework has limitations which are areas for future work.
We considered LODLs ss that additively decompose across the dimensions of y. allowing to isolate the effects fit to DL from the generalization performance across the dimensions of y. Future work may learn models that generalize across dimensions, allowing for even greater scalability.
We demonstrate that the of a LODL to the empirical neighborhood around the ground truth label is highly correlated with the eventual decision quality. While the Gaussian neighborhood method does yield models that perform well, it does not correlate well with the decision quality across LODL parameterizations. It would be valuable to study the empirical neighborhood to better understand reasons for this discrepancy and potentially develop LODLs with even stronger performance.
In summary, LODL provides an alternate framework for machine learning which informs decision making, opening up new avenues towards models which are both high- nigh-performing and easily trained.
Acknowledgments and Disclosure of Funding
Research was sponsored by the ARO and was accomplished under Grant Number: W911NF-18-1- 0208. Wilder was supported by the Schmidt Science Fellows program.
10

---

References
[1] Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Advances in Neural Information Processing Systems, 32, 2019.
[2] Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J. Zico Kolter. Differentiable mpe for end-to-end planning and control. In Advances in Neural Information Processing Sys- tems, volume 31, 2018.
[3] Brandon Amos, Vladlen Koltun, and J Zico Kolter. The limited multi-label projection layer. arXiv preprint arXiv: www.i190608777, 2019.
[4] Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Phillipe Vert, and Fran- cis Bach. Learning with differentiable pertubed optimizers. Advances in Neural Information Processing Systems, -09959999999, 2020.
[5] Chris Cameron, Jason Hartford, Taylor Lundy, and Kevin Leyton-Brown, The perils of learning before optimizing. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 2022.
[6] Emir Demirović, Peter Stuckey, James Bailey, Jeffrey Chan, Chris Leckie, Kotagiri Ramamo- hanarao, and Tias Guns. An investigation into prediction+ optimisation for the knapsack prob- lem. In International Conference Integration Constraint Programming, Artificial Intelli- gence, and Operations Research, pages 241 Springer, 2019.
[7] Priya Donti, Brandon Amos, and Zico Kolter. Task-based end-to-end model learning in stochastic optimization. Advances Neural Information Processing Systems, 30, 2017.
[8] Adam N Elmachtoub and Paul Grigas. Smart "predict, then optimize". Management Science, 2021.
[9] Aaron Ferber, Bryan Wilder, Bistra Dilkina, and Milind Tambe. MIPaaL: Mixed integer pro- gram as a layer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1504-1511,202
[10] Joseph Futoma, Michael Hughes, and Finale Doshi-Velez. POPCORN:
Partially observed prediction constrained reinforcement learning. In International Conference on Artificial Intel- ligence and Statistics, pages 3578-3588. PMLR, 2020.
[11] Ali Ugur Guler, Emir Demirovic, Jeffrey Chan, James Bailey, Christopher Leckie, and Peter J Stuckey. Divide and learn: A divide and conquer approach for predict+ optimize. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 2022.
[12] Michael Hughes, Gabriel Hope, Leah Weiner, Thomas McCoy, Roy Perlis, Erik Sudderth, and Finale Doshi-Velez. Semi-supervised cnediction-constrinne topic models. In Proceedings of the Twenty- First International Conference on Artificial Intelligence and Statistics, pages 1067-1076, 2018
[13] Folasade Olubusola Isinkaye, Yetunde O Folajimi, and Bolande Adefowoke Ojokoh. Recom- mendation systems: Principles, methods and evaluation. Egyptian Informatics Journal, 16(3): 261-273. 2015.
[14] Debarun Kar, Benjamin Ford, Shahrzad Gholami, Fei Fang. Andrew Plumptre, Milind Tambe, Margaret Driciru, Fred Wanyama, Aggrey Rwetsiba, Mustapha Nsubaga, and Joshua Mabonga. Cloudy with chance of poaching: Adversary behavior modeling and forecasting with real-world poaching data. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, page 159-167, 2017.
[15] Jayanta Mandi and Tias Guns. Interior point solving for LP- prediction-oppimisttion. Advances in Neural Information Processing Systems, 33:7272-7282, 2020.
[16] Jayanta Mandi, Peter J Stuckey, Tias Guns, et al. Smart predict-and-optinie for hard combina- torial optimization problems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1603-1610, 2020.
11

---

[17] Harry M Markowitz and G Peter Todd. Mean-variance analysis in portfolio choice and capital markets. John Wiley & Sons, 2000.
[18] Arthur Mensch and Mathieu Blondel. Differentiable dynamic programming for structured prediction and attention. In International Conference Machine Learning, pages 3462-3471. PMLR, 2018.
[19] Richard Michaud. The Markowitz optimization enigma: Is 'optimized" optimal? Financial Analysts Journal, 45(1):31-42, 1989.
[20] Maxime Mulamba, Jayanta Mandi, Michelangelo Diligenti, Michele Lombardi, Victor Bu- carey, and Tias Guns. Contrastive losses and solution caching for Proceedings the International Joint Conferences Artificial Intelligence, 2021.
In
[21] Sanjana Narayanan, Abhishek Sharma, Catherine Zeng. and Finale Dosei-Velez. Prediction- focused mixture models. arXiv preprint arXiv:21 10, 13221, 2021.
[22] Quandl. WIKI various end-of-day data, 2022. URL httthtttp://www www.dl wwt.nnn...om/aaaaa//aaaaa
[23] Sebastian Tschiatschek, Aytune Sahin, and Andreas Krause. Differenniable submodular maxi- mization. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2731-2738, 2018.
[24] Kai Wang. Bryan Wilder, Andrew Perrault, and Milind Tambe. Automatically learning com- pact quality-aware surrogates for optimization problems. Advances Neural Information Processing Systems, 33:9586-9596, 2020.
[25] Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In Proceedings of the AAAI Con- ference Artificial Intelligence, volume 33, pages 1658-1665, 2019.
[26] Bryan Wilder, Eric Ewing, Bistra Dilkina, and Milind Tambe. End to end learning and op- timization on graphs. Advances in Neural Information Processing Systems, 32:4672-4683, 2019.
[27] Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, and Tomas Pfister. Differentiable top-k with optimal transport. Advances in Neural Information Processing Systems, 33:20520-20531 2020.
[28] Yahoo! Webscope dataset, 2007. URL https://webscope wwndeox..aaooooooooeeooo yahooo com/ ydata-
ysm-advertiser--dds--1..
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No]. or [N/A]. You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ??.
Did you include the license the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
12

---

(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL.)? [Yes] As supple- mental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In the appendix
(c) Did you report error bars (c.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] In all tables.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In the appendix
4. If you using existing assets (e.g., code, data, models) curating/releusing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No] It isn't clear exactly what the license
is, but all the datasets are publicly available.
(c) Did you include any new assets either in the supplemental material or as a URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you'l re using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [No]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "rcOMSCpWM-", "reviewer": "Reviewer_dkZJ", "summary": "This paper considers learning losses for predictive models that\nare used in a downstream optimization task.\nSection 2 summarizes the basic setup where there is a predictive\nmodel $\\hat y=M_\\theta(x)$ that creates predictions used to\nparameterize an optimization process.\nThe baselines considered are 1) 2-stage learning, which trains\nthe predictive model with an intermediate loss, and\n2) decision-focused learning, which seeks to optimize a decision loss\nwith the predictive model, defined with the objective\nof the optimization problem.\nThe locally-optimized decision losses (LODL) proposed in this paper\nseek to learn the parameters of surrogate intermediate losses\nto match the decision loss.", "questions": "I would be very willing to re-evaluate my assessment after a discussion\naround the following questions on weaknesses I've listed above.\n\n1. Can you clarify how the LODL loss on line 191 should be interpreted? Did you consider alternatives to this?\n2. Can you comment on how the experimental results connect to\n   established experimental results in DFL settings?", "limitations": "The paper does not clearly discuss limitations in a dedicated section.\nWhile parameterizing and learning an intermediate loss seems appealing,\nit seems limited by needing to specify and learn the right\nparameterization.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The idea of parameterizing surrogate/intermediate losses makes a lot\n  of sense in these two-stage settings, and the formulation considered\n  here nicely shows the benefits of learning non-Euclidean regression losses.\n  I can imagine learned losses to be a crucial,\n  impactful, and long-lasting contribution in these settings.\n+ The experimental results clearly demonstrate that the LODL is\n  able to learn a reasonable surrogate for the tasks considered\n\nWeaknesses:\n+ If we take the LODL to be the MSE loss parameterized by weights on\n  each dimension, I do not understand the objective on L191:\n  my interpretation is that it tries to make the weighted\n  MSE loss match the decision loss around the optimal prediction\n  by changing the weights.\n  Since the MSE loss and decision loss are very different quantities,\n  it like this objective will not be possible to optimize.\n+ Even though the paper experimentally shows that LODL works, the\n  results are difficult to contextualize and compare to related research.\n  For example, the web advertising task starting\n  at L222 takes one of the settings from\n  [Wilder et al., (2019)](https://ojs.aaai.org/index.php/AAAI/article/view/3982),\n  but does not provide or present the results in a way that is\n  comparable to the results in that paper. The submission would be\n  significantly easier to evaluate in comparison to this work\n  if it reproduces exactly Table 1 of Wilder et al., (2019) and\n  adds additional lines showing how well LODL performs in\n  comparison.\n\nRelated work:\n+ It could be interesting to connect the work to learned losses\n  used in meta-learning, such as in\n  [Bechtle et al. (2021)](https://arxiv.org/pdf/1906.05374.pdf),\n  which takes a much more black-box perspective to learning a\n  latent loss function.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "pxgQ3TLdWR", "reviewer": "Reviewer_4d1n", "summary": "This paper proposes methods to approximate the decision-focused loss function that quantifies the quality of a prediction function by the quality of its induced decision. The proposed method considers several classes of locally parameterized loss functions at each label value in the sample. These loss functions are convex functions of the prediction. The parameters in the loss function for each label value in the sample are estimated by minimizing the loss approximation error at randomly sampled prediction values around the corresponding label truth. ", "questions": "1. The proposed method learns the loss function parameters using samples generated by perturbing the label values (e.g., by adding gaussian noises). The magnitude of perturbation is obviously a very important tuning parameter. But how should this tuning parameter be selected? The selection of this tuning parameter is not discussed at all. Even the experiment section does not mention this. \n\n2. In the WeightedMSE and DirectedWeightedMSE losses, are the weights required to be nonnegative? Line 170 to 171 suggest that the weights are free parameters without any constraints. But negative weights along some dimensions mean that large prediction errors on those dimensions can actually decrease the loss, which doesn't seem desirable. I think negative weights may also encourage predictions that are far away from the labels in the training sets, where the loss function is least accurately estimated. This may also cause problems for the idea of using locally perturbed samples. \n\n3. The proposed approach needs to learn parameters for each label value separately. So if there are N training data points and we sample K local data points for each label value, then computing the DL loss function would require solving KN optimization problems. Isn't this very challenging for even moderately large-scale problems? Unfortunately, the computation limitation is not explicitly discussed in this paper and the problem sizes in the experiment section aren't clear about this either. For example, the perturbation process, the local sample size $K$ for the results in Table 1, and the running time information are not provided. \n\nSome other comments:\n- Is the $\\hat y^\\top Q\\hat y$ term in portfolio optimization a type? \n- The uses of the letters $n, N$ are confusing. From the equation below the line (77), $n$ refers to training data indices, and $N$ refers to training data sample size. However, in Section 5, $N$ seems to refer to the dimension of the uncertain variables, e.g.,  the number of resources or the number of stocks. \n- The processes of generating data based on real datasets in web advertising and portfolio optimization experiments are not clearly described. \n- Figure 2 seems interesting but I do not totally get the reason for the observed patterns. Can the authors provide some explanations? \n- The authors may hope to cite some additional literature on decision-focused learning, such as \n  - Elmachtoub, Adam, Jason Cheuk Nam Liang, and Ryan McNellis. \"Decision trees for decision-making under the predict-then-optimize framework.\" International Conference on Machine Learning. PMLR, 2020.\n  - Kallus, Nathan, and Xiaojie Mao. \"Stochastic optimization forests.\" Management Science (2022).\n  - Hu, Yichun, Nathan Kallus, and Xiaojie Mao. \"Fast rates for contextual linear optimization.\" Management Science (2022). \n  - Grigas, Paul, and Meng Qi. \"Integrated conditional estimation-optimization.\" arXiv preprint arXiv:2110.12351 (2021).\n  - Qi, Meng, et al. \"A practical end-to-end inventory management model with deep learning.\" Available at SSRN 3737780 (2020).", "limitations": "See the comments in the box above. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: The idea of assigning different parameters to the loss function at different label values appears novel. This is an interesting idea to improve the expressive power of the loss function class that builds on relatively simple parametric losses (to guarantee convexity). \n\nQuality: The writing quality of this paper is overall very good and the proposed idea in this paper is well executed. But some important limitations of the proposed approach (e.g., computation and scalability) lack discussion. \n\nClarity: I found this paper is overall well-written and the high-level idea of this paper is easy to grasp. One exception is that the experiment section seems to lack some important details. \n\nSignificance: Although this paper proposes an interesting idea, I am somewhat skeptical of its practicability considering the challenges in parameter tuning and the computational scalability of the proposed method. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "h3kg09N_nYk", "reviewer": "Reviewer_TJiM", "summary": "This paper proposes LODL as a surrogate to replace the original optimization loss while approximately providing gradient information of the original optimization loss. The key argument is that the gradient information may be difficult to obtain when solving complex optimization problems, such as non-convex optimization. In LODL, a surrogate loss is constructed based on variants of MSE/quadratic functions over a dataset sampled around each training sample, such that it approximates the original optimization loss and is easily differentiable.", "questions": "- The computational complexity of LODL is high. It's true that we reduce the dimensionality (Line 126), but the value of $N$ can be very large (which is typically the case in decision-focused learning where the number of training samples is large). Also, we have to learn a surrogate to approximate DL for each training sample. The total complexity will be way much higher than existing methods.\n\n- LODL is still an approximation-based method to differentiate the optimizer. There's no analytical/theoretical evidence to show that LODL can approximate the gradient of the original optimizer with respect to the predictions with sufficiently high accuracy. LODL is actually approximating DL, but this doesn't mean the gradients of DL with respect to the prediction $y$ is still well approximated by the gradient of LODL. One can easily construct counter examples in which two functions have similar values but dramatically different gradients in a neighborhood.\n\n- The way to sample $y$ to train LODL and learn $\\phi_n$ around each training sample $y_n$ is questionable. The samples are randomly generated based on a prior distribution in stage 1, but in stage 2 of learning the prediction model, the output --- predictions --- can follow a different distribution than the assumed prior distribution in stage 1. A direct consequence is that $\\phi_n$ may not be accurate for the new distribution of predictions in stage 2, raising further concerns with the use of pre-trained $\\phi_n$. This is also against the core idea of decision-focused learning where we want to learn the predictions by considering the entire decision pipeline as a single process.", "limitations": "Yes.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ Constructing surrogates to obtain the gradients of the downstream optimization with respect to the predictions is important for decision-focused learning.\n\nCons:\n- The computational complexity of LODL is high. It's true that we reduce the dimensionality (Line 126), but the value of $N$ can be very large (which is typically the case in decision-focused learning where the number of training samples is large). Also, we have to learn a surrogate to approximate DL for each training sample. The total complexity will be way much higher than existing methods.\n\n- LODL is still an approximation-based method to differentiate the optimizer. There's no analytical/theoretical evidence to show that LODL can approximate the gradient of the original optimizer with respect to the predictions with sufficiently high accuracy. LODL is actually approximating DL, but this doesn't mean the gradients of DL with respect to the prediction $y$ is still well approximated by the gradient of LODL. One can easily construct counter examples in which two functions have similar values but dramatically different gradients in a neighborhood.\n\n- The way to sample $y$ to train LODL and learn $\\phi_n$ around each training sample $y_n$ is questionable. The samples are randomly generated based on a prior distribution in stage 1, but in stage 2 of learning the prediction model, the output --- predictions --- can follow a different distribution than the assumed prior distribution in stage 1. A direct consequence is that $\\phi_n$ may not be accurate for the new distribution of predictions in stage 2, raising further concerns with the use of pre-trained $\\phi_n$. This is also against the core idea of decision-focused learning where we want to learn the predictions by considering the entire decision pipeline as a single process.\n\n- Some analysis of LODL would be useful, e.g., sampling complexity and generalization bounds.\n\n- The authors are suggested to highlight the targeted scenario of LODL rather than general decision-focused learning. For example, in convex problems, we can efficiently and accurately differentiate the optimizer with respect to predictions, and so LODL is not needed or advantageous. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5CFbirRWnup", "reviewer": "Reviewer_NGrH", "summary": "This paper proposes a novel locally optimized decision loss (LODL) for decision focused learning (DFL). The LODL loss is a parameterized function trained with the true decision loss as the target. The LODL loss is a relatively faithful measure of the decision quality, and can provide informative gradient for the DFL training.  The authors run the experiments on various optimization tasks including Linear Model, Web Advertising, and Portfolio Optimization, verifying the effectiveness of LODL. Also, the authors product some ablation studies and shows how well the LODL represents the decision quality.", "questions": "1. The design of LODL is interesting, but I have a concern about the complexity of fitting the local loss function.  The design fits one loss function for each sample by MSE. When the prediction space is large and needs a large number of samples, the training complexity would be very high.  The authors may discuss more about the training complexity.\n\n2. The authors show the fitting quality of LODL by measuring the mean absolute error (MAE). From the results, we also know that the LODL may not easily fit some kinds of task loss for some inputs. It would be interesting to have some study about the relationship between the fitting quality, the gradient, and the task loss or different inputs. This may give more observations supporting the claims.", "limitations": "The limitations of the design is clearly listed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Originality \nThe idea of training a parameterized function to approximate the decision loss is interesting. Comparing with the MSE loss in 2-stage method, the LODL is more faithful to the decision quality. Comparing with the surrogate loss in DFL, the LODL can be easier to design. \n\n### Significance\nThe idea is novel for DFL.  \n\n### Quality\nThe proposed method is compared with multiple baselines including  2-stage, DFL, and the LODL by NN on various resource allocation problems, which verifies the advantages of LODL for some problems.\n\n### Clarity\nThe paper is well-written and easy to follow.  \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept