A UTOMATIC L OSS F UNCTION S EARCH FOR P REDICT -T HEN -O PTIMIZE P ROBLEMS WITH S TRONG R ANKING P ROPERTY
Anonymous authors
Paper under double-blind review
A BSTRACT
Combinatorial optimization problems with parameters to be predicted from side information are commonly seen in a variety of problems during the paradigm shifts from reactive decision making to proactive decision making.
Due to the misalignment between the continuous prediction results and the discrete decisions in optimization problems, it is hard to achieve a satisfactory prediction result with the ordinary l 2 loss in the prediction phase.
To properly connect the prediction loss with the optimization goal, in this paper we propose a total group preorder (TGP) loss and its differential version called approximated total group preorder (ATGP) loss for predict-then-optimize (PTO) problems with strong ranking property.
These new losses are provably more robust than the usual l 2 loss in a linear regression setting and have great potential to extend to other settings.
We also propose an automatic searching algorithm that adapts the ATGP loss to PTO problems with different combinatorial structures.
Extensive experiments on the ranking problem, the knapsack problem, and the shortest path problem have demonstrated that our proposed method can achieve a signiﬁcant performance compared to the other methods designed for PTO problems.
I NTRODUCTION
Many decision making processes under uncertainty in real world are solving combinatorial problems with parameters unknown to the decision maker.
A traditional method to address the uncertainty issue is to add assumptions on the distributions of parameters (Hentenryck & Bent, 2006).
Alternatively, a recent approach is to predict the unknown parameters from the correlated features using machine learning methods and solve the combinatorial problems based on the predictions.
This paradigm, which is called predict-then-optimize (PTO), has been widely employed in practice systematically (Elmachtoub & Grigas, 2021).
For example, Google Maps estimates the travel times of roads in the trafﬁc to compute a shortest path (Lau, 2020); Computer clusters predict the processing times and resource demands of computation tasks to make good job scheduling on servers (Mao et al., 2016); Hedge funds forecast the return rates of different stocks to optimize their portfolio in the next trading day (Thomson, 2021).
However, the commonly used l 2 loss function ( i.e., l 2 distance between predictions and true values) usually cannot achieve ideal decision results in predict-then-optimize problems (Demirovi´c et al., 2019).
This misalignment between the l 2 loss in prediction and quality of the decision comes from the misalignment between the continuity on the prediction problem and the discontinuity on the structure of the combinatorial optimization problem.
A straightforward remedy of the above misalignment problem is to adjust the loss to reﬂect the difference in objective values of the optimization solutions generated using the predicted and observed parameters, called Smart “Predict, then Optimize” (SPO) loss (Elmachtoub & Grigas, 2021).
However, due to the combinatorial nature of the optimization problems, the SPO loss is usually piecewise ﬂat and has multiple discontinued points.
As a result, the derivatives of SPO loss is either zero or nonexistent, which prohibits the training of gradient-based deep learning algorithms.
Besides, most solvers for combinatorial optimization problems are not differentiable, which cannot be directly incorporated with the widely adopted gradient-based learning approaches nowadays.
[Figure]
Current approaches to minimizing the SPO loss can be categorized into two classes.
The ﬁrst one is to differentiate the discrete optimization solver by approximating the objective with a certain family of functions, and this series of methods works when the combinatorial optimization problem is linear (Wilder et al., 2019a;b; Poganˇci´c et al., 2019).
The other class of research tries to propose new surrogate loss functions for SPO loss, e.g., SPO+ loss for linear programming (Elmachtoub &
Grigas, 2021) and piecewise linear loss for dynamic programming (Stuckey et al., 2020).
However, these surrogate loss functions are problem-speciﬁc ( i.e., depending on the type of the combinatorial optimization problem) and require much domain knowledge from experts to design.
Instead of designing the surrogate loss metric for each optimization problem manually, an alternative approach is to ﬁnd a good loss metric from a predeﬁned search space in an automatic manner, which is inspired by the recent progress in automated machine learning (AutoML) (Zoph & Le, 2017; Pham et al., 2018;
Liu et al., 2018).
These AutoML approaches usually deﬁne an appropriate search space and then conduct some search algorithms, e.g., reinforcement learning and genetic algorithm, to ﬁnd a good metric.
Li et al.
and
Li et al.
(2021) studied the automatic loss function search problem for computer vision tasks.
They deﬁned the search space by replacing the mainstream metrics for semantic segmentation with their differentiable approximations.
However for predict-then-optimize problems, there is no such evaluation metrics available except for SPO loss.
Moreover, these metrics, which are designed for segmentation tasks in computer vision domain, are not suitable to evaluate the prediction results for the combinatorial optimization problems.
In this paper, we propose a framework of automatic loss function search called APOC (Automatic
Prediction and Optimization Connector) to tackle a wide range of predict-then-optimize problems whose optimal solution is determined by the total group preorder of the parameters of the combinatorial problem, called strong ranking property (see for Deﬁnition 2 for details.).
These problems have the same optimal solution for different sets of parameters, as long as those sets of parameters preserve the same total group preorders.
The key idea is to build the proper search space in which different loss functions capture the partial comparisons between different groups of the parameters of the optimization problem.
Our contributions are summarized as follows: 1) We theoretically prove that l 2 loss is not an ideal choice in a linear regression setting of PTO problems; 2) We propose the total group preorder loss for PTO problems and relax it to a differentiable approximated version, which is ﬁtted using a family of Derivative-of-Guassian wavelet functions to capture the group comparison relationship among the items with variable sizes and weights; 3)
We propose an effective method to automatically search for a differentiable surrogate loss based on approximated group total preorders of items for
PTO problems called APOC; 4) The proposed APOC method has been validated on three classic combinatorial problems with strong ranking property.
R ELATED W ORK
Predict-then-optimize (PTO) problems capture a common pipeline of machine-learning-assisted optimization solvers: predicting the unknown parameters of the optimization problem from the contextual information and then optimizing based on the predicted parameters (Elmachtoub & Grigas, 2021).
However, many combinatorial optimization problems have piecewise ﬂat value function landscapes which prohibit an end-to-end learning with gradient-based prediction models.
One line of work is to differentiate the optimization solvers for speciﬁc problems and embed them as layers into the network architecture through which the gradients can be propagated, which originates from the differentiable optimization (Amos, 2019).
Grover et al.
(2018) proposes a sorting network for ranking problems by relaxing the permutation matrix output by sorting algorithms to the unimodal row-stochastic matrix.
For the node clustering problem, Wilder et al.
(2019b) differentiates the K-means algorithm by assigning nodes to clusters according to a soft-min function.
Poganˇci´c et al.
(2019) interpolates the value of function of linear programming using piecewise linear functions and embeds as layers the solvers of linear programming.
The other line of work focuses on searching for a surrogate loss function to approximate the value function of the optimization problem, so-called regret in decision theory (Bell, 1982).
Elmachtoub & Grigas (2021) derives a convex upper bound on the regret of linear programming, called SPO+ loss.
Wilder et al. (2019a) proposes the QTPL loss by adding a quadratic penalty term to the continuous relaxation of the regret such that results from differentiating over quadratic programs can be used.
Mandi & Guns (2020) overcomes the non-differentiablity of the regret by adding a logarithmic barrier term, called IntOpt.
Stuckey et al.
(2020) approximates the value function of dynamic programming problems as piecewise linear functions with learn-able parameters.
Yoon et al.
(2013) proposes mean objective cost of uncertainty (MOCU), the difference between the value functions of a robust solution and the optimal solution, to evaluate the uncertainty of parameters in decision making, e.g., experiment design (Boluki et al., 2018) and active learning (Zhao et al., 2021).
Loss function guides the machine learning algorithms to produce good predictions for different tasks, which is usually designed with domain knowledge from experts (Masnadi-Shirazi & Vasconcelos, 2008; Bruch et al., 2019).
Automatic search of suitable loss function without domain knowledge has recently received much attention from the computer vision community.
Li et al.
(2019) uses reinforcement learning algorithms to learn better loss functions with good generalization and transfer-ability on different vision tasks.
Wang et al.
(2020) adopts both random search and reinforcement learning algorithms to search for loss functions on face recognition problems.
Li et al.
(2021) explores the possibility of searching loss function automatically from scratch for generic tasks, e.g., semantic segmentation, object detection, and pose estimation.
However, the search spaces in these methods are specially designed for vision tasks and cannot be directly applied to PTO problems.
Most work on automatic loss search follows the searching algorithms used in AutoML.
We refer to He et al.
(2021) for comprehensive survey on searching methods in AutoML.
Natural questions are: 1) whether we can design a suitable search space for loss functions in PTO problems; and 2) whether there is a strategy to search adequate loss functions systematically for different optimization problems using techniques from AutoML.
ATGP LOSS FOR PREDICT - THEN - OPTIMIZE PROBLEMS 3. M ISALIGNMENT IN P REDICT - THEN - OPTIMIZE PROBLEM
The problems we consider are the type of predict-then-optimize (PTO) problems in the following formulation.
For an optimization problem in the following form
[Equation]
where z is the decision variable, c ∈ R d is the parameter of the objective function U , and Z is the feasible set which does not depend on c .
A decision maker needs to solve (1) without knowing the exact value of c .
Instead, the decision maker will observe a feature vector x ∈ R k .
The goal of the decision maker is to predict the value of c and then optimize (1) based on the prediction ˆ c .
To distinguish with the parameters of the prediction models ( e.g., weights of neural networks), we call c the PTO parameter.
Instead of focusing on the prediction error between c and ˆ c ( e.g., l 2 distance), the decision maker cares more about a high-quality decision result generated from the prediction.
Elmachtoub & Grigas (2021) characterizes this quantity by the Smart “Predict, then Optimize” (SPO) loss, which equals to the loss incurred by solving the optimization problem (1) using ˆ c instead of the true parameter c .
[Equation]
where z ∗ ( c ) = arg max z ∈
Z U c ( z ) and U ∗ c
= U c ( z ∗ ( c )) .
c 2 2 3 4 5 6 7 8 c 1 2 3 4 5 6 7 8 0.
(a) SPO (blue) vs. l 2 (green).
c
2 2 3 4 5 6 7 8 c 1 2 3 4 5 6 7 8 0.
(b) SPO (blue) vs. TGP (red).
c
2 2 3 4 5 6 7 8 c 1 2 3 4 5 6 7 8 0.
(c) SPO (blue) vs. ATGP (purple).
Figure 2a shows the misalignment between the l 2 distance green and the SPO loss blue in a ranking PTO problem.
In this ranking PTO problem, there are two items to be ranked.
The true values of the items are c 1 = 4 .
9 and c 2 = 5 . 1 , which are also the values to predict.
The prediction denoted by the magenta triangle has a lower SPO loss than the orange one, and therefore yields a better solution.
However, prediction model that minimizes the l 2 distance will prefer the orange one (whose l 2 distance is lower) and produce a worse solution with regard to the ranking task.
However, the SPO loss of most PTO problems is a piecewise linear function due to the combinatorial structure in the objective function U c ( z ) and the solution map z ∗ ( c ) .
As a result, its derivatives in the predicted PTO parameter ˆ c is either zero or nonexistent, which prohibits the usage of prediction models whose training is based on gradient descent algorithms.
In this paper, we consider a collection of combinatorial optimization problems with strong ranking property .
For this collection of optimization problems, we propose the total group preorder (TGP) loss and its differentiable variant approximated total group preorder (ATGP) loss as the training targets in the prediction stage.
The ATGP loss will approximate the SPO loss with smooth shape, which is friendly and effective for the gradient-based training manner, as is shown in Figure 2c.
3. S TRONG RANKING PROPERTY AND (A)TGP LOSS
The strong ranking property is a stronger version of the ranking property proposed by Demirovi´c et al.
The ranking property from Demirovi´c et al.
(2019) requires that two instances of a combinatorial optimization problem have the same solution if the pairwise orders of the parameters of two instances are the same, which captures the importance of total preorder to the optimality of the ranking problems.
However, the total preorder is not sufﬁcient to ﬁnd an optimal solution for many combinatorial problems with group-wise comparisons in nature, e.g., knapsack problem with ﬁxed item weights 1 and shortest path problem.
To extend this ranking property to these problems, we propose a stronger version of ranking property which takes the group-wise preorder into consideration.
Deﬁnition 1 (Total group preorder) .
For a combinatorial optimization problem with parameter c = ( c 1 , . . .
, c d ) , the total group preorder TGP( c ) =
(cid:110)
( A, B ) : non-empty subsets A, B ⊂ { 1 , . . .
, d } and A ∩ B = ∅ such that (cid:80) i ∈
A c i ≥ (cid:80) j ∈ B c j (cid:111) .
For a ranking problems of 3 items with true weights c 1 = 1 , c 2 = 2 , c 3 = 3 , then the total group preorder is TGP( c ) =
For many combinatorial optimization problems, it sufﬁces to know the total group preorder of the parameters to yield the optimal solution.
Deﬁnition 2 (Strong ranking property) .
A combinatorial optimization problem satisﬁes strong ranking property if for any two instances P and P (cid:48) with parameters c and c (cid:48) respectively, TGP ( c ) =
TGP ( c (cid:48) )
= ⇒ P and P (cid:48) have the same solution.
The strong ranking property indicates that the prediction algorithm only needs to learn the preorders between different sets of groups in PTO parameters.
Inspired by the strong ranking property, we propose the TGP loss function which captures the difference between the TGP under predicted PTO parameter ˆ c and that under the true parameter c .
Deﬁnition 3 (TGP loss) .
[Equation]
where sign( x ) is a point-wise sign function and L is a matrix whose elements can be any real numbers, called TGP matrix.
To capture the complete TGP, we need L to be (cid:0) 2 d 2 (cid:1) ×
d which needs exponentially growing memory as the number of PTO parameter d increases.
However, this can be avoided by noticing that the optimal solutions of different optimization problems are determined by different subset of its TGP, e.g., { ( { 3 } , { 1 } ), ( { 3 } , { 2 } ), ( { 2 } , { 1 } ) } for the ranking problem mentioned above.
Instead, we ﬁx the number of rows in L and let each row of L act as a linear ﬁlter that extracts out a group-wise comparison within c so that each L represents a subset of TGP; as a result, different optimization problems will have different TGP matrices.
Note that TGP loss is not differentiable because the sign function on ˆ c is not differentiable, we derive a differentiable version of it by replacing the sign function with hyperbolic tangent function, called approximated total group preorder (ATGP) loss.
Deﬁnition 4 (ATGP loss) .
[Equation]
where γ is a temperature hyperparameter that controls the degree that the TGP loss is approximated to and tanh is a point-wise hyperbolic tangent function.
Lemma 1.
For any ˆ c, c ∈ R d and TGP matrix L , we have
[Equation]
Here the hyperbolic tangent function is used to approximate the sign function with a differentiable functional form.
Figure 2c visualizes the ATGP loss for the ranking PTO problem with TGP matrix
L = [1 , − 1] and γ = 0 .
3. R OBUSTNESS OF ATGP LOSS
Despite of the difﬁculty on theoretically proving the non-optimum of the commonly used l 2 loss in the prediction stage, we can prove in a simple setting with linear regression as the prediction algorithm that the ATGP loss is generally better than the traditional l 2 loss for the ranking problem.
Speciﬁcally, ATGP loss is more robust to noise added to the observation of the PTO parameters in the training data set.
Learning to rank with linear regression Suppose there is a set of training examples (cid:8)(cid:0)
x ( i ) , c
( i ) (cid:1) : i ∈ S (cid:9) in the training set S identically and independently sampled from some distribution D .
Each example has a feature vector x ( i ) ∈
R k and a vector c ( i ) ∈ R d of response variables.
Here the response variables are the values of d items that the decision maker aims to rank.
The decision maker predicts the value of items using a linear regression model, i.e., compute a matrix H ∈ R d × k such that for a new pair ( x, c ) ∼ D , the linear estimator Hx predicts a score for each response variable in c given x .
The decision maker will rank the items according to the predicted values Hx .
An example is routing recommendation in which the feature vector x encodes the context information like weather, time, temperature and c ∈ R d is the congestion levels of the d routes within a map.
The goal is to ﬁnd the n routes with the least congestion levels.
Let the ground truth linear transformation from the feature points to response variables be H ∗ , and the ground truth response variables be ¯ c ( i )
=
H ∗ x ( i ) .
The noisy response variables that we actually observe are c ( i ) =
¯ c ( i ) + ε ( i ) where ε ( i ) is the noise i.i.d.
over each sample i .
For a loss function (cid:96) , we denote its empirical risk minimizing estimator ˆ H (cid:96) =
arg min H (cid:80) i ∈ S (cid:96) ( Hx ( i ) , c ( i ) ) , so the predicted values given feature x
( i ) would be ˆ c
( i ) (cid:96) =
ˆ H (cid:96) x .
We deﬁne the robustness of a loss function as the minimum level of noise such that there exists at least one sample where the predicted values have different total group preorder as the ground truth values, which intuitively stands for the minimum efforts to “corrupt” the system.
Deﬁnition 5 (Robustness) .
The robustness R ( (cid:96) ) of a loss function (cid:96) is deﬁned to be
[Equation]
We show that as long as the sample that’s most easily corrupted does not share the same lowdimensional subspace of other samples, the robustness of ATGP loss function is no lower than the l 2 loss.
Theorem 1.
Let i ∗ be the index of the ground truth optimization parameter which is most sensitive to noise perturbation, i.e., inf ε {(cid:107) ε (cid:107) 22 :
[Equation]
(¯ c ( j ) + ε ) (cid:54) =
TGP(¯ c ( j ) ) } , ∀ j ∈ S .
If the linear space expanded by all feature vectors except i ∗ has low rank structure (i.e., rank (cid:0) { x ( i ) : i ∈ S/
{ i ∗ }} (cid:1) < k ) and x ( i ∗ ) doesn’t belong to this space, then there exists a TGP matrix L such that
[Equation]
The choice of TGP matrix L depends on the speciﬁc type of the optimization problem U .
A TGP matrix suitable for top- n ranking problem may perform badly on the knapsack problem and shortest path problem.
To solve this, we propose a searching algorithm to automatically ﬁnd the suitable
TGP matrices for different optimization problems with strong ranking property.
A DAPTIVE ATGP L OSS F UNCTION S EARCH WITH APOC
To search for the proper ATGP loss functions for different optimization problems, we propose a search algorithm called APOC (Automatic Prediction and Optimization Connector), following the spirit of AutoML (Zoph & Le, 2017; Pham et al., 2018;
Li et al., 2021).
To make the size of the searching space for the TGP matrix stable and preserve the ﬂexibility of comparing items, we parameterize the rows of TGP matrix with a set of discretized Derivative-of-Gaussian (DoG) ﬁlters, which are wavelet ﬁlters widely used for edge detection in computer vision (Kennedy & Basu, 1999;
Zhang et al., 2020).
The wavelet ﬁlters can well capture the weighted group-wise comparison among the items with variable group size and are updated based on the prediction quality induced by them.
4. S EARCH SPACE AND WAVELET FILTER
Recall that for the PTO parameter c , we want tanh( γLc )
≈ sign( Lc ) to approximately capture different group-wise comparisons between the entries of c .
To this end, we parameterize the TGP matrix L by a set of sliding discretized Derivatives-of-Gaussian (DoG) ﬁlters.
Each ﬁlter has a reception ﬁeld which will select a subset of entries of c , and assigns weights to the selected entries in the reception ﬁeld, as is shown in Figure 3.
The inner product between the selected entries in the reception ﬁeld and the weights form an entry in Lc .
The reception ﬁeld will slide along c with a stride of 1.
We denote by G ( x, a ) the probability density function of Gaussian distribution with mean zero and standard deviation a , then the weight of ﬁlter i with parameter ( n i , a i ) assigned to the j -th item in the reception ﬁeld is [Equation] r =1 ν 2 ( r [Equation]
n i , a i ) where ν
[Equation]
j
[Equation] n i , a [Equation] Consequently, these sliding ﬁlters can detect different group-wise comparison patterns and encodes them into tanh( γLc ) .
For representing the TGP matrix L , we use θ to denote the parameters ( n 1 , a 1 , . . . , n D , a D ) of all
D DoG ﬁlters used.
With our wavelet ﬁlter representation approach, the search of an adequate TGP matrix over the entire matrix space degenerates to ﬁnding the optimal θ , which saves the computation cost greatly and makes it not necessarily dependent on the size of c .
0 4 9 14 19 24 29 34 39 items in the reception field 0.
w e i g h t s (a) ( n, a ) =
(1 , 1) 0 4 9 14 19 24 29 34 39 items in the reception field 0.
w e i g h t s (b) ( n, a ) =
(3 , 1) 0 4 9 14 19 24 29 34 39 items in the reception field -0.
w e i g h t s (c) ( n, a ) =
4. P ARAMETER OPTIMIZATION
Algorithm 1 shows the details of our approach.
Speciﬁcally, at each round t , we explore the loss function space by creating M trajectories, each trajectory with a set of DoG ﬁlter parameter µ m sampled independently from a truncated normal distribution N truncated ( θ t − 1 , σ 2 I ) .
The truncated normal distribution has support on [0 , + ∞ ) .
We denote the probability density function of µ ∼
N truncated ( θ, σ 2 I ) as π ( µ | θ ) .
For each loss function (cid:96) m , the prediction model is then trained to minimizes (cid:96) m using stochastic gradient descent (SGD) and the learned model f ˆ w makes predictions on the validation set { ˆ c ( i ) : i ∈ V } .
We evaluate the current policy by calculating the negative total
SPO loss of f ˆ w on the validation set R mt = − (cid:80) i ∈ V (cid:96) SPO (ˆ c ( i ) , c ( i ) ) .
The sampling policy can be updated with commonly used reinforcement learning algorithms.
Two classic algorithms are listed here in our work, and other policy updating algorithms are also applicable.
The ﬁrst one we have used is REINFORCE algorithm
(Williams, 1992)
[Equation]
Besides, the PPO2 algorithm (Schulman et al., 2017) has also been applied:
[Equation]
[Table]
[Table] [Table] [Table] [Table]
[Table] i (cid:96) ∗ is the ATGP loss parameterized by θ ∗ . Following the common practice in policy gradient algorithms, we subtract a baseline value from R mt to reduce the variance of the gradient
E XPERIMENTS
The proposed APOC method is applied to three predict-then-optimize problems with real-world datasets to validate its performance over the state-of-the-art methods.
Those three PTO problems contain optimization problems with different representative combinatorial structures, i.e., ranking, knapsack, and shortest path problem.
Empirically, our experimental results demonstrate that significant performance gains can be achieved against some previously proposed algorithms by ﬁnding a better loss for the prediction phase.
5. D ATASET DESCRIPTION
Energy Cost-Aware Scheduling This dataset contains two years of historical energy price data from the day-head market of the Irish Single Electricity Market Operator (SEM-O).
The energy prices for each half-hour of a day need to be predicted with some given feature data (Simonis et al.) including estimates of the temperature, wind speed, CO 2 intensity, etc.
We need to select a subset of 48 half-hours in one day based on the predicted prices to schedule the machines to minimize the total expenditure of the day.
Here we formulate the optimization problem as a top- k ranking problem , where the order of the top- k half-hours in a day determines the minimal total expenditure of the day.
We evaluate algorithms with k ∈ { 5 , 10 , 15 } .
The data set contains 37872 instances.
Knapsack Formulation of Portfolio Optimization We use the historical daily price and volume data from 2004 to 2017 from partition SP500 of the Quandl WIKI dataset (QUANDL, 2020).
The task is to predict the future returns for a set of 505 candidate stocks, and generate daily portfolios based on the predictions.
The optimization problem is formulated as a knapsack problem in which the decision maker needs to choose stocks based on estimated returns to maximize the total expected reward under the ﬁxed budget.
The data set provides 150 instances.
Shortest Path We use the processed data from Mandi & Guns (2020) where the goal is to ﬁnd the shortest path of the given source-destination pairs on a Twitter ego network (McAuley & Leskovec, 2012).
The network consists of 231 nodes and 2861 edges where the edge weights need to be predicted using node and edge features, with 115 instances in total.
5. E XPERIMENTAL SETTINGS
Baseline methods We compare with the follow PTO methods.
l 2 distance: two-stage method with conventional l 2 regression.
QPTL/IntOpt: exerts quadratic/log-barrier regularization on the solution parameters to obtain non-zero gradient for differentiable training.
SPO+: uses continuous convex surrogate of the SPO loss function for sub-gradient based training.
NeuralSort: differentiably relaxing the permutation matrix in sorting problems.
DiffOpt: a recent method which approximates the gradient by the solution difference with perturbed prediction parameters.
The outlines of these methods are discussed in Section 2.
Experiment settings For each experiment, we use the same prediction model architecture and train/validation/test splits across different methods for fair comparisons.
We use Adam optimizer for parameter training across all experiments, where hyperparameters (learning rate & ATGP temperature γ ) are obtained by grid search using the validation performance.
For Energy Cost-Aware
Scheduling, the prediction model is instantiated to be a 4-layer neural network (hidden layer sizes of 500-300 with ReLU as activation function) with learning rate 5 × 10 − 5 and γ = 5 . 0 .
For Shortest
Path, the prediction model is a 4-layer network (hidden layer sizes of 100-100 with ReLU as activation function) with learning rate 5 × 10 − 4 and γ = 3 . 0 .
For Portfolio Optimization, we use a 5-layer neural network (hidden layer sizes of 100-100-100 with ReLU as activation function) with learning rate 5 × 10 − 5 , γ = 3 . 0 .
We use Gurobi optimizer to solve all combinatorial optimization problems and use CVXPY (Diamond & Boyd, 2016; Agrawal et al., 2018) and QPTH (Amos & Kolter, 2017) package for differentiable optimization.
5. R ESULTS
This subsection shows the evaluated performance of our algorithm on the three PTO problems described above.
The goal is three-fold: 1) to demonstrate that our algorithm outperforms the previous proposed algorithms by a signiﬁcant margin; 2) to analyze the sensitivity of our algorithm to the hyperparameters; 3) to validate the effectiveness of our proposed DoG ﬁlters by an ablation study.
Performance Comparison The results in Table 1 show that our proposed APOC algorithm yields signiﬁcantly lower SPO losses than other algorithms on all three PTO problems.
For Energy Cost- [Figure] Aware Scheduling and shortest path problem, the ATGP loss function searched by REINFORCE algorithm induces better predictions while PPO2 algorithm is better on knapsack problem.
[Table]
Hyperparameter sensitivity
In our searching algorithm, there are two key hyperparameters: the temperature parameter γ ∈ (0 , + ∞ ) and the exploration ratio σ ∈ (0 , + ∞ ) .
Each hyperparameter stands for a trade-off in the searching process.
Lower temperature γ gives a smoother ATGP loss which is beneﬁcial to the training of the prediction model but as Lemma 1 states, the ATGP loss with higher temperature γ approximates the true TGP loss closer.
The exploration ratio σ balances a classic exploration-exploitation trade-off in the searching algorithm.
Sampling policy with smaller σ exploits the feedback more from the last step while the policy with larger σ tends to explores a larger area of the parameter space.
We present the inﬂuences of these two hyperparameters to our searching algorithm in Figure 4.
The results show that the proposed method is generally robust and insensitive to these hyperparameters.
Ablation Study To understand the contribution of the DoG wavelet ﬁlter, we conduct an ablation study that compares the performance of our APOC algorithm with the searching algorithm in which the TGP matrix is parameterized by a fully connected dense layer.
As is shown in right panel of
Figure 4, the wavelet ﬁlter yields a signiﬁcant decrease in the SPO loss on the test set.
C ONCLUSION
To tackle the disalignment between prediction loss and optimization goal in PTO problems, we propose to use the differentiable ATGP loss which captures the total group preorder.
A corresponding
APOC method is further proposed to automatically search for a good surrogate loss with wavelet ﬁlters to capture the group-wise comparison of different sizes and weights.
The method is well validated on three different PTO problems and shows great potential for more PTO problems.
Starting from the limitations of this work, the following directions are expected for future work. The ﬁrst one is to extend the scope of APOC framework to PTO problems beyond the ones with strong ranking property. Besides, it is also worth exploring how to reduce the searching time of the iterations within APOC framework.
R EPRODUCIBILITY S TATEMENT
Source code We have made the source code for the proposed APOC available in the following repository: https://github . com/AutoPredOptConnector/APOC .
Proof of Lemma 1 and Theorem 1 We have attached the complete proof of Lemma 1 and Theorem 1 in the Appendix.
A PPENDIX . P ROOF OF L EMMA
Proof.
Notice that the sign function is the derivative of a “V”-shape function: sign( x [Equation] =
[Equation]
x>
[Equation]
[Equation]
<=
[Equation]
arg
[Equation]
0) where arg max( x, 0) =
[1 , 0] (cid:62) if x ≥ 0 and [0 , 1] (cid:62) otherwise.
We can use softmax function to approximate the argmax, i.e., arg max( x, 0) =
lim γ → + ∞ (cid:34) e 2 γx e 2 γx + e 2 γx + (cid:35) and (cid:20) (cid:21) (cid:62) (cid:32)(cid:34) e 2 γx e 2 γx + 1 e 2 γx + (cid:35) − (cid:34) e − 2 γx e − 2 γx + 1 e − 2 γx + (cid:35)(cid:33) =
e 2 γx −e 2 γx += tanh( γx )
Combined all equalities together, we have sign( x ) =
lim γ → + ∞ tanh( γx ) .
. P ROOF OF T HEOREM
Notations.
For a matrix A , A i refers to its i -th row and A : ,i refers to its i -th column.
L ( A ) denotes the column space of A and P A is the orthogonal projection matrix onto L ( A ) .
A + denotes the pseudo-inverse of A ; || A || F = (cid:112)(cid:80)
i ||
A
i || 22 is the Frobenius norm of A .
For convenience of introducing our proof, we describe our data model and assumption in matrix notations.
Let N = | S | be the number of samples, we can row-stack the feature vectors to be matrix
X ∈ R N × k .
Also let the ground truth linear regression parameter matrix be
H ∗ ∈ R k × d ( H ∗ is transposed here compared to the main text since we’re viewing the feature/response variables as row vectors instead of column vectors), so the row-stack matrix of ground truth response values is
C ∗ =
XH ∗ .
We observe C
= C ∗ + E where E is the noise matrix and the noise level is deﬁned to be || E || 2 F .
For a certain loss metric l , the minimizer is denoted by
[Equation]
and the prediction is denoted by
[Equation]
Any speciﬁc optimization problem with [JY]: generalized ranking property corresponds to a subset of its total group pre-orders.
We formalize this with the following deﬁnition.
Deﬁnition 6.
A function f is called a sub-order function if it maps a vector to a subset of its total group pre-orders.
We will also call f ( a ) the sub-orders of a induced by f .
Now a sub-order function f would specify the optimization problem at hand.
Each sub-order function f would also correspond to a TGP matrix whose rows compares the sub-groups induced by f ; we denote this matrix by L f .
Take Ranking problems for example, f ( a ) =
f Ranking ( a ) would map a to the subset of TGP( a ) where the elements ( u 1 , u 2 ) satisfy | u 1 | = | u 2 | = 1 , and L f would be a matrix that maps a vector to another vector of its pairwise differences.
Now for sample matrix A and label matrix B , the two loss metrics whose robustness to noise we aim to compare are:
[Equation]
Given a sub-order function f and a vector a , we deﬁne ∆ f ( a ) to be the minimum perturbation on a to change a ’s sub-orders induced by f ; similarly for a matrix A which consists of a row-stack of samples.
Deﬁnition 7.
For a vector a or a matrix A ,
[Equation]
Let us reformulate the assumption for Theorem 1 with the above notations.
Assumption 1 (Reformulated) .
The feature matrix X has a low rank structure (meaning the samples belong to a linear subspace of less than k -dimension) except for the sample indexed by g which can induce different sub-orders with minimum perturbation, i.e., ∆ f ( C ∗ g ) =
∆ f ( C ∗ ) .
Validity and Intuition of Assumption 1 : it is easy to construct data models which satisﬁes this assumption; details at the end of the document.
This assumption says that the sample which is most likely a outlier in the dataset does not share the low-rank structure of other samples.
Considering the low-rank structure of most real world datasets and the nature of outliers, this assumption is very likely to hold intuitively.
We also believe our conclusion can be extended to non-linear settings where the low-rank linear subspace in this assumption is replaced by low-dimensional manifolds.
Proof for Theorem 1.
We’re now ready to present the proof of Theorem 1, which is an immediate consequence of Lemma 2 and Lemma 3 below.
Lemma 2.
Under Assumption 1, we have
[Equation]
Proof.
For l 2 loss the solution is given by
[Equation]
and thus
[Equation]
Let U ∈ R N × k be a orthogonal basis for L ( X ) , then P X = UU T .
Let a orthogonal basis of the complement of L ( X ) be U ⊥ .
[ U ; U ⊥ ]
= U full then forms an orthonormal basis of R N .
Let a normal vector of the subspace in Assumption 1 be ¯ α ∈ R k and let the index for the special sample be g .
Then ∆ f ( C ∗ ) =
∆ f ( C ∗ g ) and also ¯ α T x i =
for i (cid:54) =
g , ¯ α T x g (cid:54) = 0 .
Now, we can scale ¯ α to α s.t.
α T x g = 1 .
Therefore we now have
[Equation]
where e g is the g -th standard basis (one-hot) vector.
Now e g ∈ L ( X ) =
L ( U ) and thus basis vectors in U ⊥ are all perpendicular to e g .
This means the g -th entry of any basis vector in U ⊥ is 0 .
Since any row of U full has unit norm, the g -th row of U must have unit norm, which also indicates that the g -th row of P X = UU T has unit norm.
Let the g -th row of P X be p ∈ R n .
Now, we can ﬁnd a E s.t.
P X E changes the sub-orders exactly in the g -th row of C ∗ .
The noise level of such a matrix E would then be an upper bound on R ( (cid:96) l 2 ) .
Denote c ∗ =
C ∗ g .
Let ( i, j ) be the pair in f ( c ∗ ) with smallest | c ∗ i − c ∗ j | .
We simply need to ﬁnd E : ,i , E : ,j ∈ R N with minimum || E : ,i ||
+ || E : ,j ||
s.t.
c ∗ i + p T E : ,i =
c ∗ j + p T E : ,j .
It can be easily seen that the optimal solution is given by
[Equation]
and thus the minimum noise level is
[Equation]
by which our proof is done.
Lemma 3.
∆ f ( C ∗ ) ≤ R ( (cid:96) ATGP ) .
Proof.
The proof of this lemma is rather simple; recall that the ATGP loss seeks to ﬁnd H which minimizes (cid:96) ATGP ( XH, C ∗ + E )
=
|| tanh( γ · L f XH ) − sign( L f ( C ∗ + E )) || 2 F . Now as long as E doesn’t change the sub-orders in C ∗ ’s rows, sign( L f ( C ∗ + E ))
= sign( L f C ∗ ) so (cid:96) ATGP remains intact under the noise E , and thus it would still give the optimal solution.
Therefore the minimum noise level which changes the sub-order in at least one of C ∗ ’s rows would always be a lower bound for R ( (cid:96) ATGP ) .
This bound should have a large gap with the best bound since the decision boundary usually shouldn’t change a lot even when we corrupt a portion of the samples.
Here we only corrupt one sample.
Example of constructing data that satisﬁes Assumption 1 .
Generating X which satisﬁes the requirement is straightforward.
Now let H 1 , H 2 ∈ R d be nearly orthogonal vectors of x g .
Randomly generate H 3 , ..., H n and ﬂip their signs s.t.
k − 1 out of them have H Tj x g < 0 the rest have
H
Tj x g > 0 .
This ensures that H T 1 x g , H T 2 x g are the k -th and k + 1 -th biggest items in C ∗ g, : .
By genericity, they are the closest pair in all pairs of entries of C ∗ and therefore ∆ k
( C ∗ ) =
∆ k ( C ∗ g ) .