Automatic Loss Function Search for Predict-Then-Optimize Problems with Strong Ranking Property

1 INTRODUCTION . Many decision making processes under uncertainty in real world are solving combinatorial problems with parameters unknown to the decision maker . A traditional method to address the uncertainty issue is to add assumptions on the distributions of parameters ( Hentenryck & Bent , 2006 ) . Alternatively , a recent approach is to predict the unknown parameters from the correlated features using machine learning methods and solve the combinatorial problems based on the predictions . This paradigm , which is called predict-then-optimize ( PTO ) , has been widely employed in practice systematically ( Elmachtoub & Grigas , 2021 ) . For example , Google Maps estimates the travel times of roads in the traffic to compute a shortest path ( Lau , 2020 ) ; Computer clusters predict the processing times and resource demands of computation tasks to make good job scheduling on servers ( Mao et al. , 2016 ) ; Hedge funds forecast the return rates of different stocks to optimize their portfolio in the next trading day ( Thomson , 2021 ) . However , the commonly used l2 loss function ( i.e. , l2 distance between predictions and true values ) usually can not achieve ideal decision results in predict-then-optimize problems ( Demirović et al. , 2019 ) . This misalignment between the l2 loss in prediction and quality of the decision comes from the misalignment between the continuity on the prediction problem and the discontinuity on the structure of the combinatorial optimization problem . A straightforward remedy of the above misalignment problem is to adjust the loss to reflect the difference in objective values of the optimization solutions generated using the predicted and observed parameters , called Smart “ Predict , then Optimize ” ( SPO ) loss ( Elmachtoub & Grigas , 2021 ) . However , due to the combinatorial nature of the optimization problems , the SPO loss is usually piecewise flat and has multiple discontinued points . As a result , the derivatives of SPO loss is either zero or nonexistent , which prohibits the training of gradient-based deep learning algorithms . Besides , most solvers for combinatorial optimization problems are not differentiable , which can not be directly incorporated with the widely adopted gradient-based learning approaches nowadays . Current approaches to minimizing the SPO loss can be categorized into two classes . The first one is to differentiate the discrete optimization solver by approximating the objective with a certain family of functions , and this series of methods works when the combinatorial optimization problem is linear ( Wilder et al. , 2019a ; b ; Pogančić et al. , 2019 ) . The other class of research tries to propose new surrogate loss functions for SPO loss , e.g. , SPO+ loss for linear programming ( Elmachtoub & Grigas , 2021 ) and piecewise linear loss for dynamic programming ( Stuckey et al. , 2020 ) . However , these surrogate loss functions are problem-specific ( i.e. , depending on the type of the combinatorial optimization problem ) and require much domain knowledge from experts to design . Instead of designing the surrogate loss metric for each optimization problem manually , an alternative approach is to find a good loss metric from a predefined search space in an automatic manner , which is inspired by the recent progress in automated machine learning ( AutoML ) ( Zoph & Le , 2017 ; Pham et al. , 2018 ; Liu et al. , 2018 ) . These AutoML approaches usually define an appropriate search space and then conduct some search algorithms , e.g. , reinforcement learning and genetic algorithm , to find a good metric . Li et al . ( 2019 ) and Li et al . ( 2021 ) studied the automatic loss function search problem for computer vision tasks . They defined the search space by replacing the mainstream metrics for semantic segmentation with their differentiable approximations . However for predict-then-optimize problems , there is no such evaluation metrics available except for SPO loss . Moreover , these metrics , which are designed for segmentation tasks in computer vision domain , are not suitable to evaluate the prediction results for the combinatorial optimization problems . In this paper , we propose a framework of automatic loss function search called APOC ( Automatic Prediction and Optimization Connector ) to tackle a wide range of predict-then-optimize problems whose optimal solution is determined by the total group preorder of the parameters of the combinatorial problem , called strong ranking property ( see for Definition 2 for details. ) . These problems have the same optimal solution for different sets of parameters , as long as those sets of parameters preserve the same total group preorders . The key idea is to build the proper search space in which different loss functions capture the partial comparisons between different groups of the parameters of the optimization problem . Our contributions are summarized as follows : 1 ) We theoretically prove that l2 loss is not an ideal choice in a linear regression setting of PTO problems ; 2 ) We propose the total group preorder loss for PTO problems and relax it to a differentiable approximated version , which is fitted using a family of Derivative-of-Guassian wavelet functions to capture the group comparison relationship among the items with variable sizes and weights ; 3 ) We propose an effective method to automatically search for a differentiable surrogate loss based on approximated group total preorders of items for PTO problems called APOC ; 4 ) The proposed APOC method has been validated on three classic combinatorial problems with strong ranking property . 2 RELATED WORK . Predict-then-optimize ( PTO ) problems capture a common pipeline of machine-learning-assisted optimization solvers : predicting the unknown parameters of the optimization problem from the contextual information and then optimizing based on the predicted parameters ( Elmachtoub & Grigas , 2021 ) . However , many combinatorial optimization problems have piecewise flat value function landscapes which prohibit an end-to-end learning with gradient-based prediction models . One line of work is to differentiate the optimization solvers for specific problems and embed them as layers into the network architecture through which the gradients can be propagated , which originates from the differentiable optimization ( Amos , 2019 ) . Grover et al . ( 2018 ) proposes a sorting network for ranking problems by relaxing the permutation matrix output by sorting algorithms to the unimodal row-stochastic matrix . For the node clustering problem , Wilder et al . ( 2019b ) differentiates the K-means algorithm by assigning nodes to clusters according to a soft-min function . Pogančić et al . ( 2019 ) interpolates the value of function of linear programming using piecewise linear functions and embeds as layers the solvers of linear programming . The other line of work focuses on searching for a surrogate loss function to approximate the value function of the optimization problem , so-called regret in decision theory ( Bell , 1982 ) . Elmachtoub & Grigas ( 2021 ) derives a convex upper bound on the regret of linear programming , called SPO+ loss . Wilder et al . ( 2019a ) proposes the QTPL loss by adding a quadratic penalty term to the continuous relaxation of the regret such that results from differentiating over quadratic programs can be used . Mandi & Guns ( 2020 ) overcomes the non-differentiablity of the regret by adding a logarithmic barrier term , called IntOpt . Stuckey et al . ( 2020 ) approximates the value function of dynamic programming problems as piecewise linear functions with learn-able parameters . Yoon et al . ( 2013 ) proposes mean objective cost of uncertainty ( MOCU ) , the difference between the value functions of a robust solution and the optimal solution , to evaluate the uncertainty of parameters in decision making , e.g. , experiment design ( Boluki et al. , 2018 ) and active learning ( Zhao et al. , 2021 ) . Loss function guides the machine learning algorithms to produce good predictions for different tasks , which is usually designed with domain knowledge from experts ( Masnadi-Shirazi & Vasconcelos , 2008 ; Bruch et al. , 2019 ) . Automatic search of suitable loss function without domain knowledge has recently received much attention from the computer vision community . Li et al . ( 2019 ) uses reinforcement learning algorithms to learn better loss functions with good generalization and transfer-ability on different vision tasks . Wang et al . ( 2020 ) adopts both random search and reinforcement learning algorithms to search for loss functions on face recognition problems . Li et al . ( 2021 ) explores the possibility of searching loss function automatically from scratch for generic tasks , e.g. , semantic segmentation , object detection , and pose estimation . However , the search spaces in these methods are specially designed for vision tasks and can not be directly applied to PTO problems . Most work on automatic loss search follows the searching algorithms used in AutoML . We refer to He et al . ( 2021 ) for comprehensive survey on searching methods in AutoML . Natural questions are : 1 ) whether we can design a suitable search space for loss functions in PTO problems ; and 2 ) whether there is a strategy to search adequate loss functions systematically for different optimization problems using techniques from AutoML . 3 ATGP LOSS FOR PREDICT-THEN-OPTIMIZE PROBLEMS . 3.1 MISALIGNMENT IN PREDICT-THEN-OPTIMIZE PROBLEM . The problems we consider are the type of predict-then-optimize ( PTO ) problems in the following formulation . For an optimization problem in the following form maximize z Uc ( z ) subject to z ∈ Z ( 1 ) where z is the decision variable , c ∈ Rd is the parameter of the objective function U , and Z is the feasible set which does not depend on c. A decision maker needs to solve ( 1 ) without knowing the exact value of c. Instead , the decision maker will observe a feature vector x ∈ Rk . The goal of the decision maker is to predict the value of c and then optimize ( 1 ) based on the prediction ĉ . To distinguish with the parameters of the prediction models ( e.g. , weights of neural networks ) , we call c the PTO parameter . Instead of focusing on the prediction error between c and ĉ ( e.g. , l2 distance ) , the decision maker cares more about a high-quality decision result generated from the prediction . Elmachtoub & Grigas ( 2021 ) characterizes this quantity by the Smart “ Predict , then Optimize ” ( SPO ) loss , which equals to the loss incurred by solving the optimization problem ( 1 ) using ĉ instead of the true parameter c. ` SPO ( ĉ , c ) = E [ U∗c − Uc ( z∗ ( ĉ ) ) ] where z∗ ( c ) = arg maxz∈Z Uc ( z ) and U∗c = Uc ( z ∗ ( c ) ) . Figure 2a shows the misalignment between the l2 distance ( green ) and the SPO loss ( blue ) in a ranking PTO problem . In this ranking PTO problem , there are two items to be ranked . The true values of the items are c1 = 4.9 and c2 = 5.1 , which are also the values to predict . The prediction denoted by the magenta triangle has a lower SPO loss than the orange one , and therefore yields a better solution . However , prediction model that minimizes the l2 distance will prefer the orange one ( whose l2 distance is lower ) and produce a worse solution with regard to the ranking task . However , the SPO loss of most PTO problems is a piecewise linear function due to the combinatorial structure in the objective function Uc ( z ) and the solution map z∗ ( c ) . As a result , its derivatives in the predicted PTO parameter ĉ is either zero or nonexistent , which prohibits the usage of prediction models whose training is based on gradient descent algorithms . In this paper , we consider a collection of combinatorial optimization problems with strong ranking property . For this collection of optimization problems , we propose the total group preorder ( TGP ) loss and its differentiable variant approximated total group preorder ( ATGP ) loss as the training targets in the prediction stage . The ATGP loss will approximate the SPO loss with smooth shape , which is friendly and effective for the gradient-based training manner , as is shown in Figure 2c .

Summary: This paper studies a problem where the goal is to solve an optimization problem, but key parameters of the optimization problem (like a road network’s edge weights) are unknown and can only be predicted from historical data. This line of research is called “predict then optimize.” For the prediction step, the first thing one might try would be to minimize a standard loss function like the l2 loss, but this loss function may not necessarily lead the downstream optimization problem to return the highest quality solutions. This paper studies a way to learn a good loss function for the downstream optimization problem at hand.  To give an example, if the optimization problem is to rank a set of elements, even if one can predict the elements’ values with low l2-error, the resulting ranking might be totally incorrect. Motivated by this example, this paper’s results apply to combinatorial optimization problems that satisfy a “strong ranking property,” which essentially means that the optimization problem’s solution only depends on group-wise comparisons of the problem’s parameters (rather than on the specific values of the parameters themselves).  Let $c \in R^d$ be the true parameters and $\hat{c}$ be the learned parameters. The authors propose that the loss function one should optimize should ideally have the form $||sign(L\hat{c}) - sign(Lc)||$, where $L$ is a matrix of size ${2^d \choose 2} \times d$ that allows for all $2^d \choose 2$ group-wise comparisons between all groups of parameters. Since this ideal loss function is computationally intractable, the authors propose replace $sign$ with $\tanh$. Then, they propose fixing the number of rows of $L$ and parameterizing the matrix using a set of discretized Derivative-of-Gaussian filters, which they optimize using the reinforcement learning algorithm REINFORCE [Williams ‘92] (the details here are a bit unclear to me).  They evaluate their algorithm for three problems: a scheduling problem, portfolio optimization, and shortest path. They compare against a number of different predict-then-optimize algorithms from prior research and show that their algorithm has much better losses on the problems studied.