Automatic Loss Function Search for Predict-Then-Optimize Problems with Strong Ranking Property

1 INTRODUCTION . Many decision making processes under uncertainty in real world are solving combinatorial problems with parameters unknown to the decision maker . A traditional method to address the uncertainty issue is to add assumptions on the distributions of parameters ( Hentenryck & Bent , 2006 ) . Alternatively , a recent approach is to predict the unknown parameters from the correlated features using machine learning methods and solve the combinatorial problems based on the predictions . This paradigm , which is called predict-then-optimize ( PTO ) , has been widely employed in practice systematically ( Elmachtoub & Grigas , 2021 ) . For example , Google Maps estimates the travel times of roads in the traffic to compute a shortest path ( Lau , 2020 ) ; Computer clusters predict the processing times and resource demands of computation tasks to make good job scheduling on servers ( Mao et al. , 2016 ) ; Hedge funds forecast the return rates of different stocks to optimize their portfolio in the next trading day ( Thomson , 2021 ) . However , the commonly used l2 loss function ( i.e. , l2 distance between predictions and true values ) usually can not achieve ideal decision results in predict-then-optimize problems ( Demirović et al. , 2019 ) . This misalignment between the l2 loss in prediction and quality of the decision comes from the misalignment between the continuity on the prediction problem and the discontinuity on the structure of the combinatorial optimization problem . A straightforward remedy of the above misalignment problem is to adjust the loss to reflect the difference in objective values of the optimization solutions generated using the predicted and observed parameters , called Smart “ Predict , then Optimize ” ( SPO ) loss ( Elmachtoub & Grigas , 2021 ) . However , due to the combinatorial nature of the optimization problems , the SPO loss is usually piecewise flat and has multiple discontinued points . As a result , the derivatives of SPO loss is either zero or nonexistent , which prohibits the training of gradient-based deep learning algorithms . Besides , most solvers for combinatorial optimization problems are not differentiable , which can not be directly incorporated with the widely adopted gradient-based learning approaches nowadays . Current approaches to minimizing the SPO loss can be categorized into two classes . The first one is to differentiate the discrete optimization solver by approximating the objective with a certain family of functions , and this series of methods works when the combinatorial optimization problem is linear ( Wilder et al. , 2019a ; b ; Pogančić et al. , 2019 ) . The other class of research tries to propose new surrogate loss functions for SPO loss , e.g. , SPO+ loss for linear programming ( Elmachtoub & Grigas , 2021 ) and piecewise linear loss for dynamic programming ( Stuckey et al. , 2020 ) . However , these surrogate loss functions are problem-specific ( i.e. , depending on the type of the combinatorial optimization problem ) and require much domain knowledge from experts to design . Instead of designing the surrogate loss metric for each optimization problem manually , an alternative approach is to find a good loss metric from a predefined search space in an automatic manner , which is inspired by the recent progress in automated machine learning ( AutoML ) ( Zoph & Le , 2017 ; Pham et al. , 2018 ; Liu et al. , 2018 ) . These AutoML approaches usually define an appropriate search space and then conduct some search algorithms , e.g. , reinforcement learning and genetic algorithm , to find a good metric . Li et al . ( 2019 ) and Li et al . ( 2021 ) studied the automatic loss function search problem for computer vision tasks . They defined the search space by replacing the mainstream metrics for semantic segmentation with their differentiable approximations . However for predict-then-optimize problems , there is no such evaluation metrics available except for SPO loss . Moreover , these metrics , which are designed for segmentation tasks in computer vision domain , are not suitable to evaluate the prediction results for the combinatorial optimization problems . In this paper , we propose a framework of automatic loss function search called APOC ( Automatic Prediction and Optimization Connector ) to tackle a wide range of predict-then-optimize problems whose optimal solution is determined by the total group preorder of the parameters of the combinatorial problem , called strong ranking property ( see for Definition 2 for details. ) . These problems have the same optimal solution for different sets of parameters , as long as those sets of parameters preserve the same total group preorders . The key idea is to build the proper search space in which different loss functions capture the partial comparisons between different groups of the parameters of the optimization problem . Our contributions are summarized as follows : 1 ) We theoretically prove that l2 loss is not an ideal choice in a linear regression setting of PTO problems ; 2 ) We propose the total group preorder loss for PTO problems and relax it to a differentiable approximated version , which is fitted using a family of Derivative-of-Guassian wavelet functions to capture the group comparison relationship among the items with variable sizes and weights ; 3 ) We propose an effective method to automatically search for a differentiable surrogate loss based on approximated group total preorders of items for PTO problems called APOC ; 4 ) The proposed APOC method has been validated on three classic combinatorial problems with strong ranking property . 2 RELATED WORK . Predict-then-optimize ( PTO ) problems capture a common pipeline of machine-learning-assisted optimization solvers : predicting the unknown parameters of the optimization problem from the contextual information and then optimizing based on the predicted parameters ( Elmachtoub & Grigas , 2021 ) . However , many combinatorial optimization problems have piecewise flat value function landscapes which prohibit an end-to-end learning with gradient-based prediction models . One line of work is to differentiate the optimization solvers for specific problems and embed them as layers into the network architecture through which the gradients can be propagated , which originates from the differentiable optimization ( Amos , 2019 ) . Grover et al . ( 2018 ) proposes a sorting network for ranking problems by relaxing the permutation matrix output by sorting algorithms to the unimodal row-stochastic matrix . For the node clustering problem , Wilder et al . ( 2019b ) differentiates the K-means algorithm by assigning nodes to clusters according to a soft-min function . Pogančić et al . ( 2019 ) interpolates the value of function of linear programming using piecewise linear functions and embeds as layers the solvers of linear programming . The other line of work focuses on searching for a surrogate loss function to approximate the value function of the optimization problem , so-called regret in decision theory ( Bell , 1982 ) . Elmachtoub & Grigas ( 2021 ) derives a convex upper bound on the regret of linear programming , called SPO+ loss . Wilder et al . ( 2019a ) proposes the QTPL loss by adding a quadratic penalty term to the continuous relaxation of the regret such that results from differentiating over quadratic programs can be used . Mandi & Guns ( 2020 ) overcomes the non-differentiablity of the regret by adding a logarithmic barrier term , called IntOpt . Stuckey et al . ( 2020 ) approximates the value function of dynamic programming problems as piecewise linear functions with learn-able parameters . Yoon et al . ( 2013 ) proposes mean objective cost of uncertainty ( MOCU ) , the difference between the value functions of a robust solution and the optimal solution , to evaluate the uncertainty of parameters in decision making , e.g. , experiment design ( Boluki et al. , 2018 ) and active learning ( Zhao et al. , 2021 ) . Loss function guides the machine learning algorithms to produce good predictions for different tasks , which is usually designed with domain knowledge from experts ( Masnadi-Shirazi & Vasconcelos , 2008 ; Bruch et al. , 2019 ) . Automatic search of suitable loss function without domain knowledge has recently received much attention from the computer vision community . Li et al . ( 2019 ) uses reinforcement learning algorithms to learn better loss functions with good generalization and transfer-ability on different vision tasks . Wang et al . ( 2020 ) adopts both random search and reinforcement learning algorithms to search for loss functions on face recognition problems . Li et al . ( 2021 ) explores the possibility of searching loss function automatically from scratch for generic tasks , e.g. , semantic segmentation , object detection , and pose estimation . However , the search spaces in these methods are specially designed for vision tasks and can not be directly applied to PTO problems . Most work on automatic loss search follows the searching algorithms used in AutoML . We refer to He et al . ( 2021 ) for comprehensive survey on searching methods in AutoML . Natural questions are : 1 ) whether we can design a suitable search space for loss functions in PTO problems ; and 2 ) whether there is a strategy to search adequate loss functions systematically for different optimization problems using techniques from AutoML . 3 ATGP LOSS FOR PREDICT-THEN-OPTIMIZE PROBLEMS . 3.1 MISALIGNMENT IN PREDICT-THEN-OPTIMIZE PROBLEM . The problems we consider are the type of predict-then-optimize ( PTO ) problems in the following formulation . For an optimization problem in the following form maximize z Uc ( z ) subject to z ∈ Z ( 1 ) where z is the decision variable , c ∈ Rd is the parameter of the objective function U , and Z is the feasible set which does not depend on c. A decision maker needs to solve ( 1 ) without knowing the exact value of c. Instead , the decision maker will observe a feature vector x ∈ Rk . The goal of the decision maker is to predict the value of c and then optimize ( 1 ) based on the prediction ĉ . To distinguish with the parameters of the prediction models ( e.g. , weights of neural networks ) , we call c the PTO parameter . Instead of focusing on the prediction error between c and ĉ ( e.g. , l2 distance ) , the decision maker cares more about a high-quality decision result generated from the prediction . Elmachtoub & Grigas ( 2021 ) characterizes this quantity by the Smart “ Predict , then Optimize ” ( SPO ) loss , which equals to the loss incurred by solving the optimization problem ( 1 ) using ĉ instead of the true parameter c. ` SPO ( ĉ , c ) = E [ U∗c − Uc ( z∗ ( ĉ ) ) ] where z∗ ( c ) = arg maxz∈Z Uc ( z ) and U∗c = Uc ( z ∗ ( c ) ) . Figure 2a shows the misalignment between the l2 distance ( green ) and the SPO loss ( blue ) in a ranking PTO problem . In this ranking PTO problem , there are two items to be ranked . The true values of the items are c1 = 4.9 and c2 = 5.1 , which are also the values to predict . The prediction denoted by the magenta triangle has a lower SPO loss than the orange one , and therefore yields a better solution . However , prediction model that minimizes the l2 distance will prefer the orange one ( whose l2 distance is lower ) and produce a worse solution with regard to the ranking task . However , the SPO loss of most PTO problems is a piecewise linear function due to the combinatorial structure in the objective function Uc ( z ) and the solution map z∗ ( c ) . As a result , its derivatives in the predicted PTO parameter ĉ is either zero or nonexistent , which prohibits the usage of prediction models whose training is based on gradient descent algorithms . In this paper , we consider a collection of combinatorial optimization problems with strong ranking property . For this collection of optimization problems , we propose the total group preorder ( TGP ) loss and its differentiable variant approximated total group preorder ( ATGP ) loss as the training targets in the prediction stage . The ATGP loss will approximate the SPO loss with smooth shape , which is friendly and effective for the gradient-based training manner , as is shown in Figure 2c .

Summary: In this paper, the authors propose a novel loss function called the approximated total group preorder (ATGP) loss function, which aims to address the potential limitations of l2 loss in predict-then-optimize (PTO) problems. The authors argue that the use of l2 loss in the prediction phase is "misaligned" with the final goal when the ultimate objective is to make discrete decisions in a combinatorial optimization setting. While the so-called SPO (smart PTO) loss, recently proposed by Elmachtoub and Grigas (2021) provides a remedy, the authors note that it cannot be used with gradient-based learning algorithms. The authors aim to rectify this issue by proposing a novel loss function called the total group preorder (TGP) loss - for better alignment - and then an algorithm for automatically searching for an approximate TGP (ATGP) loss function so that it is differentiable and therefore can take advantage of efficient gradient-based learning schemes.