A Regression Approach to Learning-Augmented Online Algorithms

1 Introduction . A recent trend in online algorithms has seen the use of future predictions generated by ML techniques to bypass pessimistic worst-case lower bounds . A growing body of work has started to emerge in this area in the last few years addressing a broad variety of problems in online algorithms such as rent or buy , caching , metrical task systems , matching , scheduling , experts learning , stopping problems , and others ( see related work for references ) . The vast majority of this literature is focused on using ML predictions in online algorithms , but does not address the question of how these predictions are generated . This raises the question : what can we learn from data that will improve the performance of online algorithms ? Abstractly , this question comes in two inter-dependent parts : the first part is a learning problem where we seek to learn a function that maps the feature domain to predicted parameters , and the second part is to re-design the online algorithm to use these predictions . In this paper , we focus on the first part of this design pipeline , namely we develop a regression approach to generating ML predictions for online algorithms . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Before delving into this question further , we note that there has been some recent research that focuses on the learnability of predicted parameters in online algorithms . Recently , Lavastida et al . [ 33 ] , building on the work of Lattanzi et al . [ 32 ] , took a data-driven algorithms approach to design online algorithms for scheduling and matching problems via learned weights . In this line of work , the goal is to observe sample inputs in order to learn a set of weights that facilitate better algorithms for instances from a fixed distribution . In contrast , Anand et al . [ 4 ] relied on a classification learning approach for the Ski Rental problem , where they aimed to learn a function that maps the feature set to a binary label characterizing the optimal solution . But , in general , the value of the optimal solution is a real-valued function , which motivates a regression approach to learning-augmented online algorithms that we develop in this paper . To formalize the notion of an unknown optimal solution that we seek to learn via regression , we use the online search ( ONLINESEARCH ) framework . In this framework , there is as an input sequence Σ = σ1 , σ2 , . . . available offline , and the actual online input is a prefix of this sequence ΣT = σ1 , σ2 , . . . , σT , where the length of the prefix T is revealed online . Namely , in each online step t > 0 , there are two possibilities : either the sequence ends , i.e. , T = t , or the sequence continues , i.e. , T > t. The algorithm must maintain , at all times t , a solution that is feasible for the current sequence , i.e. , for the prefix Σt = σ1 , . . . , σt . The goal is to obtain a solution that is of minimum cost among all the feasible solutions for the actual input sequence ΣT . We will discuss applicability of the ONLINESEARCH framework in more detail in Section 1.2 , but for a quick illustration now , consider the ski rental problem in this framework . In this problem , if the sequence continues on day t , then the algorithm must rent skis if it has not already bought them . In generalizations of the ski rental problem to multiple rental options , the requirement is that one of the rental options availed by the algorithm must cover day t. We will show in Section 1.2 that we can similarly model several other classic online problems in the ONLINESEARCH framework . We use the standard notion of competitive ratio , defined as the worst case ratio between the algorithm ’ s cost and the optimal cost , to quantify the performance of an online algorithm . For online algorithms with predictions , we follow the terminology in [ 41 ] that is now standard : we say that the consistency and robustness of an algorithm are its competitive ratios for correct predictions and for arbitrarily incorrect predictions respectively . Typically , we fix consistency at 1 + for a hyper-parameter and aim to minimize robustness as a function of . We make some mild assumptions on the problem . First , we assume that solutions are composable , i.e. , that adding feasible solutions for subsequences ensures feasibility over the entire sequence ; second , that cost is monotone , i.e. , the optimal cost for a subsequence is at most that for the entire sequence ; and third , that the offline problem is ( approximately or exactly ) solvable . These assumptions hold for essentially all online problems we care for . 1.1 Our Contributions . As a warm up , we first give an algorithm called DOUBLE for the ONLINESEARCH problem without predictions in Section 2 . The DOUBLE algorithm has a competitive ratio of 4 . We build on the DOUBLE algorithm in Section 3 , where we give an algorithm called PREDICT-AND-DOUBLE for the ONLINESEARCH problem with predictions . We show that the PREDICT-AND-DOUBLE algorithm has a consistency of 1 + and robustness of O ( 1/ ) , for any hyper-parameter > 0 . We also show that this tradeoff between consistency and robustness is asymptotically tight . Our main contributions are in Section 4 . In this section , we model the question of obtaining a learningaugmented algorithm for the ONLINESEARCH problem in a regression framework . Specifically , we assume that the input comprises a feature vector x that is mapped by an unknown real-valued function f to an input for the ONLINESEARCH problem z . In the training phase , we are given a set of labeled samples of the form ( x , z ) from some ( unknown to the algorithm ) data distribution D. The goal of the learning algorithm is to produce a mapping from the feature space to algorithmic strategies for the ONLINESEARCH problem , such that when it gets an unlabeled ( test ) sample x from the same distribution D , the algorithmic strategy corresponding to x obtains a competitive solution for the actual input z in the test sample ( that is unknown to the algorithm ) . The learning algorithm employs a regression approach in the following manner . It assumes that the function f is from a hypothesis class F , and obtains an empirical minimizer in F for a carefully crafted loss function on the training samples . The design of this loss function is crucial since a bound on this loss function is then shown to translate to a bound on the competitive ratio of the algorithmic strategy . ( Indeed , we will show later that because of this reason , standard loss functions used in regression are inadequate for our purpose . ) Finally , we use statistical learning theory for real-valued functions to bound the sample complexity of the learner that we designed . Using the above framework , we show a sample complexity bound of O ( H·d ) for obtaining a competitive ratio of 1 + , where H and d respectively represent the log-range of the optimal cost and a measure of the expressiveness of the function class F called its pseudo-dimension.1 We also extend this result to the so-called agnostic setting , where the function class F is no longer guaranteed to contain an exact function f that maps x to z , rather the competitive ratio is now in terms of the best function in this class that approximates f . We also prove nearly matching lower bounds for our sample complexity bounds in the two models . Our framework can also be extended to the setting where the offline optimal solution is hard to compute , but there exists an algorithm with competitive ratio c given the cost of optimal solution . In that case our algorithms gives a competitive ratio c ( 1 + ) , which can still be better than the competitive ratio without predictions ( see examples in next subsection ) . 1.2 Applicability of the ONLINESEARCH framework . The ONLINESEARCH framework is applicable whenever an online algorithm benefits from knowing the optimal value of the solution . Many online problems benefit from this knowledge , which is sometimes called advice in the online algorithms literature . For concreteness , we give three examples of classic problems – ski rental with multiple options , online scheduling , and online bin packing – to illustrate the applicability of our framework . Our algorithm PREDICT-AND-DOUBLE ( explained in more detail in section 3 ) successively predicts the optimal value of the solution and appends the corresponding solution to its output . Ski Rental with Multiple Options . Generalizations of the ski rental problem with multiple options have been widely studied ( e.g. , [ 1 , 34 , 37 , 19 ] ) , recently with ML predictions [ 44 ] . Suppose there are V options ( say coupons ) at our disposal , where coupon i costs us Ci and is valid for di number of days . Given such a setup , we need to come up with a schedule : { ( tk , ik ) , k = 1 , 2 . . . } that instructs us to buy coupon ik at time tk . ( The classic ski rental problem corresponds to having only two coupons C1 = 1 , d1 = 1 and C2 = B , d2 → ∞ . ) Our ONLINESEARCH framework is applicable here : a solution that allows us to buy coupons valid time t is also a valid solution for all times s ≤ t. Further , PREDICT-AND-DOUBLE can be implemented efficiently as we can compute OPT ( t ) , for any time t using a dynamic program . Online Scheduling . Next , we consider the classic online scheduling problem where the goal is to assign jobs arriving online to a set of identical machines so as to minimize the maximum load on any machine ( called the makespan ) . For this algorithm , the classic list scheduling algorithm [ 26 ] has a competitive ratio of 2 . A series of works [ 23 , 14 , 29 , 2 ] improved the competitive ratio to 1.924 , and currently the best known result has competitive ratio of ( approx ) 1.92 [ 20 ] ; in fact , there are nearly matching lower bounds [ 25 ] . However , if the optimal make-span ( OPT ) is known , then these lower bounds can be overcome , and a significantly better competitive ratio of 1.5 can be obtained in this setting [ 17 ] ( see also [ 10 , 31 , 21 , 22 ] ) . The ONLINESEARCH framework is applicable here with a slight modification : whenever PREDICT-AND-DOUBLE tries to buy a solution corresponding to a predicted value of OPT , we execute the 1.5-approximation algorithm based on this value . The problem still satisfies the property that a solution for t jobs is valid for any prefix . We get a competitive ratio of 1.5 +O ( ) that significantly outperforms the competitive ratio of 1.92 without predictions . Online Bin Packing . As a third example , we consider the online bin packing problem . In this problem , items arrive online and must be packed into fixed-sized bins , the goal being to minimize the number of bins . ( We can assume w.l.o.g. , by scaling , that the bins are of unit size . ) Here , it is known that the critical parameter that one needs to know/predict is not OPT but the number of items of moderate size , namely those sized between 1/2 and 2/3 . If this is known , then there is a simple 1.5-competitive algorithm [ 5 ] , which is not achievable without this additional knowledge . Again , our ONLINESEARCH framework can be used to take advantage of this result . In this case , the application is not as direct , because predicting OPT does not yield the better algorithm . Nevertheless , 1Intuitively , the notion of pseudo-dimension extends that of the well-known VC dimension from binary to real-valued functions . an inspection of the algorithm in [ 5 ] reveals the following strategy : The items are partitioned into three groups . The items of size≥ 2/3 are assigned individual bins , items of size between 1/3 and 1/2 are assigned separate bins where at least two of them are assigned to each bin , and the remaining items are assigned a set of common bins . Clearly , the first two categories can be handled online without any additional information ; this means that we can define a surrogate OPT ( call it OPT′ ) that only captures the optimal number of bins for the common category . Note that the of prediction of OPT ’ serves as a substitute for knowing the numbers of items of moderate size . Now , if OPT′ is known , then we can recover the competitive ratio of 3/2 by using a simple greedy strategy . This now allows us to use the ONLINESEARCH framework where we predict OPT′ . As earlier , the ONLINESEARCH framework can be applied with slight modification : whenever PREDICT-AND-DOUBLE tries to buy a solution corresponding to a predicted value of OPT′ , we execute the 1.5-competitive algorithm based on this value . The problem still satisfies the property that a solution for t items is valid for any prefix .

Summary: Learning-augmented algorithms have been recently proposed as a way to utilize machine-learned predictions in online algorithm design. This paper considers the related problem of how to learn from past data so that online algorithms can obtain improved performance. Specifically, the paper introduces a general online search problem and design online algorithms for it, both without using predictions and using them. The online search framework is fairly general and encompasses well-studied problems such as generalized ski-rental, load balancing and bin packing. Given a single real-valued predicted parameter, the authors design an algorithm that is $(1+\epsilon)$-consistent and $O(1/\epsilon)$ robust.   Secondly (and more interestingly), the paper designs a learning algorithm to efficiently learn such a parameter given sample access to a data distribution. The authors show that utilizing the structure of the online algorithm, one can define an asymmetric loss function such that the empirical minimizer of this loss function serves as a good prediction for the online algorithm.