Learning Predictions for Algorithms with Predictions
Abstract
A burgeoning paradigm in algorithm design is the ﬁeld of algorithms with predictions , in which algorithms can take advantage of a possibly-imperfect prediction of some aspect of the problem.
While much work has focused on using predictions to improve competitive ratios, running times, or other performance measures, less effort has been devoted to the question of how to obtain the predictions themselves, especially in the critical online setting.
We introduce a general design approach for algorithms that learn predictors: (1) identify a functional dependence of the performance measure on the prediction quality and (2) apply techniques from online learning to learn predictors, tune robustness-consistency trade-offs, and bound the sample complexity.
We demonstrate the effectiveness of our approach by applying it to bipartite matching, ski-rental, page migration, and job scheduling.
In several settings we improve upon multiple existing results while utilizing a much simpler analysis, while in the others we provide the ﬁrst learning-theoretic guarantees.
Introduction
Algorithms with predictions, a subﬁeld of beyond-worst-case analysis of algorithms [36], aims to design methods that make use of machine-learned predictions in order to reduce runtime, error or some other performance cost.
Mathematically, for some prediction x , algorithms in this ﬁeld are designed such that their cost C t ( x ) on an instance t is upper-bounded by some measure U t ( x ) of the quality of the prediction on that instance.
The canonical example here is that the cost of binary search on a sorted array of size n can be improved from O (log n ) to ≤ U t ( x ) = 2 log η t ( x ) , where η t ( x ) is the distance between the true location of a query t in the array and the location predicted by the predictor x [36].
In recent years, algorithms whose cost depends on the quality of possibly imperfect predictions have been developed for numerous important problems, including caching [41, 24, 34], scheduling [30, 43], ski-rental [29, 1, 15], bipartite matching [16], page migration [22], and many more [10, 17, 36].
While there has been a signiﬁcant effort to develop algorithms that use earned predictions, until very recently [13, 33] there has been less focus on actually learning to predict.
For example, of the works listed only two on ski-rental [1, 15] and one other [16] show sample complexity guarantees, and none consider the important online learning setting, in which problem instances may not come from a ﬁxed distribution.
This is in contrast to the related area of data-driven algorithm design [19, 3], which has established techniques such as dispersion [6] and others [8, 11] for deriving learning-theoretic guarantees, leading to end-to-end results encompassing both learning and computation.
It is also despite the fact that, as we see in this work, learning even simple predictors is in many cases a non-trivial problem.
We bridge this gap and provide a framework for obtaining learning-theoretic guarantees for algorithms with predictions.
In addition to improving sample complexity bounds, we show how to learn the parameters of interest in an setting with low overall regret.
We accomplish this using a two-
[Table] [Table]
step approach inspired by recent work on theoretical meta-learning [25], which has been used to derive numerous multi-task learning results by optimizing regret-upper-bounds that encode the tasksimilarity [31, 32, 9, 26].
As evidenced by our results in Table 1, we believe the following two-step framework below holds similar potential for obtaining guarantees for algorithms with predictions:
For a given algorithm, derive a convenient-to-optimize upper bound
U t ( x ) on the cost C t ( x ) that depends on both the prediction x and information speciﬁc to instance t returned once the algorithm terminates, e.g. the optimum in combinatorial optimization.
We ﬁnd that in many cases such bounds already exist, and the quality of the prediction can be measured by a distance from some ground truth obtained from the output, a quantity that is usually convex and thus learnable.
Apply online learning to obtain both regret guarantees against
adversarial sequences and sample complexity bounds for i.i.d.
instances.
We provide pseudo-code for a generic setup in Algorithm 1.
Table 1 summarizes instantiations of our framework on multiple problems.
Our approach is designed to be simple-to-execute, leaving much of the difﬁculty to what the ﬁeld is already good at: designing algorithms and proving prediction-quality-dependent upper bounds on their costs.
Once the latter is accomplished, our framework leverages problem-speciﬁc structure to design a customized learning algorithm for each problem, leading to strong regret and sample complexity guarantees.
In particular, in multiple settings we improve upon existing results in either sample complexity or generality, and in all cases we are the ﬁrst to show regret guarantees in the online setting.
This demonstrates the usefulness of and need for such a theoretical framework for studying these problems.
We summarize the diverse set of contributions enabled by our theoretical framework below:
Bipartite matching: Our starting example builds upon the work on minimum-weight bipartite matching using the Hungarian algorithm by Dinitz et al.
We show how our framework leads directly to both the ﬁrst regret guarantees in the online setting and new sample complexity bounds that improve over the previous approach by a factor linear in the number of nodes.
In the Appendix we show similar strong improvements for b -matching and other graph algorithms.
Page migration: We next study a more challenging application, online page migration , and show how we can adapt the algorithmic guarantee of Indyk et al.
[22] into a learnable upper bound for which we can again provide both adversarial and statistical guarantees.
Learning linear maps with instance-feature inputs: Rather than assume the existence of a strong ﬁxed prediction, it is often more natural to assume each instance comes with features that can be input into a predictor such as a linear map.
Our approach yields the ﬁrst guarantees for learning linear predictors for algorithms with predictions, which we obtain for the two problem settings above and also for online job scheduling using makespan minimization [30].
Tuning robustness-consistency trade-offs: Many bounds for online algorithms with predictions incorporate parameterized trade-offs between trusting the prediction or falling back on a worst-case approximation.
This suggests the usefulness of tuning
the trade-off parameter, which we instantiate on a simple job scheduling problem with a ﬁxed predictor.
Then we turn to the more challenging problem of simultaneously tuning the trade-off and learning predictions, which we achieve on two variants of the ski-rental problem.
For the discrete case we give the only learning-theoretic guarantee, while for the continuous case our bound uses a dispersion assumption [6] that, in the i.i.d.
setting, is a strictly weaker assumption than the log-concave requirement of Diakonikolas et al.
Related work
Algorithms with predictions is a type of beyond-worst-case analysis of algorithms [42]; along with areas like smooth analysis [45] and data-driven algorithm design [3], it takes advantage of the fact that real-world instances are not worst-case.
Inspired by success in applications such as learned indices [27], there has been a great deal of theoretical study focusing on algorithms whose guarantees depend on the quality of a given predictor (c.f. the Mitzenmacher and Vassilvitskii [36] survey).
The actual learning of this predictor has been studied less [1, 15, 16] and rarely in the online setting; we aim to change this with our study.
Some papers improve online learning itself using predictions [39, 23, 14], but they also assume known predictors or only learn over a small set of policies, and their goal is minimizing regret not computation.
In-general, we focus on showing how algorithms with predictions can make use of online learning rather than on new methods for the latter.
Several works [4, 40, 2] use learning while advising an algorithm, in-effect taking a learning-inspired approach to better make use of a prediction within an algorithm, whereas we focus on learning the prediction outside of the target algorithm.
Our paper presents the ﬁrst general framework for efﬁciently learning useful predictors.
Data-driven algorithm design is a related area that has seen more learning-theoretic effort [19, 6, 3].
At a high-level, it often studies tuning parameters such as the gradient descent step-size [19] or settings of branch and bound [5], whereas the predictors in algorithms with predictions guess the sequence in an online algorithm [22] or the actual outcome of the computation [16].
The distinction can be viewed as terminological, since a prediction can be viewed as a parameter, but it can mean that in our settings we have full information about the loss function since it is typically some discrepancy between the full sequence or computational outcome and the prediction.
In contrast, in data-driven algorithm design getting the cost of each parameter often requires additional computation, leading to (semi-)bandit settings [7].
A more salient difference is that data-driven algorithm design guarantees compete with the parameter that minimizes average cost but do not always quantify the improvement attainable via learning; in algorithms with predictions we do generally quantify this improvement with an upper bound on the cost that depends on the prediction quality, but we usually only compete with the parameter that is optimal for prediction quality, which is not always cost-optimal.
We do adapt data-driven algorithm design tools like dispersion [6] for algorithms with predictions.
Our two-step approach to providing guarantees for algorithms with predictions is inspired by the
Average Regret-Upper-Bound Analysis (ARUBA) framework [25] for studying meta-learning [18].
Instead of instances they have tasks with different data-points, and the upper bounds are on learningtheoretic quantities such as regret rather than computational costs such as runtime.
Mathematically,
ARUBA takes advantage of similar structure in the regret-upper-bounds that we ﬁnd in algorithms with predictions, namely that the upper bounds encode some measure of the quality of the prediction (in their case an initialization for gradient descent) via a comparison to the ground-truth (in their case the optimal parameter).
However, whereas in ARUBA the need to know this optimal parameter after seeing a task is a weakness that does not hold in practice, in algorithms with predictions the corresponding quantity—the feedback listed in Table 1—is generally known after seeing the instance.
Framework overview and application to bipartite matching
In this section we outline the theoretical framework for designing algorithms and proving guarantees for learned predictors.
As an illustrative example we will use the Hungarian algorithm for bipartite matching, for which Dinitz et al.
[16] demonstrated an instance-dependent upper bound on the running time using a learned dual vector.
Along the way, we will show an improvement to their sample complexity bound together with the ﬁrst online results for this setting.
Bipartite matching:
For a bipartite graph on n nodes and m edges, min-weight perfect matching (MWPM) asks for the perfect matching with the least weight according to edge-costs c ∈
Z m ≥ 0 .
A common approach here is the Hungarian algorithm, a convex optimization-based approach for which
Dinitz et al.
[16] showed a runtime bound of ˜ O
( m √ n min {(cid:107) x − x ∗ ( c ) (cid:107) 1 , √ n } ) , where x ∈ Z n initializes the duals in a primal-dual algorithm and
x ∗ ( c ) ∈
Z n is dual of the optimal solution; note that the latter is obtained for free after running the Hungarian method.
Step 1 - Upper bound:
The ﬁrst step of our approach is to ﬁnd a suitable function U t ( x ) of the prediction x that (a) upper bounds the target algorithm’s cost C t ( x ) , (b) can be constructed completely once the algorithm terminates, and (c) can be efﬁciently optimized.
These qualities allow learning the predictor in the second step.
The requirements are similar to those of ARUBA for showing results for meta-learning [25], although there the quantity being upper-bounded was regret, not algorithmic cost.
method AlgorithmWithPrediction that takes advice from X and returns upper bounds U t on its cost.
The goal of OnlineAlgorithm is low regret over the sequence U t , so that on-average C t is upper bounded by the smallest possible average of U t , up to some error decreasing in T .
For speciﬁc instantiations of algorithms and feedback see Table 1.
initialize x 1 ∈ X for OnlineAlgorithm for instance t = 1 , . .
, T do obtain instance I t run AlgorithmWithPrediction( I t , x t ) suffer cost C t ( x t ) ≤ U t ( x t ) get feedback to construct upper bound
U t x t +1 ← OnlineAlgorithm( { U t } ts =1 , x 1 )
Many guarantees for algorithms with predictions are already amenable to being optimized, although we will see that they can require some massaging in order to be useful.
In many cases the guarantee is a distance metric between the prediction x and some instance-dependent perfect prediction x ∗ , which is convex and thus straightforward to learn.
This is roughly true of our bipartite matching example, although taking the minimum of a constant and the distance (cid:107) x − x ∗ ( c ) (cid:107) 1 between the predicted and actual duals makes the problem non-convex.
However, we can further upper bound their result by ˜
O ( m √ n (cid:107) x − x ∗ ( c ) (cid:107) 1 ) ; note that Dinitz et al.
[16] also optimize this quantity, not the tighter upper bound with the minimum.
While this might seem to be enough for step one, Dinitz et al.
[16] also require the prediction x to be integral, which is difﬁcult to combine with standard online procedures.
In order to get around this issue, we show that rounding any nonnegative real vector to the closest integer vector incurs only a constant multiplicative loss in terms of the (cid:96) 1 -distance.
Claim 3.1.
Given any vectors x ∈ Z n and y ∈ R n , let ˜y ∈
Z n be the vector whose elements are those of y rounded to the nearest integer.
Then (cid:107) x − ˜y
(cid:107) 1 ≤ 2 (cid:107) x − y (cid:107) 1 .
Proof.
Let S ⊂ [ n ] be the set of indices i ∈ [ n ] for which x
[ i ] ≥ y
[ i ] ⇐⇒
˜y
[ i ] = (cid:100)
y
[ i ] (cid:101) .
For i ∈ [ n ] \ S we have | x
[ i ] − y
[ i ] | ≥ 1 / 2 ≥ | ˜y [ i ] − y
[ i ] | so it follows by the triangle inequality that
[Equation]
Combining this projection with the convex relaxation above and the result of Dinitz et al.
[16] shows that for any predictor x ∈ R n we have (up to afﬁne transformation) a convex upper bound U t ( x )
= (cid:107) x − x ∗ ( c t ) (cid:107) 1 on the runtime of the Hungarian method, as desired.
We now move to step two.
Step 2 - Online learning: Once one has an upper bound U t on the cost, the second component of our approach is to apply standard online learning algorithms and results to these upper bounds to obtain guarantees for learning predictions.
In online learning, on each of a sequence of rounds t = 1 , . . .
, T we predict x t ∈ X and suffer U t ( x t ) for some adversarially chosen loss function U t : X (cid:55)→ R that we then observe; the goal is to use this information to minimize regret (cid:80)
Tt =
1 U t ( x t ) − min x ∈X U t ( x ) , with the usual requirement being that it is sublinear in T and thus decreasing on average over time.
For bipartite matching, we can just apply regular projected online (sub)gradient descent ( OGD ) to losses U t ( x )
= (cid:107) x − x ∗ ( c t ) (cid:107) 1 , i.e. the update rule x t +1 ← arg min x ∈X α (cid:104)∇ U t ( x t ) , x (cid:105) + 1 2 (cid:107) x
(cid:107) 2 2 for appropriate step-size α > 0 ; as shown in Theorem 3.1, this yields sublinear regret via a textbook result.
The simplicity here is the point: by relegating as much of the difﬁculty as we can to obtaining an easyto-optimize upper bound in step one, we make the actual learning-theoretic component easy.
However, as we show in the following sections, it is not always easy to obtain a suitable upper bound, nor is it always obvious what online learning algorithm to apply, e.g. if the upper bounds are non-convex.
Our use of online learning is motivated by three factors: (1) doing well on non-i.i.d.
instances is important in practical applications, e.g. in job scheduling where resource demand changes over time; (2) its extensive suite of algorithms lets us use different methods to tailor the approach to speciﬁc settings and obtain better bounds, as we exemplify via our use of exponentiated gradient over the simplex geometry in Section 4 and KT-OCO over unbounded Euclidean space in Section 5; (3) the existence of classic online-to-batch procedures for converting regret into sample complexity guarantees [12], i.e. bounds on the number of samples needed to obtain an ε -suboptimal predictor w.p. ≥ 1 − δ .
While online-tobatch conversion can be suboptimal [20], as we show in Theorems 3.1, B.2, and D.1 its application to various graph algorithms with predictions problems improves upon existing sample complexity results.
For completeness, we formalize online-to-batch conversion as Lemma A.1 in the Appendix.
We now show how to apply the second online learning step to bipartite matching by improving upon the result of Dinitz et al.
[16] in Theorem 3.1; the improvement is the entirely new regret bound against adversarial cost vectors and a ˜ O ( n ) lower sample complexity.
Note how the proof needs only their existing algorithmic contribution, Claim 3.1, and some standard tools in online convex optimization.
Theorem 3.1.
Suppose we have a ﬁxed bipartite graph with n ≥ 3 vertices and m ≥ 1 edges.
For any cost vector c ∈ Z m ≥ 0 and any dual vector x ∈ R n there exists an algorithm for MWPM that runs in time
[Equation]
for U ( x ) = (cid:107) x − x ∗ ( c ) (cid:107) 1 , where x ∗ ( c ) the optimal dual vector associated with c . 2.
There exists a poly-time algorithm s.
t. for any δ, ε > 0 and distribution D over integer m -vectors with (cid:96) ∞ -norm ≤ C it takes O (cid:16)(cid:0) Cnε (cid:1) 2 log 1 δ (cid:17) samples from D and returns ˆx s.t. w.p. ≥ 1 − δ :
[Equation]
Let c 1 , . . .
c
T ∈
Z
m
≥
be an adversarial sequence of m -vectors
with
(cid:96)
∞ -norm ≤
C
Then
OGD
with appropriate step-size has
regret
[Equation]
Proof.
The ﬁrst result follows by combining Dinitz et al.
[16, Theorem 13] with Claim 3.1.
For the third result, let x t be the sequence generated by running OGD [46] with step size C/ √ 2 T on the losses
U t ( x ) = (cid:107) x − x ∗ ( c t ) (cid:107) 1 over domain [ − C, C ] n .
Since these losses are √ n -Lipschitz and the duals are C √ n -bounded in Euclidean norm the regret guarantee follows from Shalev-Shwartz [44, Corollary 2.7].
For the second result, we apply standard online-to-batch conversion to the third result, i.e. we draw T = Ω (cid:16)(cid:0) Cnε (cid:1) 2 log 1 δ (cid:17) samples c t , run OGD as above on the resulting losses U t , and set ˆx = 1 T (cid:80)
Tt =1 x t to be the average of the resulting predictions x t .
The result follows by Lemma A.1.
This concludes an overview of our two-step approach for obtaining learning guarantees for algorithms with predictions.
To summarize, we propose to (1) obtain simple-to-optimize upper bounds U t ( x ) on the cost of the target algorithm on instance t as a function of prediction x and (2) optimize
U t ( x ) using online learning.
While conceptually simple, even in this illustrative example it already improves upon past work; in the sequel we demonstrate further results that this approach makes possible.
Note that, like Dinitz et al.
[16], we are also able to generalize Theorem 3.1 to b -matchings, which we do in Appendix B; another advantage of our approach is that it lets us prove online and statistical learning in the case where the demand vector b varies across instances rather than staying ﬁxed as in Dinitz et al.
Finally, in Appendix D we also improve upon the more recent learning-theoretic results of Chen et al.
[13] for related graph algorithms with predictions problems.
Predicting requests for page migration
Equipped with our two-step approach for deriving guarantees for learning predictors, we investigate several more important problems in combinatorial optimization, starting with the page migration problem.
Our results demonstrate that even for learning such simple predictors there are interesting technical challenges in deriving a learnable upper bound.
Nevertheless, once this is accomplished the second step of our approach is again straightforward.
Page migration: Consider a server that sees a sequence of requests s [1] , . . .
, s [ n ] from metric space ( K , d ) and at each timestep decides whether to change its state a [ i ] ∈ K at cost Dd ( a [ i − 1] , a [ i ] ) for some D > 1 ; it then suffers a further cost d ( a [ i ] , s [ i ] ) .
The online page migration (OPM) problem is then to minimize the cost to the server.
Recently, Indyk et al.
[22] studied a setting where we are given a sequence of predicted points ˆ s [1] , . .
, ˆ s [ n ] ∈ K to aid the page migration algorithm.
They show that if there exists γ, q ∈ (0 , 1) s.t. γD ∈ [ n ] and for any i ∈ [ n ] we have (cid:80) i + γD − 1 j = i
[Equation]
then there exists an algorithm with competitive ratio (1 + γ )(1 + O ( q )) w.r.t.
to the ofﬂine optimal.
This algorithm depends on γ but not q , so we study the setting where γ is ﬁxed.
Deriving an upper bound: As in the previous section, the predictions are discrete, so to use our approach we must convert it into a continuous problem.
As we have ﬁxed γ , the competitive ratio is an afﬁne function of the following upper bound on q :
[Equation]
We assume that the set of points K is ﬁnite with indexing k = 1 , . . .
, |K| and use this to introduce our continuous relaxation, a natural randomized approach converting the problem of learning a prediction into n experts problems on |K| experts.
For each j ∈ [ n ] we deﬁne a probability vector p [ j ] ∈ (cid:52) |K| governing the categorical r.v.
ˆ s [ j ] , i.e. Pr { ˆ s [ j ] = k } = p [ j,k ] ∀ k ∈ K .
Under these distributions the expected competitive ratio will be (1 + γ )(1 + O ( E ˆ s ∼ p Q
(ˆ s, s ))) , for p the product distribution of the vectors p j .
Note that forcing each p j to be a one-hot vector recovers the original approach with no loss, so optimizing E ˆ s ∼ p Q
(ˆ s, s ) over p ∈ (cid:52) n |K| would ﬁnd a predictor that ﬁts the original result.
However, E ˆ s ∼ p Q (ˆ s, s ) is not convex in p .
The simplest relaxation is to replace the maximum by summation, but this leads to a worst-case bound of O (cid:16) nγD (cid:17) .
We instead bound E ˆ s ∼ p Q
(ˆ s, s ) —and thus also the expected competitive ratio—by a function of the following maximum over expectations:
[Equation]
where s [ j,k ] = 1 s [ j ] = k ∀ k ∈ K , i.e. s [ j ] encodes the location in K of the j th request.
As a maximum over n − γD + 1 convex functions this objective is also convex.
Note also that if U s ( p ) is zero—i.e. the probability vectors are one-hot and perfect—then E ˆ s ∼ p Q (ˆ s, s ) ≥ q will also be zero.
In fact, q is upper-bounded by a monotonically increasing function of U s ( p ) that is zero at the origin, but as this function is concave and non-Lipschitz (c.f. Figure 1) we incur an additive O (cid:16)
[Equation] [Equation]
γD (cid:17) loss to obtain an online-learnable upper bound.
This is formalized in the following result (proof in A.1).
Lemma 4.1.
There exist constants a < e, b < 2 /e
and a monotonically increasing function f : [0 , ∞ ) (cid:55)→ [0 , ∞ )
s.t. f (0)
=
and
[Equation]
We now have an convex bound on the competitive ratio for the OPM algorithm of Indyk et al.
For both this and bipartite matching we resorted to a relaxation of a discrete problem.
However, whereas before we only incurred a multiplicative loss (c.f. Claim 3.1), here we have an additive loss that makes the bound meaningful only for γD (cid:29) log n .
However, as the method we propose optimizes U s ( p ) , which bounds q with no additive error via the function f in Lemma 4.1, in-practice we may expect it to help minimize q in all regimes.
Note that the non-Lipschitzness near zero that prevents using f for formal regret guarantees comes from the poor tail behavior of Poisson-like random variables with small means, which we do not expect can be signiﬁcantly improved.
Learning guarantees: Having established an upper bound, in Theorem 4.2 we again show how a learning-theoretic result follows from standard online learning.
This time, instead of OGD we run exponentiated (sub)gradient ( EG ) [44], a classic method for learning from experts, on each of n simplices to learn the probabilities p
[ j ] ∀ j ∈ [ n ] .
The multiplicative update x t +1 ∝ x t (cid:12) exp( − α ∇ U t ( x t )) of EG is notable for yielding regret logarithmic in the size |K| of the simplices, which is important for large metric spaces.
Note that as the relaxation is randomized, our algorithms output a dense probability vector; to obtain a prediction for OPM we sample ˆ s t [ j ] ∼ p [ j ] ∀ j ∈ [ n ] .
Theorem 4.2.
Let ( K , d ) be a ﬁnite metric space.
For any request sequence s and any set of probability vectors p ∈ (cid:52) n |K| there exists an algorithm for OPM with expected competitive ratio
[Equation]
There exits a poly-time algorithm s
t. for any δ, ε > 0 and distribution D over request sequences s ∈ K n it takes O (cid:18)(cid:16) γDε (cid:17) 2 (cid:0) n 2 log |K| + log 1 δ (cid:1)(cid:19) samples from D and returns ˆp s.t. w.p. ≥ 1 − δ :
[Equation]
Let s 1 , . . .
, s T be an adversarial sequence of request sequences.
Then updating the distribution p t [ j ] over (cid:52) |K| at each timestep j ∈ [ n ] using EG with appropriate step-size has regret
[Equation]
Proof.
The ﬁrst result follows by combining Indyk et al.
[22, Theorem 1] with Lemma 4.1.
For the third let p t be generated by running n exponentiated gradient algorithms with step-size (cid:113) log |K| 2 γ 2 D 2 T on losses U s t ( p ) over (cid:52) n |K| .
Since these are γD -Lipschitz and the maximum entropy is log |K| , the regret follows by [44, Theorem 2.15].
For the second result, apply standard online-to-batch conversion to the third, i.e. draw T = Ω (cid:18)(cid:16) γDε (cid:17) 2 (cid:0) n 2 log |K| + log 1 δ (cid:1)(cid:19)
samples s t , run EG on U s t ( p ) as above, and set ˆp = 1 T (cid:80)
Tt =1 p t to be the average of the resulting actions.
The result follows by Lemma A.1.
As before, this result ﬁrst shows how the quantity of interest—here the competitive ratio—is upperbounded by an afﬁne function of some quality measure U s ( p ) , for which we then provide regret and statistical guarantees using online learning.
The difﬁculty deriving a suitable bound exempliﬁes the technical challenges that arise in learning predictors, and may also be encountered in other sequence prediction problems such as TCP [10].
Nevertheless, our approach does yield an online procedure that incurs only O ( log n γD ) additive error over Indyk et al.
[22] in the case of a perfect predictor and, unlike their work, we provide an algorithm for learning the predictor itself.
In Appendix C.2 we also show an auto-regressive extension which does not require learning a distribution for each timestep j ∈ [ n ] .
Learning linear predictors with instance-feature inputs
So far we have considered only ﬁxed predictors, either optima-in-hindsight in the online setting or a population risk minimizers for i.i.d.
data.
Actual instances can vary signiﬁcantly and so a ﬁxed predictor may not be very good, e.g. in the example of querying a sorted array it means always returning the same index.
In the online setting one can consider methods that adapt to dynamic comparators [46, 23, 37], which are also applicable to our upper bounds; however, these still need measures such as the comparator path-length to be small, which may be more reasonable in some cases but not all.
We instead study the setting where all instances come with instance-speciﬁc features, a natural and practical assumption [27, 30] that encompasses numerical representations of the instance itself—e.g. bits representing a query or a graph—or other information such as weather or day of the week.
These are passed to functions—e.g. linear predictors, neural nets, or trees—whose parameters can be learned from data.
We study linear predictors, which are often amenable to similar analyses as above since the composition of a convex and afﬁne function is convex.
For example, it is straightforward to extend the matching results to learning linear predictors of duals.
OPM is more challenging because the outputs must lie in the simplex, which can be solved by learning rectangular stochastic matrices.
Both sets of results are shown in Appendix C.
Notably, for page migration our guarantees cover the auto-regressive setting where the server probabilities are determined by a ﬁxed linear transform of past states.
Our main example will be online job scheduling via minimizing the fractional makespan [30], where we must assign each in a sequence of variable-sized jobs to one of m machines.
Lattanzi et al.
[30] provide an algorithm that uses predictions ˆw ∈ R m> 0 of “good” machine weights w ∈ R m> 0 to assign jobs based on how well ˆw corresponds to machine demand; the method has a performance guarantee of
O (log min { max i ˆw
[ i ] w [ i ] , m } ) .
They also discuss learning linear and other predictors, but without guarantees.
We study linear prediction of the logarithm of the machine weights, which makes the problem convex, and assume features lie in the f -dimensional simplex.
For simplicity we only consider learning the linear transform from features to predictors and not the intercept, as the former subsumes the latter.
For the online result, we use KT-OCO [38, Algorithm 1], a parameter-free subgradient method with update x t +1 ← 1+ (cid:80) ts =1 (cid:104) g s , x s (cid:105) t + (cid:80) ts =1 g s for g s = ∇ U s ( x s ) ; it allows us to not assume any bound on the machine weights and thus to compete with the optimal linear predictor in all of R m × f .
Theorem 5.1.
Consider online restricted assignment with m ≥ 1 machines [30, Section 2.1].
For predicted logits x ∈ R m there is an algorithm whose fractional makespan has competitive ratio
[Equation]
for U ( x ) = (cid:107) x − log w (cid:107) ∞ , where w ∈ R m> 0 are good machine weights [30, Section 3].
There exists a poly-time algorithm s.t.
for any δ, ε > 0 and distribution D over machine (weight, feature) pairs ( w , f ) ∈
R m> 0 × (cid:52) f s.t.
(cid:107)
log w (cid:107) ∞ ≤ B the algorithm takes O (cid:16)(cid:0) Bε (cid:1) 2 (cid:0) mf + log 1 δ (cid:1)(cid:17) samples from D and returns ˆA ∈ R m × f s.t. w.p. ≥ 1 − δ
[Equation]
Let ( w 1 , f 1 ) , . . .
, ( w T , f T ) ∈ R m> 0 × (cid:52) f be an adversarial sequence of (weights, feature) pairs.
Then for any A ∈ R m × f KT-OCO has regret
T
[Equation]
=
[Equation]
If we restrict to matrices with entries bounded by B then OGD with appropriate step-size has regret
[Equation]
Proof.
The ﬁrst result follows by substituting max i exp( x [ i ] )
w [ i ] for η in Lattanzi et al.
[30, Theorem 3.1] and upper bounding the maximum by the (cid:96) ∞ -norm.
For the third, since U t is 1-Lipschitz w.r.t.
the Euclidean norm we apply the guarantee for KT-OCO [38, Algorithm 1] using ε = 1 and the subgradients of (cid:107)
A t f t − log w t (cid:107) ∞ as rewards [38, Corollary 5].
The result for B -bounded A follows by applying
OGD with step-size B (cid:113) mf 2 T over (cid:107) A (cid:107) max ≤ B [44, Corollary 2.7].
Finally, the second result follows by applying online-to-batch conversion to the latter result, i.e. draw T = Ω (cid:16)(cid:0) Bε (cid:1) 2 (cid:0) mf + log 1 δ (cid:1)(cid:17) samples ( w t , f t ) , run OGD on the resulting losses (cid:107)
Af t − log w t (cid:107) ∞ as above, and set ˆA =
T (cid:80) Tt =
1 A t to be the average of the resulting actions A t .
The result follows by Lemma A.1.
This guarantee is the ﬁrst we are aware of for learning non-static predictors in the algorithms with predictions literature.
It demonstrates both how to extend ﬁxed predictor results to learning linear predictors—note that the former is recovered by having f t = 1 1 ∀ t —and how to handle unbounded predictor domains.
The ability to provide such guarantees is another advantage of our approach.
Tuning robustness-consistency trade-offs for scheduling and ski-rental
We turn to tuning robustness-consistency trade-offs, introduced in Lykouris and Vassilvitskii [34].
This trade-off captures the tension between following the predictions when they are good (consistency) and doing not much worse than the worst-case guarantee in either case (robustness).
In many cases, this trade-off can be made explicit, by a parameter λ ∈ [0 , 1] .
The setting of λ is crucial, yet previous work left the decision to the end-user.
Here we show that it is often eminently learnable in an online setting.
We then demonstrate how to accomplish a much harder task—tuning λ at the same time as learning to predict—on two related but technically very different variants of the ski-rental problem.
This meta-application highlights the applicability of our approach to non-convex upper bounds.
Robustness-consistency trade-offs: Most problems studied in online algorithms with predictions usually have existing worst-case guarantees on the competitive ratio, i.e. a constant γ ≥ 1 on how much worse (multiplicatively) a learning-free algorithm does relative to the ofﬂine optimal cost OPT t on instance t .
While the goal of algorithms with predictions is to use data to do better than this worst-case bound, an imperfect prediction may lead to much worse performance.
As a result, most guarantees strive to upper bound the cost of an algorithm with prediction x on an instance t as follows:
[Equation]
Here u t is some measure of the quality of x on instance t , λ ∈ [0 , 1] is a parameter, f is a monotonically increasing function that ideally satisﬁes f (0)
= 1 , and g t is a monotonically decreasing function that ideally satisﬁes g t (1)
=
γ OPT t .
A very common structure is f ( λ )
= 1 / (1 − λ ) and g t ( λ ) =
γλ OPT t .
For example, consider job scheduling with predictions, a setting where we are given n jobs and their predicted runtimes with total absolute error η and must minimize the sum of their completion times when running on a single server with pre-emption.
Here Kumar et al.
[29, Theorem 3.3] showed that a preferential round-robin algorithm has competitive ratio at most min (cid:110)
+2 η/n 1 − λ , 2 λ (cid:111) .
Thus if we know the prediction is perfect we can set λ = 0 and obtain the optimal cost (consistency); on the other hand, if we know the prediction is poor we can set λ =and get the (tight) worst-case guarantee of two (robustness).
Of course in-practice we often do not know how good a prediction is on a speciﬁc instance t ; we thus would like to learn to set λ , i.e. to learn how trustworthy our prediction is.
As a ﬁrst step, we can consider doing so when we are given a prediction for each instance and thus only need to optimize over λ .
For example, the just-discussed problem of job scheduling has competitive ratio upper-bounded by
U t ( λ ) = min (cid:110) 1+2 η t /n t 1 − λ , 2 λ (cid:111) for n t and η t the number of jobs and the prediction quality, respectively, on instance t .
Assuming a bound B on the average error makes U t Lipschitz, so we can apply the exponentially weighted average forecaster [28, Algorithm 1], also known as the exponential forecaster .
This algorithm, whose action at each time t + 1 is to sample from the distribution with density ρ t +1 ( · ) ∝ ρ 1 ( · ) exp( − α (cid:80) ts =
U s ( · )) , has the following regret guarantee (proof in A.2):
Corollary 6.1.
For the competitive ratio upper bounds U t of the job scheduling problem with average prediction error η/n t at most B the exponential forecaster with appropriate step-size has expected regret
[Equation]
Thus a standard learning method produces a sequence λ t that performs as well as the best λ asymptotically.
We next consider the more difﬁcult problem of simultaneously tuning λ learning to predict.
Ski-rental: We instantiate this challenge on ski-rental, in which each task t is a ski season with an unknown number of days n t ∈ Z ≥ 2 ; to ski each day, we must either buy skis at price b t or rent each day for the price of one.
The optimal ofﬂine behavior is to buy iff b t < n t , and the best algorithm has worst-case competitive ratio e/ ( e − 1) .
Kumar et al.
[29] and Bamas et al.
[10, Theorem 2] further derive an algorithm with the following robustness-consistency trade-off between blindly following a prediction x and incurring cost u t ( x ) =
b t 1 x>b t + n t 1 x ≤ b t or going with the worst-case guarantee:
[Equation]
Assuming a bound of N ≥ 2 on the number of days and B > 0 on the buy price implies that U t is bounded and Lipschitz w.r.t.
λ .
We can thus run exponentiated gradient on the functions U t to learn a categorical distribution over the product set
[ N ] × { δ/ 2 , . .
, 1 − δ/ 2 } for some δ s.t.
/δ ∈
Z ≥ 2 .
This yields the following bound on the expected regret (proof in A.3).
Corollary 6.2.
For the competitive ratio upper bounds U t of the discrete ski-rental problem the randomized exponentiated gradient algorithm with an appropriate step-size has expected regret
[Equation]
Thus via an appropriate discretization the sequence of predictions ( x t , λ t ) does as well as the joint optimum on this problem.
However, we can also look at a case where we are not able to just discretize to get low regret.
In particular, we consider the continuous ski-rental problem, where each day n t > 1 is a real number, and study how to pick thresholds x after which to buy skis, which has cost u t ( x ) = n t 1 n t ≤ x + ( b t + x )1 n t >x .
Note that x = 0 and x = N recovers the previous setting where our decision was to buy or not at the beginning.
For this setting, Diakonikolas et al.
[15] adapt an algorithm of Mahdian et al.
[35] to bound the cost as follows:
[Equation]
While the bound is simpler as a function of λ , it is discontinuous in x because u t is piecewiseLipschitz.
Since one cannot even attain sublinear regret on adversarially chosen threshold functions, we must make an assumption on the data.
In particular, we will assume the days are dispersed :
Deﬁnition 6.3.
A set of (possibly random) points n 1 , . .
. , n T ∈ R are β -dispersed
if ∀ ε ≥ T − β the expected number in any ε -ball is ˜ O ( εT ) , i.e. E max x ∈ [0 ,N ] | [ x ± ε ] ∩ { n 1 , . . .
, n T }| =
˜ O ( εT ) .
Dispersion encodes the stipulation that the days, and thus the discontinuities of u t ( x, λ ) , are not too concentrated.
In the i.i.d.
setting, a simple condition that leads to dispersion with β = 1 /is the assumption that the points are drawn from a κ -bounded distribution [6, Lemma 1].
Notably this is a strictly weaker assumption than the log-concave requirement of Diakonikolas et al.
[15] that they used to show statistical learning results for ski-rental.
Having stipulated that the ski-days are β -dispersed, we can show that it implies dispersion of the loss functions [6] and thus obtain the following guarantee for the exponential forecaster applied to U t ( x, λ ) (proof in A.4):
Corollary 6.4.
For cost upper bounds U t of the continuous ski-rental problem the exponential forecaster with an appropriate step-size has expected regret
[Equation]
Thus in two mathematically quite different settings of ski-rental we can directly apply online learning to existing bounds to not only learn online the best action for ski-rental, but to at the same time learn how trustworthy the best action is via tuning the robustness-consistency trade-off.
Conclusion and future work
The ﬁeld of algorithms with predictions has been successful in circumventing worst case lower bounds and showing how simple predictions can improve algorithm performance.
However, except for a few problem-speciﬁc approaches, the question of how to predict has largely been missing from the discussion.
In this work we presented the ﬁrst general framework for efﬁciently learning useful predictions and applied it to a diverse set of previously studied problems, giving the ﬁrst low regret learning algorithms, reducing sample complexity bounds, and showing how to learn the best consistency-robustness trade-off.
One current limitation is the lack of more general-case guarantees for simultaneously tuning robustness-consistency and learning the predictor, which we only show for ski-rental.
There are also several other avenues for future work.
The ﬁrst is to build on our results and provide learning guarantees for other problems where the algorithmic question of how to use predictions is already addressed.
Another is to try to improve known bounds by solving the problems holistically: developing easy-to-learn parameters in concert with developing algorithms that can use them.
Finally, there is the direction of identifying hard problems: what are the instances where no reasonable prediction can help improve an algorithm’s performance?
Acknowledgments
We thank Yilin Yan, Alexander Smola, Shinsaku Sakaue, and Taihei Oki for helpful discussion.
This material is based on work supported in part by the National Science Foundation under grants
CCF-1535967, CCF-1910321, IIS-1618714, IIS-1705121, IIS-1838017, IIS-1901403, IIS-2046613, and SES-1919453; the Defense Advanced Research Projects Agency under cooperative agreements
HR00112020003 and FA875017C0141; a Simons Investigator Award; an AWS Machine Learning
Research Award; an Amazon Research Award; a Bloomberg Research Grant; a Microsoft Research
Faculty Fellowship; an Amazon Web Services Award; a Facebook Faculty Research Award; funding from Booz Allen Hamilton Inc.; a Block Center Grant; and a Facebook PhD Fellowship.
Any opinions, ﬁndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reﬂect the views of any of these funding agencies.
[-for all t > 
-]
[-+
-]
[-We then have t E p max i ∈ [ n − γD +
-]
[-Let Λ = { kδ } (cid:98) 
-]
[-For b ∈ R n ≥ 
-]
[-for any δ, ε > 0 and any distribution D over (cost, demand) vector pairs in Z m ≥ 
-]
[-Z m ≥ 
-]
[-and any distribution D over (cost, demand, feature) vector triples in Z m ≥ 
-]
[-Z m ≥ 
-]
[-Finally, we discuss the the applicability of our framework to the results in Lindermayr and Megow [33], who study how to prioritize among n jobs by predicting the best permutation of them under weights w ∈ R n ≥ 0 and processing requirements p ∈ R n ≥ 
-]