Logarithmic Regret from Sublinear Hints

p T ) in the standard setting . In this work , we study the question of whether an algorithm really requires a hint at every time step . Somewhat surprisingly , we show that an algorithm can obtain O ( log T ) regret with just O ( p T ) hints under a natural query model ; in contrast , we also show that o ( p T ) hints can not guarantee better than ⌦ ( p T ) regret . We give two applications of our result , to the well-studied setting of optimistic regret bounds and to the problem of online learning with abstention . 1 Introduction . There has been a spate of work on improving the performance of online algorithms with the help of externally available hints . The goal of these works is to circumvent worst-case bounds and exploit the capability of machine-learned models that can potentially provide these hints . There have been two main lines of study . The first is for combinatorial problems , where the goal has been to be improve the competitive ratio of online algorithms ; problems considered here include ski-rental [ 9 , 17 ] , caching [ 13 , 19 , 26 ] , scheduling [ 1 , 12 , 17 , 22 ] , matching [ 16 , 18 ] , etc . The second is in the learning theory setting , where the goal has to been to improve the regret of online optimization algorithms . A series of recent papers showed how to achieve better regret guarantees , assuming that we have a hint about the cost function before the algorithm makes a choice . For many variants of the online convex optimization problem , works such as [ 25 , 8 , 2 , 3 ] studied the power of having prior information about a cost function . Works such as [ 29 ] have also studied improved regret bounds in partial information or bandit settings . In all these works , a desirable property is to ensure consistency , which demands better performance with better quality hints , and robustness , which guarantees a certain level of performance with poor quality or even adversarially bad hints . Recall the standard online optimization model [ 30 ] , which is a game between an algorithm and an adversary . In each round , the algorithm plays a point and the adversary responds with a cost function that is visible to the algorithm , and the cost in this round is measured by evaluating the cost function on the played point . In online linear optimization , the cost function is linear . The regret of an algorithm is the worst-case difference between the cost of the algorithm and the cost of an algorithm 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . that plays a fixed point in each round . A natural way to incorporate hints or prior information is to give the algorithm access to a hint about the cost function at a given round before it chooses a point to play at that round . In this sense , hints are present gratis [ 25 ] . The availability of a hint in each round seems natural in some settings , e.g. , when cost functions change gradually with time [ 25 , 4 ] ) . However , it can be prohibitive in many others , for instance , if hints are obtained by using expensive side information , or if they are generated by a running a computationally expensive ML model . Furthermore , hints can also be wasteful when the problem instance , or even a large sub-instance , is such that the algorithm can not really derive any substantial benefit from their presence . This leads to the question of strengthening the online learning with hints model by making it parsimonious . In this work , we pursue this direction , where we offer the algorithm the flexibility to choose to ask for a hint before it plays the point . It then becomes an onus on the algorithm to know when to ask for a hint and how to use it judiciously , while ensuring both consistency and robustness . The performance of such an algorithm is measured not only by its regret but also by the number of hints it uses . Our contributions . We now present a high level summary of our results . For ease of exposition , we will defer the formal statements to the respective sections . All our results are for the problem of online linear optimization ( OLO ) when the domain is the unit ` 2 ball ; the cost function at every step is defined using a cost vector ct ( and the cost or loss is the inner product of the point played with ct ) . We call a hint perfect if it is the same as the cost vector at that time step , good if it is weakly correlated with the cost vector , and bad otherwise . As our main result , we show that for OLO in which hints are guaranteed to be good whenever the algorithm asks for a hint , there is an efficient randomized algorithm that obtains O ( log T ) regret using only O ( p T ) hints . We extend our result to the case when |B| hints can be bad ( chosen in an oblivious manner , as we will discuss later ) , and give an algorithm that achieves a regret bound of O ( p |B| log T ) , while still asking for O ( p T ) hints . It is interesting to contrast our result with prior work : Dekel et al . [ 8 ] obtained an algorithm with O ( log T ) regret , when a good hint is available in every round . Bhaskara et al . [ 2 ] made this result robust , obtaining a regret bound of O ( p |B| log T ) , when there are |B| bad hints . Our result improves upon these works by showing the same asymptotic regret bounds , but using only O ( p T ) hints . Our result also has implications for optimistic regret bounds [ 25 ] ( where we obtain the same results , but with fewer hints ) and for online learning with abstention [ 24 ] ( where we can bound the number of abstentions ) . We also show two lower bounds that show the optimality of our algorithm . The first is regarding the minimum number of hints needed to get O ( log T ) regret : we show that any ( potentially randomized ) algorithm that uses o ( p T ) hints will suffer a regret of ⌦ ( p T ) . The second and more surprising result is the role of randomness : we show that any deterministic algorithm that obtains O ( log T ) regret must use ⌦ ( T/ log ( T ) ) hints , even if each of them is perfect . This shows the significance of having a randomized algorithm and an oblivious adversary . Finally we extend our results to the unconstrained OLO setting ( see Section 6 ) , where we design a deterministic algorithm to obtain O ( log3/2 T ) regret ( suitably defined for the unconstrained case ) when all hints are good , and a randomized algorithm to obtain O ( log T ) regret , which can be extended to the presence of bad hints . There are three aspects of our results that we find surprising . The first is even the possibility of obtaining O ( log T ) regret using only a sublinear number of hints . The second is the sharp threshold on the number of hints needed to obtain logarithmic regret ; the regret does not gracefully degrade when the number of hints is below O ( p T ) . The third is the deterministic vs randomized separation between the constrained and the unconstrained cases , when all queried hints are good . We now present some intuition why our result is plausible . Consider the standard “ worst-case ” adversary for OLO : random mean-zero costs . This case is “ hard ” because the learner achieves zero expected cost , but the competitor achieves p T total cost . However , if we simply play a hint ht on O ( p T/↵ ) rounds , each such round incurs ↵ cost , which is enough to cancel out the p T regret while making only O ( p T ) hint queries . Thus , the standard worst-case instances are actually easy with hints . More details on this example and an outline of our algorithm are provided at the start of Section 3 . Organization . Section 2 provides the necessary background . The main algorithm and analysis for the constrained OLO case are in Section 3 . The extensions and applications of this can be found in Section 4 . Section 5 contains the lower bounds and Section 6 contains the algorithms and analyses for the unconstrained case . All missing proofs are in the Supplementary Material . 2 Preliminaries . Let k · k denote the ` 2-norm and Bd = { x 2 Rd | kxk 1 } denote the unit ` 2 ball in Rd . We use the compressed sum notation and use c1 : t to denote Pt i=1 ci and kck21 : t to denote Pt i=1 kcik2 . Let ~c = c1 , . . . , cT be a sequence of cost vectors . Let [ T ] = { 1 , . . . , T } . OLO problem and Regret . The constrained online linear optimization ( OLO ) problem is modeled as a game over T rounds . At each time t 2 [ T ] , an algorithm A plays a vector xt 2 Bd , and then an adversary responds with a cost vector ct 2 Bd . The algorithm incurs cost ( or loss ) hxt , cti at time t. The total cost incurred by the algorithm is costA ( ~c ) = PT t=1hxt , cti . The regret of the algorithm A with respect to a ‘ comparator ’ or benchmark vector u 2 Bd is RA ( u , ~c ) = costA ( ~c ) costAu ( ~c ) = TX t=1 hxt u , cti , where Au is the algorithm that always plays u at every time step . The regret of an algorithm A is its worst-case regret with respect to all u 2 Bd : RA ( ~c ) = sup u2Bd RA ( u , ~c ) . Hints and query cost . Let ↵ > 0 be fixed and known . In this paper we consider the OLO setting where , at any round t before choosing xt , an algorithm A is allowed to obtain a hint ht 2 Bd . If hht , cti ↵kctk2 , we say that the hint is ↵-good . If A opts to obtain a hint at time t , then it incurs a query cost of ↵kctk2 ; the query cost is 0 if no hint was obtained at time t. The definition of regret stays the same and we denote it by RA , ↵ ( · ) . The total query cost of A is given by QA , ↵ ( ~c ) =PT t=1 t · ↵kctk2 where t is an indicator function used to denote whether A queried for a hint at time t. Note that the algorithm does not actually know the query cost for a round until the end of the round . If A is a randomized algorithm , the notions of expected regret and expected query cost follow naturally . We consider the setting when the adversarial choice of the hint ht and cost vector ct at time t is oblivious to whether the algorithm queries for a hint at time t but can depend adaptively on all previous decisions . More generally , we consider the case that some subset of the hints are “ bad ” in the sense that hct , hti < ↵kctk2 ; we let B denote the set of such indices t. Although we assume ↵ is known to our algorithms , we do not assume any information about B . Further , our algorithm is charged ↵kctk2 for querying a hint even if the hint was bad .

Summary: This paper considers the Online Linear Optimization (OLO) problem with additional hints (predictions) in which at every round $t\in[T]$, the algorithm can decide to receive the hint $h_t$ to help choose an action $x_t$. Upon committing to the action, a loss vector $c_t$ is revealed and the algorithm incurs the cost $\langle c_t,x_t \rangle$. In case the algorithm had received the hint, it also incurs a query cost of $\alpha |c_t|^2$ where $\alpha$ is a pre-specified constant. While prior works had shown that having access to good hints at every round leads to improved logarithmic regret bound (instead of the standard $O(\sqrt{T})$ regret bound of OLO), this work shows that achieving such improved regret bounds is possible with only $O(\sqrt{T})$ hints. Moreover, they show that $o(\sqrt{T})$ hints would not be enough, and the $\Omega (\sqrt{T})$ regret bound can not be avoided in that case. Finally, some extensions of the framework (including the one which allows for bad hints) and the corresponding regret bound for each setting are provided.