SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yossi Azar  
Tel Aviv University  
azar@tau.ac.il
Debmalya Panigrahi  
Duke University  
debmalya@cs.duke.edu
Noam Touitou*  
Amazon  
noamtwx@gmail.com
In recent years, there has been an increasing focus on designing online algorithms with (machine-learned) predictions. The ideal learning-augmented algorithm is comparable to the optimum when given perfect predictions (consistency), to the best online approximation for arbitrary predictions (robustness), and should interpolate between these extremes as a smooth function of the prediction error. In this paper, we quantify these guarantees in terms of a general property that we call discrete-smoothness and achieve discrete-smooth algorithms for online covering, specifically the facility location and set cover problems. For set cover, our work improves the results of Bamas, Maggiori, and Svensson (2020) by augmenting consistency and robustness with smoothness guarantees. For facility location, our work improves on prior work by Almanza et al. (2021) by generalizing to nonuniform costs and also providing smoothness guarantees by augmenting consistency and robustness.
The field of learning-augmented online algorithms has gained rapid prominence in recent years. The basic premise is to provide an online algorithm with additional (machine-learned) predictions about the future to help bypass worst-case lower bounds. Since machine-learned predictions can be noisy in general, a key desideratum of the model is that the competitive ratio of the online algorithm should degrade gracefully with prediction error. In particular, the cost of the algorithm should be bounded against that of the predicted solution (called consistency) or that of an online algorithm without predictions (called robustness) and should smoothly interpolate between the two with increase in prediction error (called smoothness). (The terms consistency and robustness were originally coined by Purohit, Svitkina, and Kumar [38].) While robustness and consistency are problem-independent notions, smoothness depends on prediction error which has been defined in a problem-specific manner. In this paper, we introduce a novel, problem-independent notion of smoothness called discrete-smoothness that applies to any combinatorial problem. As illustrative applications of this new framework, we design discrete-smooth (learning-augmented) algorithms for two classic problems, facility location and set cover, which improve and generalize previous results for these problems due to Almanza et al. (NeurIPS '21 [1]) and Bamas et al. (NeurIPS '20 [11]).
First, we introduce discrete-smoothness. Suppose we are given a problem instance of size  $n$ . Let OPT be a solution for this instance. (The reader may think of OPT as an optimal solution, although our guarantees will hold for any feasible solution.) Let the predicted solution be  $S$ . Ideally,  $S = \mathrm{OPT}$ ; therefore, in general, OPT comprises two parts: the predicted part  $\mathrm{OPT}|_S \coloneqq \mathrm{OPT} \cap S$  and the unpredicted part  $\mathrm{OPT}|_{\overline{S}} \coloneqq \mathrm{OPT} \setminus S$ . On the predicted part  $\mathrm{OPT}|_S$ , the algorithm has a meaningful signal from the prediction but the noise in the signal is given by the overprediction  $s_\Delta \coloneqq |S \setminus \mathrm{OPT}|$ . Naturally, the competitive ratio of the algorithm on this part will degrade with increase in this noise. On the unpredicted part  $\mathrm{OPT}|_{\overline{S}}$ , the algorithm does not have any signal from the prediction and
*This paper does not relate to the author's work at Amazon.
cannot hope for a better competitive ratio than that of an online algorithm without prediction. Slightly abusing notation, we use  $\mathrm{OPT}|_{S}$ ,  $\mathrm{OPT}|_{\overline{S}}$  to denote both the aforementioned sets of items and their total cost; putting the two together, a learning-augmented algorithm ALG should satisfy
$$
\mathrm {A L G} \leq O (f (s _ {\Delta})) \cdot \mathrm {O P T} | _ {S} + O (f (n)) \cdot \mathrm {O P T} | _ {\bar {S}}, \tag {1}
$$
where  $O(f(\cdot))$  is the competitive ratio without prediction. We call the property of Equation (1) discrete-smoothness.
Let us first argue that Equation (1) recovers consistency and robustness. Consistency follows from setting  $S = \mathrm{OPT}$ ; then, Equation (1) demands a constant approximation to OPT. Similarly, robustness follows from the fact that for any  $S$ , the right hand side of Equation (1) is at most  $O(f(n)) \cdot \mathrm{OPT}$ .
Next, we show that the two terms  $f(s_{\Delta})$  and  $f(n)$  in Equation (1) are the best possible. For the first term, consider a prediction  $S$  comprising the entire instance (of size  $n$ ); in this case, we cannot hope for the better than  $f(n)$ -competitive algorithm; thus,  $f(s_{\Delta})$  is necessary in the first term. And, for the second term, consider an empty prediction  $S = \emptyset$ , in which case we again cannot hope for a better than  $f(n)$ -competitive algorithm; thus,  $f(n)$  is necessary in the second term. Note that the asymmetry between these two terms is necessary: specifically,  $f(n)$  cannot be replaced by  $f(|\mathrm{OPT} \setminus S|)$  since that would imply an  $f(\mathrm{OPT})$ -competitive online algorithm when  $S = \emptyset$ . This is impossible, e.g., for the set cover problem.
A technical subtlety of the definition of discrete-smoothness (Equation (1)) is that given a fixed prediction  $S$ , the minimum value of the right hand side might actually be a solution OPT that is different from an optimal solution to the problem instance. So, although the solution OPT is intuitively an optimal solution, we require that a discrete-smooth algorithm satisfy Equation (1) for all feasible solutions OPT, and not just optimal solutions.
We apply discrete-smoothness to the classic problems of online facility location and set cover. For these problems, we obtain results that improve on prior work. We describe these next.
Online Facility Location with Predictions. In the online facility location problem, we are given offline a metric space  $(X,\delta)$  of  $m\coloneqq |X|$  points, where each point  $\nu \in X$  has an associated facility opening cost  $o_{\nu}\geq 0$ . On receiving an online request for a client at some location  $x\in X$ , the online algorithm must connect the client to an open facility at some location  $\nu \in X$  incurring connection cost  $\delta (x,\nu)$ . At any time, the algorithm is also allowed to open a facility at any location  $\nu \in X$  by incurring the opening cost  $o_{\nu}$ . (Note that a client cannot update her connection even if a closer facility is opened later.) The total cost of the algorithm is the sum of opening costs of opened facilities and connection costs of clients.
The first result for the online facility location problem is due to Meyerson [33] who obtained a randomized algorithm with a competitive ratio of  $O(\log n)$  for  $n$  requests. This result was first derandomized [18], and later the competitive ratio slightly improved to  $O\left(\frac{\log n}{\log \log n}\right)$  [19], by Fotakis. This latter bound is asymptotically tight. More recently, the online facility location problem has been considered in the context of machine-learned predictions (OFLP) by several papers [20, 1, 22]. Of these, the work of Almanza et al. [1] is the closest to our work (the other papers use metric error measures that are incomparable to our results). In [1], the offline input additionally contains a predicted solution of facilities  $S \subseteq X$ , where we denote  $|S| = s$ . By restricting the available facilities to the predicted set, they obtained an  $O(\log s)$ -competitive algorithm for uniform facility opening costs, under the condition that OPT  $\subseteq S$ .
We improve and generalize the Almanza et al. work by giving a discrete-smooth algorithm for the OFLP problem, i.e., an algorithm ALG that satisfies Equation (1):
Theorem 1.1. There is an algorithm ALG for online (nonuniform) facility location with a predicted solution  $S$  that satisfies for every solution OPT
$$
\mathrm {A L G} \leq O \left(\log s _ {\Delta}\right) \cdot \mathrm {O P T} | _ {S} + O (\log n) \cdot \mathrm {O P T} | _ {\bar {S}}, \tag {2}
$$
where  $s_{\Delta}$  is the number of facilities in  $S \backslash \mathrm{OPT}$  and  $n$  is the number of online requests. Here,  $\mathrm{OPT}|_S$  (resp.,  $\mathrm{OPT}|_{\overline{S}}$ ) represents the sum of opening costs of facilities in  $\mathrm{OPT} \cap S$  (resp.,  $\mathrm{OPT} \setminus S$ ) and connection costs of all clients connecting to facilities in  $\mathrm{OPT} \cap S$  (resp.,  $\mathrm{OPT} \setminus S$ ).
This generalizes and improves the Almanza et al. result in three ways:
- The result is generalized from uniform facility opening costs to arbitrary (nonuniform) costs. In fact, even for the online facility location problem (without prediction), we get an  $O(\log m)$ -competitive algorithm for arbitrary (nonuniform) facility opening costs — previously, Almanza et al. only established this for uniform costs.
- The assumption that OPT  $\subseteq S$ , i.e., the prediction contains the entire solution, is no longer required.
- If OPT  $\subseteq S$  (i.e., under the assumption of the Almanza et al. result), the competitive ratio improves from  $O(\log s)$  to  $O(\log s_{\Delta})$ . That is, the dependence is only on the prediction error and not the entire prediction.
In some situations, the length of the request sequence  $n$  can exceed the size of the metric space  $m$ . To address this situation, we show that  $n$  can be replaced by  $m$  in the above result:
Theorem 1.2. There is an algorithm ALG for online (nonuniform) facility location with a predicted solution  $S$  that satisfies for every solution OPT
$$
\mathrm {A L G} \leq O (\log s _ {\Delta}) \cdot \mathrm {O P T} | _ {S} + O (\log m) \cdot \mathrm {O P T} | _ {\bar {S}}, \tag {3}
$$
where  $m$  is the number of facilities in the metric space overall.
Online Set Cover with Predictions. In the online set cover problem, we are given offline a universe of elements  $E$  and  $m$  sets defined on them  $U \subseteq 2^{E}$  with nonnegative costs. In each online step, we get a new element  $e \in E$ . If  $e$  is not already covered by the current solution, then the algorithm must add a new set from  $U$  that contains  $e$  to its solution. The total cost of the algorithm is the sum of costs of all sets in its solution.
Alon et al. [3] gave the first algorithm for the online set cover problem by introducing the online primal dual method, and obtained a competitive ratio of  $O(\log m \log n)$  where  $n$  denotes the number of requests. They also proved an almost matching lower bound of  $\Omega \left( \frac{\log m \log n}{\log \log m + \log \log n} \right)$ . Bamas, Maggiori, and Svensson [11] extended their work to online set cover with predictions (OSCP), where the offline input additionally contains a predicted solution of sets  $S \subseteq U$ . They established consistency and robustness bounds for this problem by adapting the online primal dual method to use the predicted solution. The cost of their algorithm is bounded by the minimum of  $O(\log n)$  times the cost of the prediction and  $O(\log m \log n)$  times the optimal cost. However, one cannot achieve smoothness through their work without choosing a trust parameter correctly in advance of the input.
We obtain a discrete-smooth algorithm for the OSCP problem, thereby giving the first algorithm for OSCP that goes beyond only consistency and robustness and achieves a smoothness guarantee:
Theorem 1.3. There is an algorithm ALG for online set cover with a predicted solution  $S$  that satisfies for every solution OPT
$$
\mathrm {A L G} \leq O (\log s _ {\Delta} \log n) \cdot \mathrm {O P T} | _ {S} + O (\log m \log n) \cdot \mathrm {O P T} | _ {\bar {S}}, \tag {4}
$$
where  $s_{\Delta}$  is the number of sets in  $S \backslash \mathrm{OPT}$ . Here,  $\mathrm{OPT}|_S$  (resp.,  $\mathrm{OPT}|_{\overline{S}}$ ) represents the sum of costs of sets in  $\mathrm{OPT} \cap S$  (resp.,  $\mathrm{OPT} \setminus \overline{S}$ ).
1.2 Our Techniques: A Framework for Discrete-Smooth Algorithms
At a high level, our framework merges two online algorithms to obtain a discrete-smooth algorithm. The algorithms differ in the guarantees they provide. The first algorithm  $\mathrm{ALG}_1$  gets a sharper competitive ratio of  $O(f(s))$  but against the optimal solution restricted to the prediction  $S$ . The second algorithm  $\mathrm{ALG}_2$  has the standard competitive ratio of  $O(f(n))$  but against the unconstrained optimum OPT. The main challenge in the combiner algorithm (call it ALG) is to decide how to route online requests to the two algorithms. The natural choice would be to decide this based on whether  $\mathrm{OPT}|_S$  or  $\mathrm{OPT}_{\overline{S}}$  serves the request in OPT: in the first case, the request should be routed to  $\mathrm{ALG}_1$  and in the second case, it should be routed to  $\mathrm{ALG}_2$ . But, of course, we do not know OPT and therefore don't know  $\mathrm{OPT}|_S$  and  $\mathrm{OPT}_{\overline{S}}$ .
Before we describe the combiner strategy, consider the properties that these algorithms need to satisfy.
- First, consider the subset of requests served by  $\mathrm{OPT}|_{S}$ . Intuitively,  $\mathrm{ALG}_1$  should be competitive on these requests, which means that we need a stronger property from  $\mathrm{ALG}_1$  that its cost on any subset of requests is competitive against the optimal solution for this subset. We call this property subset competitiveness. $^2$  Symmetrically, subset competitiveness of  $\mathrm{ALG}_2$  ensures that it is competitive on the requests in  $\mathrm{OPT}|_{\overline{S}}$ .
- Next, we need a guarantee on the cost of  $\mathrm{ALG}_1$  on  $\mathrm{OPT}|_{\overline{S}}$ , and symmetrically, of  $\mathrm{ALG}_2$  on  $\mathrm{OPT}|_{S}$ . For this, we first augment  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  to address the prize-collecting version of the original problem, where each online request can be ignored at a penalty cost. (Note that this is more general than the original problem where every online request must be served, since the latter can be recovered by setting the penalties to be infinitely large.) Setting the penalties appropriately, we ensure that the total penalty of the requests in  $\mathrm{OPT}|_{S}$  is bounded against the cost of  $\mathrm{ALG}_1$  on those requests (similarly for  $\mathrm{OPT}|_{\overline{S}}$ ).
- Finally, we require that the cost of  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  on any set of requests is bounded against the total penalty of the requests. We call this strengthened competitiveness w.r.t. penalties the Lagrangian property<sup>3</sup>. Note that this ensures that the cost of  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  on  $\mathrm{OPT}|_{\overline{S}}$ ,  $\mathrm{OPT}|_S$  are respectively bounded.
Now, we give the formal definition of Lagrangian subset-competitiveness that we motivated above. We use  $\mathrm{ALG}(Q'|Q)$  to refer to the total cost of ALG incurred when addressing a subset  $Q' \subseteq Q$  as part of running on an input  $Q$ . For any prize collecting solution SOL for input  $Q$ , we separate its total cost into  $\mathrm{SOL}^b(Q)$  (buying cost) and  $\mathrm{SOL}^p(Q)$  (penalty cost). We formalize the Lagrangian subset-competitiveness property below:
Definition 1.4 (Lagrangian subset-competitive algorithm). Let ALG be a randomized prize-collecting algorithm running on an input  $Q$ . For any competitive ratio  $\beta$ , we say that ALG is Lagrangian  $\beta$ -subset-competitive if for every subset  $Q' \subseteq Q$  we have
$$
\mathbb {E} \left[ \operatorname {A L G} \left(Q ^ {\prime} \mid Q\right) \right] \leq \beta \cdot \operatorname {O P T} ^ {b} \left(Q ^ {\prime}\right) + O (1) \cdot \operatorname {O P T} ^ {p} \left(Q ^ {\prime}\right) \tag {5}
$$
If in the equation above we replace the unconstrained optimum (OPT) by the optimal solution that can only use the prediction  $S$ , we say that ALG is Lagrangian  $\beta$ -subset-competitive w.r.t.  $S$ .
We now give the combiner algorithm:
Algorithm 1: Smooth merging framework (The combiner algorithm)
1 Let  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  be two prize-collecting Lagrangian subset-competitive algorithms.
2 Event Function UPONREQUEST(q)
3 Let  $\alpha$  be the minimum penalty such that releasing  $(q, \alpha)$  to  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  would result in the request being served in either  $\mathrm{ALG}_1$  or  $\mathrm{ALG}_2$ . (The value of  $\alpha$  can be determined by a standard "guess-and-double".)
4 Release  $(q,\alpha)$  to both  $\mathrm{ALG}_1$  and  $\mathrm{ALG}_2$ . Buy the items bought by  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  as a result of this step.
The algorithm is simple: for a new online request  $q$ , the framework chooses the minimum penalty  $\alpha$  which ensures that at least one of the two constituent algorithms  $\mathrm{ALG}_1, \mathrm{ALG}_2$  would actually serve  $q$  (instead of paying the penalty).  $(q, \alpha)$  is then presented as a (prize-collecting) request to both algorithms. (Recall that the combined algorithm is for the non-prize-collecting problem, but the individual algorithms  $\mathrm{ALG}_1, \mathrm{ALG}_2$  are for the prize-collecting problem.) At this stage, one of the algorithms serves the request (due to the choice of  $\alpha$ ) while the other may choose to pay the penalty. The combiner algorithm now simply buys all items bought by either algorithm.
Finally, we state the guarantees of the combiner algorithm informally. (For a formal description, see Appendix C.)
Theorem 1.5. (Informal) If  $\mathrm{ALG}_1$ ,  $\mathrm{ALG}_2$  are Lagrangian  $\beta$ -subset-competitive algorithms for  $\beta = f(s)$ ,  $f(n)$  respectively, then Algorithm 1 satisfies the discrete-smoothness property (Equation (1)).
2Our subset-competitiveness property is similar to [9].
<sup>3</sup>Our Lagrangian competitiveness is similar to the Lagrangian multiplier preserving property in approximation algorithms for prize-collecting problems, e.g., [37, 26].
Applications of Theorem 1.5: Section 3 and Appendix B give Lagrangian subset-competitive algorithms for facility location, and Section 4 gives a Lagrangian subset-competitive algorithm for set cover. Given these constituent algorithms, we use Theorem 1.5 to prove Theorem 1.1 and Theorem 1.2 for facility location and Theorem 1.3 for set cover. These proofs are given in Appendix D.
Related Work. There is a growing body of work in online algorithms with predictions in the last few years (see, e.g., the surveys [35, 36]). This model was introduced by Lykouris and Vassilvitskii for the caching problem [32] and has since been studied for a variety of problem classes: rent or buy [27, 25, 21, 41, 5, 39], covering [11], scheduling [27, 41, 10, 28, 34, 30, 8], caching [31, 40, 24, 13], matching [29, 16, 7, 23], graph problems [6, 22, 1, 14, 4, 20, 9], and so on. Prior works on online facility location with predictions either do not consider prediction error [1] or use continuous notions of error [22, 20], such as functions of the distances between predicted and optimal facilities. Our discrete notion of error refers only to whether an optimal item is predicted. Similarly, prior work on online set cover with predictions [11, 4] also does not consider prediction error. Finally, we note that discrete prediction error (similar to this paper) as well as hybrids between discrete and continuous error have also been considered [42, 9, 14] but the prediction here is on the input rather than the solution.
2 The Framework
We now describe some of the concepts of the framework in more detail.
Reduction from  $s_{\delta}$  to  $s$ . Recall that we seek discrete-smooth algorithms, i.e., satisfying Equation (1). Our first step is to give a generic reduction that allows us to slightly weaken the guarantee to the following:
$$
\operatorname {A L G} \leq O (f (s)) \cdot \operatorname {O P T} | _ {S} + O (f (n)) \cdot \operatorname {O P T} | _ {\bar {S}}, \tag {6}
$$
where  $O(f(\cdot))$  is the competitive ratio without predictions. We give a reduction from an algorithm that satisfies Equation (6) to one that satisfies Equation (1):
Theorem 2.1. Given an algorithm  $\mathrm{ALG}'$  such that  $\mathrm{ALG}' \leq O(f(s)) \cdot \mathrm{OPT}|_S + O(g) \cdot \mathrm{OPT}|_{\bar{S}}$ , there exists an algorithm ALG such that  $\mathrm{ALG} \leq O(f(s_{\Delta})) \cdot \mathrm{OPT}|_S + O(g) \cdot \mathrm{OPT}|_{\bar{S}}$ .
The proof of this theorem, in Appendix E, is roughly the following: for every integer  $i$ , once the cost of the algorithm exceeds  $2^i$ , we buy the cheapest predicted items of total cost at most  $2^i$ , and then remove them from the prediction. While  $2^i < \mathrm{OPT}$ , the total cost is  $O(1) \cdot \mathrm{OPT}$ ; once  $2^i$  exceeds OPT, the size of the prediction is at most  $s_{\Delta}$ , and Equation (6) implies Equation (1).
Monotonicity. An additional, natural property that we demand from a constituent algorithm in our smooth combination framework is that increasing the penalty of input requests does not decrease the cost incurred by the algorithm. This is stated formally in the following definition.
Definition 2.2. We say that a prize-collecting algorithm ALG is monotone if, fixing the input request prefix  $((q_i,\pi_i))_{i = 1}^{k - 1}$  and current request  $(q_{k},\pi_{k})$  , then increasing  $\pi_{k}$  does not decrease  $\mathrm{ALG}(q_k,\pi_k)$
Online amortization. Our framework extends to the case where Lagrangian subset-competitiveness and monotonicity are satisfied by amortized costs instead of actual costs. This is important because for some problems, the actual cost expressly prohibits subset competitiveness. For example, consider facility location: given an input of multiple, identical requests with very small penalty, the algorithm should eventually stop paying penalties and open a facility. However, for the specific request upon which the facility is opened, the cost of the algorithm is much larger than the penalty for that request, the latter being optimal for just that request. To overcome this complication, we allow the cost for a request to be amortized over previous requests, and call this online amortization.
First, we define online amortization of costs, and define a "monotone" online amortization which can be used in our framework.
Definition 2.3 (online amortization). Let  $Q = ((q_{1},\pi_{1}),\dots ,(q_{n},\pi_{n}))$  be an online input given to ALG. An online amortization or OA is a number sequence  $(\mathrm{OA}(q,\pi))_{(q,\pi)\in Q}$  such that:
1.  $\mathrm{ALG}(Q)\leq \sum_{(q,\pi)\in Q}\mathrm{OA}(q,\pi).$
2.  $\mathrm{OA}(q_i,\pi_i)$  is only a function of  $(q_{1},\pi_{1}),\dots ,(q_{i},\pi_{i})$  , and can thus be calculated online.
When considering the amortized cost of an algorithm, we use similar notation to the actual cost: on an input  $Q$ , we use  $\mathrm{OA}(Q)$  to denote the total amortized cost. We also use  $\mathrm{OA}(Q'|Q)$  to denote the total amortized cost incurred on a request subset  $Q' \subseteq Q$ . In addition, for a request  $(q, \pi)$  in the input  $Q$ , we use  $\mathrm{OA}(q, \pi)$  to refer to the amortized cost of  $(q, \pi)$ ; here the input  $Q$  is clear from context.
Definition 2.4 (monotone online amortization). We call an online amortization OA monotone if (a) fixing previous requests, increasing the penalty of request  $(q,\pi)$  never decreases  $\mathrm{OA}(q,\pi)$ , and (b) when the algorithm pays penalty for  $(q,\pi)$  then  $\mathrm{OA}(q,\pi) \geq \pi$ .
The Main Theorem We are now ready to state the main theorem of our algorithmic framework. We use  $\beta_{1}$  and  $\beta_{2}$  to denote the competitive ratios of  $\mathrm{ALG_1}$  and  $\mathrm{ALG_2}$ ; the reader should think of  $\beta_{1}$  as  $O(f(s))$  and  $\beta_{2}$  as  $O(f(n))$ , i.e.,  $\beta_{2} \gg \beta_{1}$ .