Online Facility Location with Multiple Advice

1 Introduction . Clustering is a central topic in unsupervised learning [ Jain and Dubes , 1988 , Aggarwal and Reddy , 2014 , Gan et al. , 2020 ] In the past few years , its online version has gained a lot of attention [ Zhang et al. , 2004 , Liberty et al. , 2016 , Lattanzi and Vassilvitskii , 2017 , Cohen-Addad et al. , 2021 ] . In this paper we are interested in the fundamental question of whether machine-learned advice , especially when coming from multiple sources , can boost the performance of clustering algorithms in the online setting . In particular , we are interested in designing online algorithms with access to multiple advice that exhibit strong theoretical guarantees together with a good experimental performance . To explore the interplay between online algorithms and multiple advice we focus on facility location , a classic clustering problem with a long and rich history in computer science and operations research [ Korte and Vygen , 2018 ] , which can also be seen as the Lagrangian relaxation of k−median clustering . In the online setting of k−median clustering , it is easy to see that if the requirement of having k clusters is a hard constraint then the cost of solutions is bound to be arbitrarily far from the optimum , which makes the straightforward online formulation rather uninteresting algorithmically . There are two standard ways in which the constraint can be relaxed : bi-criteria solutions and the Lagrangian relaxation . Facility location captures the latter . In the online version of facility location , points arrive in sequence and , as soon as a point arrives , an irrevocable decision must be made : we either assign it to an existing opened facility or we open a new one to serve it . Each facility , therefore , induces a cluster , consisting of the points assigned to it . The cost of this sequence of decisions is given by the sum of the service costs ( i.e . the distance between a point and its facility ) and the total facility cost ( opening each facility has a cost , which in this paper we assume to be uniform ) . The goal is to find as cheap a clustering as possible . This online version 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . was first introduced by Meyerson [ 2001 ] , who gave an elegant O ( log ( n ) ) -approximation algorithm , and studied extensively thereafter [ Fotakis , 2003 , 2005 , 2011 , Cygan et al. , 2018 , Cohen-Addad et al. , 2019 , Ahmed et al. , 2020 , Guo et al. , 2020 ] . One of the main reasons for this heightened attention is that even in dynamic settings the resulting clustering is very stable . This is particularly useful in real-world systems where clusters are served directly to users or when they are used in a downstream machine learning model , a situation where modifying the clustering would incur too high a cost . In real-world applications , one typically has side information that could help solve the online clustering problem . This natural scenario is a recurrent theme in machine learning that focuses on how to take advantage of expert advice . In the context of online algorithms , Lykouris and Vassilvitskii [ 2018 ] and Mahdian et al . [ 2012 ] introduced a formal framework for online problems within which mathematically rigorous performance guarantees can be given in terms of the quality of the advice . In particular , they showed how to incorporate the advice in a robust way : the advice is exploited when it is good and disregarded when it is bad . All this happens automatically : the robust algorithm does not need to know in advance which is which . Remarkably , when the advice is bad the performance of the best online algorithm is matched . The approach combines mathematical rigor with practical effectiveness , adding the desirable dimension of robustness . The solution we develop for facility location is in the same spirit , with a focus on multiple advice which adds an important new dimension . Thus , the goal of this paper is to show how to take advantage of multiple advice within this rigorous algorithmic framework in the case of online facility location . In our set-up the advice comes in the form of a family of sets , each suggesting a list of facilities to open . As we show with our experiments , multiple advice of this form can be easily and efficiently produced on the basis of past data . The main technical result of this paper is the following . Suppose we have a family of sets S1 , . . . , Sk which intuitively represent sets of suggested facilities to open , and let S be their union . We develop an online randomized algorithm , called TAKEHEED , that computes a solution whose expected cost is O ( log ( |S| ) · OPT ( S ) ) , where OPT ( S ) is the best solution that can be obtained using only facilities in S. Notice that the OPT ( S ) factor is no worse than any of the suggested sets in isolation since OPT ( S ) ≤ mini OPT ( Si ) . In fact , it can be much better . For instance , every set could contain just a few good points and a lot of noise . Or , one of the sets could be good but its identity unknown . And yet , TAKEHEED separates the wheat from the chaff and delivers a near-optimal solution . This result ( which is the main technical step toward the solution ) can be seen as a solution to the version of facility location in which the facilities are constrained to be in a specific set , as opposed to the entire input metric space . The above result is essentially the best possible . In Section 4 , we prove that the log |S| factor is near-optimal . We also show that the best-known algorithms ( see Meyerson [ 2001 ] , Fotakis [ 2003 , 2005 ] ) can not match the same bound , for they can only attain an Ω ( |S| ) approximation factor in general . This is a strong indication that techniques different from those employed by those state-of-the-art algorithms are required for this problem , providing a compelling motivation for our approach which is based on the theory of Hierarchically Separated Trees ( HST ’ s ) introduced by Bartal [ 1996 ] in a seminal paper . On the practical side , our experimental analysis with real and synthetic data shows that it is easy and inexpensive to compute multiple advice of the form described , allowing TAKEHEED to outperform state-of-the-art algorithms such as those in Meyerson [ 2001 ] , Fotakis [ 2003 ] , Anagnostopoulos et al . [ 2004 ] . Furthermore , we also show that we can make our algorithm robust to erroneous advice : when S is far from any optimal set of facilities for our input , OPT ( S ) can be much larger than the actual offline optimum OPT . However , by combining TAKEHEED with known results of Mahdian et al . [ 2012 ] we can get a robust version of TAKEHEED called WARY , whose cost is O ( min { log ( |S| ) · OPT ( S ) , log ( n ) log log ( n ) · OPT } ) . Note that the second term matches the best possible bound , in view of the lower bound of Fotakis [ 2003 ] . Our experiments confirm the good behavior in practice of this robust version . Indeed , it outperforms other state-of-the-art robust solutions . One of the appealing features of TAKEHEED is that it leverages multiple advice in a simple way . The algorithm however could suffer from an “ adversarial ” attack of sorts . If very large bad sets of suggested facilities are given as part of the advice , the theoretical guarantee , which depends on the cardinality of the union of the advice sets , becomes uninteresting . To deal with this we design BUCKETHEED , an algorithm that is resilient to the presence of such large , bad advice . If the best advice set is Si∗ , the expected cost of the solution produced by this algorithm is O ( log log ( n ) · ( log ( k ) + log ( |Si∗ | ) ) · OPT ( Si∗ ) . This result is mainly of theoretical interest . Indeed , one of the strengths of TAKEHEED is that in practice small advice sets are easy to obtain and the term log ( |S| ) is going to be quite small , making the overall approach very actionable . Related Work . Facility location and online algorithms have been extensively studied in the algorithm and machine learning literature . Here we give a high-level overview of the main results . Facility Location . The offline metric ( uncapacitated ) facility location problem has been extensively studied , and various algorithms and heuristics have been formulated ( see Korte and Vygen [ 2018 ] ) . The problem is known to be NP-hard , and it can not be approximated within a factor of 1.463 unless NP ⊆ DTIME ( nlog log ( n ) ) [ Guha and Khuller , 1999 ] . The best-known polynomial-time algorithm yields a 1.488 approximation ratio [ Li , 2013 ] . Online Setting . We cast our results in the standard setting of online competitive analysis [ Borodin and El-Yaniv , 2005 ] . The online version of the metric facility location problem was introduced by Meyerson [ 2001 ] , together with a randomized algorithm providing an expected O ( log ( n ) ) approximation of the optimal solution . Subsequently , Fotakis [ 2003 ] provided a deterministic algorithm with a O ( log ( n ) / log log ( n ) ) competitive ratio and proved that no randomized algorithm can obtain an asymptotically better competitive ratio against an oblivious adversary , even if the points lie on a line . Anagnostopoulos et al . [ 2004 ] also provided an online algorithm for facility location with restricted facilities : their algorithm works only in the euclidean plane , and our procedure combining the advice can be seen as a generalization of this result in a generic metric space . Fotakis [ 2005 ] formulated a different algorithm for the general online problem , achieving a O ( log ( n ) ) −approximation of the optimal cost with a primal-dual approach . Mahdian et al . [ 2012 ] showed how to combine two online algorithms for facility location into a single algorithm achieving a constant approximation of the costs of the best algorithm . In fact , the mixing procedure can be also be used to combine an algorithm with a suggested set of facilities , or two sets of suggested facilities . Applied inductively , it can also be used to mix multiple advice . In asymptotic terms , the resulting bound is incomparable to ours ( see the Supplementary Material ) and , in practice , our approach appears to be preferable , as documented in the experimental section . Other formulations of online clustering have been proposed : e.g. , Mettu and Plaxton [ 2003 ] allow to chose the order by which processing the input , which is provided in advance ; Guo et al . [ 2020 ] consider the online model with recourse , in which one is allowed to close some facilities and reassign the clients to new ones . Guo et al . [ 2020 ] also use HSTs for their clustering problem , but in a different way to ours . Online algorithms with learned advice . There have been various previous attempts to improve online algorithms with external information . Common assumptions are related to the distribution of the input data . The most popular assumption is that data arrive according to the random arrival model , i.e . the input is a random permutation of a fixed unknown set of elements . This leads to major improvements in the competitive ratio for many problems [ McGregor , 2014 ] . Even Meyerson ’ s algorithm is shown to have a constant expected competitive ratio in this setting [ Meyerson , 2001 ] . In many other scenarios , it is common to assume that input data are sampled from a fixed distribution , like for online matching [ Mirrokni et al. , 2012 ] or job scheduling [ Mitzenmacher , 2020 ] . Recently , Lykouris and Vassilvitskii [ 2018 ] formulated the Online with Machine Learned Advice ( OMLA ) model , which is a theoretical framework for combining online algorithms with machine learning predictions . They also designed an algorithm for the caching problem in this model , which has been improved by Rohatgi [ 2020 ] and Wei [ 2020 ] . Various other problems have been tackled in this model with encouraging outcomes , like ski-rental [ Purohit et al. , 2018 ] , the Secretaries problem [ Dütting et al. , 2020 ] , and scheduling with restricted assignments [ Lattanzi et al. , 2020 ] ( see Mitzenmacher and Vassilvitskii [ 2020 ] for a survey ) . In a slightly different setting , Medina and Vassilvitskii [ 2017 ] designed an algorithm for reserve price optimization in auctions . Learned predictions have been also used to improve classical data structures [ Kraska et al. , 2018 , Mitzenmacher , 2018 , Hsu et al. , 2019 ] . A different model is the advice model , in which there is an oracle knowing the exact input and providing a small part of it to the online algorithm ( see Boyar et al . [ 2017 ] for a survey ) . Note that this oracle is supposed to be perfect and completely reliable , while in our setting , the predictions can be extremely inaccurate , nevertheless the online algorithm needs to be robust to them . Another different setting is the one of online learning with experts [ Choromanska and Monteleoni , 2012 ] : here the advice , as the proposed clustering decisions , can change at every time step , while in our setting they can not change over time . Online Facility Location with learned advice . Very recent independent work in the same spirit as ours , made public on ArXiv after the present paper was submitted , are Fotakis et al . [ 2021 ] and Jiang et al . [ 2021 ] .

Summary: The paper looks at the problem of online facility location with multiple advice.  In the online facility location problem points of a metric space arrive online and once a point arrives it either has to be connected to an already open facility, or a new facility gets opened to serve it. These decisions are irrevocable and the goal is to minimise the total cost which consists of the cost for opening the set of facilities and the sum of distances from each point to the facility that serves it. The best known algorithm for the problem is due to Fotakis and obtains a competitive ratio of $O(\log(n)/\log\log(n))$.   The current paper assumes that the algorithm has obtained $k$ different sets of points as "advice". Each such set of points $S_i$ can be thought to represent a set of suggested facilities to open, but it is unclear which -- if any -- of these sets lead to a good solution and which not. The paper uses this advice to develop algorithm TakeHeed with expected cost $O(\log|S|)\cdot OPT(S)$, where $S$ is the union of advice sets and $OPT(S)$ the best solution among the ones that are restricted to use facilities of $S$. Clearly, $OPT(S)$ cannot be worse than the optimal solution restricted to each advice set, so intuitively the algorithm is good when there are relatively few points in the advice sets and at least one of them is reasonably accurate. The algorithm can be robustified by standard techniques (Mahdian et al.) and an additional algorithm is given in the supplementary material for the (arguably not so natural case) in which $|S|$ is large. This additional algorithms compares directly to the quality of the best advice set.  The paper also contains expermints both on synthetic and real world (albeit from different settings) inputs.