SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3. There exists a polynomial-time online learning algorithm for Problem 2 (Algorithm 3), such that for any sequence  $R_1, \ldots, R_T \subseteq L(\mathcal{T})$ , its output  $y^1, \ldots, y^T$  satisfies
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t}\right) + \sum_ {t = 2} ^ {T} \left| \left| y ^ {t} - y ^ {t - 1} \right| \right| _ {\mathcal {T}} \leq \frac {3}{2} \cdot \min  _ {y ^ {*} \in \mathcal {F P} (\mathcal {T})} \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \beta \cdot \sqrt {T}
$$
for  $\beta = \mathcal{O}\left(k\cdot |L(\mathcal{T})|^{3 / 2}\cdot D_{\mathcal{T}}\cdot \max (\gamma ,1)\right)$
3.2 From Fractional to Integral Placements in HSTs
As already mentioned, the basic idea of our approach is to convert at each round  $t \geq 1$  the fractional placement  $y^{t} \in \mathcal{FP}(\mathcal{T})$  produced by Algorithm 3 into an integral facility placement  $F_{t} \subseteq L(\mathcal{T})$  with  $|F_t| = k$  on the leaves of the HST. In order to guarantee small regret, our rounding scheme should preserve both the connection and the moving cost of the fractional solution within constant factors for any possible set of arriving clients. In order to guarantee the latter, our rounding scheme Cut&Round (Algorithm 4) uses shared randomness across different rounds. Cut&Round is rather complicated and is presented in Section 5. To this end, we present its formal guarantee.
Theorem 4. There exists a linear-time deterministic algorithm, called Cut&Round (Algorithm 4), that takes as input an HST  $\mathcal{T}$ , a fractional facility placement  $y \in \mathcal{FP}(\mathcal{T})$  and a vector  $\alpha \in [0,1]^{|V|}$  and outputs a placement of  $k$  facilities  $F \gets \mathrm{Cut\&Round}(\mathcal{T}, y, \alpha)$  on the leaves of  $\mathcal{T}$  ( $F \subseteq L(\mathcal{T})$  and  $|F| = k$ ) such that
1.  $\operatorname{E}_{\alpha \sim \operatorname{Unif}(0,1)}[C_R(F)] = f_R(y)$  for all client requests  $R \subseteq L(\mathcal{T})$ .
2.  $\mathrm{E}_{\alpha \sim \mathrm{Unif}(0,1)}[\gamma \cdot M_{\mathcal{T}}(F,F^{\prime})]\leq 4\cdot ||y - y^{\prime}||_{\mathcal{T}}$  for all other fractional facility placements  $y^\prime \in \mathcal{FP}(\mathcal{T})$  and  $F^{\prime}\gets \mathrm{Cut}\& \mathrm{Round}(\mathcal{T},y^{\prime},\alpha)$
Item 1 of Theorem 4 establishes that although  $\mathrm{Cut} \& \mathrm{Round}$  is oblivious to the arrived set of clients  $R_{t} \subseteq L(\mathcal{T})$ , the expected connection cost of the output equals the fractional connection cost under  $y^{t} \in \mathcal{FP}(\mathcal{T})$ . Item 2 of Theorem 4 states that once the same random seed  $\alpha$  is used into two consecutive time steps, then the expected moving cost between the facility-placements  $F_{t}$  and  $F_{t+1}$  is at most  $\mathcal{O}(1)$ -times the fractional moving cost between  $y^{t}$  and  $y^{t+1}$ . Both properties crucially rely on the structure of the HST and consist one of the main technical contributions of our work.
3.3 Overall Online Learning Algorithm
We are now ready to formally introduce our main algorithm (Algorithm 2) and prove Theorem 1. First, we combine the algorithms from Theorems 3 and 4 to design an  $\mathcal{O}(1)$ -regret algorithm for Problem 1 on HSTs (Algorithm 1). Up next we present how Algorithm 1 can be converted into an  $\mathcal{O}(\log n)$ -regret online learning algorithm for general graphs, using the metric embedding technique of Theorem 2, resulting in our final algorithm (Algorithm 2).
Algorithm 1  $\mathcal{O}(1)$ -regret for HSTs.
1: Input: A sequence  $R_{1},\ldots ,R_{T}\subseteq L(\mathcal{T})$
2: The learner samples  $\alpha_v \sim \mathrm{Unif}(0,1)$  for all  $v \in V(\mathcal{T})$ .
3: for each round  $t = 1$  to  $T$  do
4: The learner places the  $k$  facilities to the leaves of the HST  $\mathcal{T}$  based on the output  $F_{t}:=\mathrm{Cut\&Round}(\mathcal{T},y^{t},\alpha)$ .
5: The learner learns  $R_{t} \subseteq L(\mathcal{T})$ .
6: The learner updates  $y^{t + 1}\in \mathcal{FP}(\mathcal{T})$  by running Algorithm 3 for Problem 2 with input  $R_{1},\ldots ,R_{t}$
Algorithm 2  $\mathcal{O}(\log n)$ -regret for graphs.
1: Input: A sequence  $R_{1},\ldots ,R_{T}\subseteq L(\mathcal{T})$
2: The learner embeds  $G(V,E,w)$  into a (random) HST  $\mathcal{T}$  with  $L(\mathcal{T}) = V$  via the procedure of Theorem 2.
3: for each round  $t = 1$  to  $T$  do
4: The learner selects a facility-placement  $F_{t} \subseteq V$ .
5: The learner learns  $R_{t} \subseteq V$ .
6: The learner updates  $F_{t + 1}$  by giving as input  $R_{1},\ldots ,R_{t}\subseteq L(\mathcal{T})$  to Algorithm 1 for  $\mathcal{T}$ .
7: end for
Theorem 5. For any sequence of client requests  $R_{1}, \ldots, R_{T} \subseteq L(\mathcal{T})$ , the sequence of facility-placements  $F_{1}, \ldots, F_{T} \subseteq L(\mathcal{T})$  produced by Algorithm 1 satisfies
$$
\mathbb {E} \left[ \sum_ {t = 1} ^ {T} C _ {R _ {t}} \left(F _ {t}\right) + \gamma \cdot \sum_ {t = 2} ^ {T} M _ {\mathcal {T}} \left(F _ {t}, F _ {t - 1}\right) \right] \leq 6 \cdot \min  _ {| F ^ {*} | = k} \sum_ {t = 1} ^ {T} C _ {R _ {t}} \left(F ^ {*}\right) + \beta \cdot \sqrt {T}
$$
$$
f o r \beta = \mathcal {O} (k \cdot | L (\mathcal {T}) | ^ {3 / 2} \cdot D _ {\mathcal {T}} \cdot \max  (\gamma , 1)).
$$
Theorem 5 establishes that Algorithm 1 achieves constant regret in the special case of HSTs and its proof easily follows by Theorems 3 and 4. Then, the proof of Theorem 1 easily follows by Theorem 2 and Theorem 5. All the proofs are deferred to Section C of the Appendix.
4  $\mathcal{O}(1)$ -Regret for Fractional HST Clustering
In this section we present the  $\mathcal{O}(1)$ -regret algorithm for Problem 2, described in Algorithm 3, and exhibit the key ideas in establishing Theorem 3. Without loss of generality, we can assume that the facility-weight satisfies  $\gamma \geq 1^5$ .
Algorithm 3 is the well-known online learning algorithm Follow the Regularized Leader (FTRL) with a specific regularizer  $R_{\mathcal{T}}(\cdot)$  presented in Definition 9. Our results crucially rely on the properties of this regularizer since it takes into account the HST structure and permits us to bound the fractional moving cost of FTRL.
If not, establishing our guarantees for  $\gamma = 1$  will clearly upper bound the actual moving cost.
Definition 9. Given an HST  $\mathcal{T}$ , the dilated entropic regularizer  $R_{\mathcal{T}}(y)$  over  $y \in \mathcal{FP}(\mathcal{T})$  is defined as
$$
R _ {\mathcal {T}} (y) := \sum_ {v \neq r} 2 ^ {\mathrm {l e v} (v)} \cdot (y _ {v} + \delta_ {v}) \cdot \ln \left(\frac {y _ {v} + \delta_ {v}}{y _ {p (v)} + \delta_ {p (v)}}\right)
$$
where  $\delta_v \coloneqq (k / n) \cdot |L(\mathcal{T}) \cap T(v)|$  and  $n \coloneqq |L(\mathcal{T})|$ .
Algorithm 3 FTRL with dilated entropic regularization
1: Input: An adversarial sequence  $R_{1}, \ldots, R_{T} \subseteq L(\mathcal{T})$ .
2: for  $t = 1$  to  $T$  do
3: The learner selects  $y^{t}\in \mathcal{FP}(\mathcal{T})$
4: The learner suffers cost  $f_{R_t}(y^t) + ||y^t - y^{t-1}||_{\mathcal{T}}$ .
5: The learner updates  $y^{t + 1} \gets \arg \min_{y \in \mathcal{FP}(\mathcal{T})} \left[ \sum_{s = 1}^{t} f_{R_s}(y) + (\gamma \sqrt{nT}) \cdot R_{\mathcal{T}}(y) \right]$ .
6: end for
Algorithm 3 selects at each step  $t$  the facility placement  $y^{t} \in \mathcal{FP}(\mathcal{T})$  that minimizes a convex combination of the total fractional connection cost for the sub-sequence  $R_{1},\ldots,R_{t-1}$  and  $R_{\mathcal{T}}(y)$ . The regularization term ensures the stability of the output, which will result in a bounded fractional moving cost.
Analysis of Algorithm 3. Due to space limitations, all proofs are moved to Section D of the Appendix. The primary reason for the specific selection of the regularizer at Definition 9 is that  $R_{\mathcal{T}}(\cdot)$  is strongly convex with respect to the norm  $||\cdot||_{\mathcal{T}}$  of Definition 8, as established in Lemma 1 which is the main technical contribution of the section. We use  $D = D_{\mathcal{T}}$  for the diameter of  $\mathcal{T}$ .
Lemma 1. For any vectors  $y, y' \in \mathcal{FP}(\mathcal{T})$ ,
$$
R _ {\mathcal {T}} \left(y ^ {\prime}\right) \geq R _ {\mathcal {T}} (y) + \left\langle \nabla R _ {\mathcal {T}} (y), y ^ {\prime} - y \right\rangle + \left(8 k D \gamma^ {2}\right) ^ {- 1} \cdot | | y - y ^ {\prime} | | _ {\mathcal {T}} ^ {2}
$$
The strong convexity of  $R_{\mathcal{T}}(y)$  with respect to  $||\cdot||_{\mathcal{T}}$  is crucial since it permits us to bound the moving cost of Algorithm 3 by its fractional connection cost.
Lemma 2. For any sequence  $R_{1},\ldots ,R_{T}\subseteq L(\mathcal{T})$  , the output of Algorithm 3 satisfies
$$
\sum_ {t = 2} ^ {T} \left\| y ^ {t} - y ^ {t - 1} \right\| _ {\mathcal {T}} \leq \frac {1}{2} \cdot \sum_ {t = 1} ^ {T} f _ {R _ {t}} (y ^ {t}) + \mathcal {O} (\gamma k D) \cdot \sqrt {T}
$$
We remark that using another regularizer  $R(\cdot)$  that is strongly convex with respect to another norm  $||\cdot||$  would still imply Lemma 1 with respect to  $||\cdot||$ . The problem though is that the fractional moving cost  $\sum_{t=1}^{T} ||y_t - y_{t-1}||$  can no longer be associated with the actual moving cost  $\sum_{t=1}^{T} M_{\mathcal{T}}(F_t, F_{t-1})$ . It is for this reason that using a regularizer that is strongly convex with respect to  $||\cdot||_{\mathcal{T}}$  is crucial.
Next, by adapting the standard analysis of FTRL to our specific setting, we derive Lemma 3 establishing that Algorithm 3 admits bounded connection cost.
Lemma 3. For any sequence  $R_1, \ldots, R_T \subseteq L(\mathcal{T})$ , the output of Algorithm 3 satisfies
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t}\right) \leq \min  _ {y ^ {*} \in \mathcal {F P}} \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \mathcal {O} \left(k n ^ {3 / 2} D \gamma\right) \cdot \sqrt {T}
$$
The proof of Theorem 3 directly follows by Lemma 2 and 3. We conclude the section by presenting how Step 5 of Algorithm 3 can be efficiently implemented, namely
$$
\min  _ {y \in \mathcal {F P} (\mathcal {T})} \Phi_ {t} (y) := \sum_ {s = 1} ^ {t} f _ {R _ {s}} (y) + (\gamma \sqrt {n T}) \cdot R _ {\mathcal {T}} (y).
$$
Since  $\Phi_t(y)$  is strongly convex and the set  $\mathcal{FP}(\mathcal{T})$  is a polytope, one could use standard optimization algorithms such as the ellipsoid method or projected gradient descent to approximately minimize
$\Phi_t(y)$  given access to a sub-gradient oracle for  $\Phi_t(\cdot)$ . In Claim 11 of Section D of the Appendix, we establish that the sub-gradients of  $\Phi(\cdot)$  can be computed in polynomial time and thus any of the previous methods can be used to approximately minimize  $\Phi(\cdot)$ . In Lemma 4 we establish the intuitive fact that approximately implementing Step 5 does not affect the guarantees of Theorem 3.
Lemma 4. Let  $y^{t}$  be the minimizer of  $\Phi_t(\cdot)$  in  $\mathcal{FP}(T)$  and let  $z^t\in \mathcal{FP}(\mathcal{T})$  be any point such that  $\Phi_t(z^t)\leq \Phi_t(y^t) + \epsilon$  for some  $\epsilon = \mathcal{O}(T^{-1 / 2})$ . Then,
$$
f _ {R _ {t}} (z ^ {t}) + | | z ^ {t} - z ^ {t - 1} | | _ {\mathcal {T}} \leq f _ {R _ {t}} (y ^ {t}) + | | y ^ {t} - y ^ {t - 1} | | _ {\mathcal {T}} + \mathcal {O} \left(k n ^ {3 / 2} D \gamma\right) \cdot T ^ {- 1 / 2}
$$
Remark 3. In our implementation of the algorithm, we approximately solve Step 5 of Algorithm 3 via Mirror Descent based on the Bregman divergence of  $\mathcal{R}_{\mathcal{T}}(\cdot)$ . This admits the same convergence rates as projected gradient descent but the projection step can be computed in linear time with respect to the size of the HST  $\mathcal{T}$ . We present the details of our implementation in Section C of the Appendix.
5 The Cut&Round Rounding
In this section we present our novel rounding scheme (Algorithm Cut&Round) as well as the main steps that are required in order to establish Theorem 4. To ease notation, for any real number  $x \geq 0$  we denote its decimal part as  $\delta(x) = x - \lfloor x \rfloor$ . We comment that our rounding scheme simply maintains and updates a distribution over the vertices of the HST, and can be thus implemented in polynomial-time. Similar rounding schemes, like the one presented in [9], typically maintain a distribution over all possible facility-placements, which generally cannot be implemented in polynomial-time.
Algorithm 4 Cut&Round.
1: Input: An HST  $\mathcal{T}$ , a fractional placement  $y\in \mathcal{FP}(\mathcal{T})$  and thresholds  $\alpha_{v}\in [0,1]$  for all  $v\in V(\mathcal{T})$ .
2:  $Y_{r}\gets k$
3: for levels  $\ell = h(T)$  to 1 do
4: for all nodes  $v$  with  $lev(v) = \ell$  do
5:  $Y_{rem}\gets Y_v$
6:  $y_{rem}\gets y_v$
7: for all children  $u \in \operatorname{cld}(v)$  do
8:  $Y_{u}\gets \mathrm{Alloc}(y_{u},Y_{rem},y_{rem},\alpha_{u})$
9:  $Y_{rem} \gets Y_{rem} - Y_u$
10:  $y_{rem}\gets y_{rem} - y_u$
11: end for
12: end for
13: end for
14: return  $F \coloneqq \{u \in L(\mathcal{T}) : Y_u = 1\}$ .
Algorithm 5 Alloc.
Input: Numbers  $y_{u}, y_{rem} \geq 0, Y_{rem} \in \mathbb{N}$  and  $\alpha_{u} \in [0,1]$ .
if  $Y_{rem} = \lfloor y_{rem}\rfloor$  then
if  $\delta(y_u) < \delta(y_{rem})$  then
$$
Y _ {u} \leftarrow \lfloor y _ {u} \rfloor
$$
$$
Y _ {u} \leftarrow \left\lfloor y _ {u} \right\rfloor + \mathbb {1} \left[ a _ {u} \leq \frac {\delta (y _ {u}) - \delta (y _ {r e m})}{1 - \delta (y _ {r e m})} \right]
$$
$$
\text {i f} \delta \left(y _ {u}\right) <   \delta \left(y _ {r e m}\right) \text {t h e n}
$$
$$
Y _ {u} \leftarrow \lfloor y _ {u} \rfloor + \mathbb {1} \left[ a _ {u} \leq \frac {\delta (y _ {u})}{\delta (y _ {r e m})} \right]
$$
$$
Y _ {u} \leftarrow \left\lfloor y _ {u} \right\rfloor + 1
$$
Return  $Y_{u}$
On principle, Cut&Round (Algorithm 4) assigns to each vertex  $v$  an integer number of facilities  $Y_v$  to be placed at the leaves of its sub-tree. Notice that due to sub-routine Alloc (Algorithm 5),  $Y_v$  either equals  $\lfloor y_v \rfloor$  or  $\lfloor y_v \rfloor + 1$ . Cut&Round initially assigns  $k$  facilities to the set of leaves that descend from the root  $r$ , which is precisely  $L(\mathcal{T})$ . Then, it moves in decreasing level order to decide  $Y_v$  for each node  $v$ . Once  $Y_v$  is determined (Step 5), the  $Y_v$  facilities are allocated to the sub-trees of its children  $u \in \operatorname{cld}(v)$  (Steps 7-10) via sub-routine Alloc using the thresholds  $\alpha_u$ , in a manner that guarantees that  $Y_v = \sum_{u \in \operatorname{cld}(v)} Y_u$  (see Section E.1 of the Appendix). This implies the feasibility of Cut&Round, as exactly  $k$  facilities are placed in the leaves of  $\mathcal{T}$  at the end of the process.
Assuming that the set of thresholds  $\alpha_{v}$  is randomly drawn from the uniform distribution in [0, 1], sub-routine Alloc (Algorithm 5) guarantees that  $Y_{v}$  either equals  $\lfloor y_v\rfloor$  or  $\lfloor y_v\rfloor +1$  while  $\mathbb{E}_{\alpha}[Y_v] = y_v$ . This is formally captured in Lemma 5 and is crucial in the proof of Theorem 4.
Lemma 5. Consider Algorithm 4 given as input a vector  $y \in \mathcal{FP}(\mathcal{T})$  and random thresholds  $\alpha_v \sim \mathrm{Unif}(0,1)$ . Then,
$$
Y _ {v} = \left\{ \begin{array}{l l} \lfloor y _ {v} \rfloor & \text {w i t h p r o b a b i l i t y 1 - \delta (y _ {v})} \\ \lfloor y _ {v} \rfloor + 1 & \text {w i t h p r o b a b i l i t y \delta (y _ {v})} \end{array} \right.
$$
By coupling Lemma 5 with the HST structure we are able to establish Theorem 4. The proof is technically involved and thus deferred to Section E of the Appendix.
In this work, we designed the first polynomial-time online learning algorithm for Online  $k$ -Clustering with Moving Costs that achieves  $\mathcal{O}(\log n)$ -regret with respect to the cost of the optimal static facility placement, extending the results of Fotakis et al. [31] for the special case of  $\gamma = 0$ . The cornerstone of our approach was to realize and establish that  $\mathcal{O}(1)$ -regret is plausible for HST metrics. This was achieved through designing a dilated entropic regularizer to capture the structure of the HST and combine it with the FTRL algorithm, as well as designing a lossless (up to constant factors) rounding scheme that simultaneously works for both the connection and the moving cost. Both of these components where central towards acquiring constant regret on HSTs.
A interesting future direction is to investigate whether a polynomial-time online learning algorithm with  $\mathcal{O}(1)$ -regret for the problem is theoretically possible or not. Since the  $\mathcal{O}(\log n)$ -factor is inherently lost when using HST embeddings, this would require a significantly different approach to the one presented in this work. Finally, we comment that our current optimality guarantees are with respect to the optimal static facility placement. Going beyond the notion of regret, an intriguing future direction is establishing guarantees with respect to the optimal dynamic facility-placement that moves facilities from round to round by suffering the corresponding moving cost.
This work was supported by the Swiss National Science Foundation (SNSF) under grant number 200021_205011, by Hasler Foundation Program: Hasler Responsible AI (project number 21043) and Innovation project supported by Innosuisse (contract agreement 100.960 IP-ICT).
[1] Alekh Agarwal, Daniel J. Hsu, Satyen Kale, John Langford, Lihong Li, and Robert E. Schapire. Taming the monster: A fast and simple algorithm for contextual bandits. In International Conference on Machine Learning, 2014.
[2] Nir Ailon. Improved bounds for online learning over the permutahedron and other ranking polytopes. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, AISTATS 2014, 2014.
[3] Soroush Alamdari and David B. Shmoys. A bicriteria approximation algorithm for the k-center and k-median problems. In Workshop on Approximation and Online Algorithms, 2017.
[4] Hyung-Chan An, Ashkan Norouzi-Fard, and Ola Svensson. Dynamic facility location via exponential clocks. ACM Trans. Algorithms, 13(2):21:1-21:20, 2017.
[5] Tarique Anwar, Surya Nepal, Cecile Paris, Jian Yang, Jia Wu, and Quan Z. Sheng. Tracking the evolution of clusters in social media streams. IEEE Transactions on Big Data, pages 1-15, 2022.
[6] Vijay Arya, Naveen Garg, Rohit Khandekar, Adam Meyerson, Kamesh Munagala, and Vinayaka Pandit. Local search heuristic for k-median and facility location problems. In Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, STOC '01, page 21-29. Association for Computing Machinery, 2001.
[7] Baruch Awerbuch and Robert Kleinberg. Online linear optimization and adaptive routing. J. Comput. Syst. Sci., 2008.
[58] Holakou Rahmanian and Manfred K. Warmuth. Online dynamic programming. In NIPS, 2017.
[59] Juliette Stehlé, Nicolas Voirin, Alain Barrat, Ciro Cattuto, Lorenzo Isella, Jean-François Pinton, Marco Quaggiotto, Wouter Van den Broeck, Corinne Régis, Bruno Lina, and Philippe Vanhems. High-resolution measurements of face-to-face contact patterns in a primary school. PLOS ONE, 6(8), 2011.
[60] Matthew J. Streeter and Daniel Golovin. An online algorithm for maximizing submodular functions. In 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008, 2008.
[61] Daiki Suehiro, Kohei Hatano, Shuji Kijima, Eiji Takimoto, and Kiyohito Nagano. Online prediction under submodular constraints. In Algorithmic Learning Theory, ALT 2012, 2012.
[62] Eiji Takimoto and Manfred K. Warmuth. Predicting nearly as well as the best pruning of a planar decision graph. In Theoretical Computer Science, 2000.
[63] Eiji Takimoto and Manfred K. Warmuth. Path kernels and multiplicative updates. J. Mach. Learn. Res., 2003.
[64] Chayant Tantipathananandh, Tanya Berger-Wolf, and David Kempe. A framework for community identification in dynamic social networks. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '07, page 717-726. Association for Computing Machinery, 2007.
[65] David P. Williamson and David B. Shmoys. The Design of Approximation Algorithms. Cambridge University Press, USA, 1st edition, 2011.
[66] Shota Yasutake, Kohei Hatano, Shuji Kijima, Eiji Takimoto, and Masayuki Takeda. Online linear optimization over permutations. In Proceedings of the 22nd International Conference on Algorithms and Computation, ISAAC 2011, 2011.
[67] Neal E. Young. K-medians, facility location, and the chernoff-wald bound. In Proceedings of the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '00, page 86-95. Society for Industrial and Applied Mathematics, 2000.
A Further Related Work
In this chapter of the appendix, we continue our discussion on the literature that relates to this work.
Efficient Combinatorial Online Learning. There exists a long line of research studying efficient online learning algorithms in various combinatorial domains (e.g., selection of paths, permutations, binary search trees etc.) [40, 62, 63, 41, 7, 60, 45, 22, 42, 66, 38, 37, 1, 2, 20, 30, 29]. Another related line of work studies black-box reductions converting any  $\alpha$ -approximation (offline) algorithm to an  $\mathcal{O}(\alpha)$ -regret online learning algorithm for a specific class of combinatorial optimization problems called linear optimization problems [47, 8, 46, 48, 61, 32, 39, 58, 21, 33, 36]. We remark that a key difference of our setting with the aforementioned works is that in the latter case the learner is not penalized for switching actions from round to round with an additional moving/switching cost. In the context of Problem 1 this means that  $\gamma = 0$  which is exactly the setting considered by [31]. As a result, apart from the fact that  $k$ -median does not belong in the class of linear optimization problems, the aforementioned black-box reductions do not apply to Problem 1 since they do not account for the moving cost.
The k-server Problem. Our work also relates with the rich line of literature on the  $k$ -server problem [50, 17, 49, 9, 19, 12]. In this setting there exists only 1 client at each round, while 1-lookahead is assumed, i.e. the request  $R_{t}$  is revealed prior to the action of the algorithm at step  $t$ . Moreover in  $k$ -server a facility must be placed in the exact position of the request, leading to a simpler combinatorial structure with respect to Problem  $1^{6}$ . However, in the  $k$ -server problem, instead of using the benchmark of regret, the more challenging metric of competitive ratio that measures the sub-optimality with respect to the optimal dynamic solution is used. Mostly related to ours is the work of [9] providing the first poly  $(\log n)$ -competitive algorithm for  $k$ -server by reducing the problem to the special case of HSTs. [9] first design a poly  $(\log n)$ -competitive algorithm for a fractional version of  $k$ -server at which facilities can be fractionally placed into the vertices of the HST. They then use a randomized rounding scheme to convert the fractional solution into an integral one. The basic difference of the randomized rounding scheme of [9] with the one that we introduce in this work (Algorithm Cut&Round) is that the first provides guarantees only for the moving cost of the facilities while Cut&Round provides guarantees both for the moving cost of the facilities as well as the connection cost of the clients.
Consistent  $k$ -Clustering. Another setting of clustering in the presence of unknown clients is that of Consistent  $k$ -Clustering [53, 34, 27]. In this setting, given an unknown stream of clients, a set of  $k$  facilities has to be maintained over time so that at any round  $t$ , the selected facilities form an approximately optimal solution of the sub-instance consisting of clients appeared in the time interval  $\{1, t\}$ . A basic difference of Consistent  $k$ -Clustering with Problem 1 is that in the first case the moving cost is not penalized as long as the number of swaps does not exceed a certain threshold  $\mathcal{O}(k)$ .