SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dimitris Christou*
christou@cs.utexas.edu
Stratis Skoulakis*
In this work we consider an online learning problem, called Online  $k$ -Clustering with Moving Costs, at which a learner maintains a set of  $k$  facilities over  $T$  rounds so as to minimize the connection cost of an adversarially selected sequence of clients. The learner is informed on the positions of the clients at each round  $t$  only after its facility-selection and can use this information to update its decision in the next round. However, updating the facility positions comes with an additional moving cost based on the moving distance of the facilities. We present the first  $\mathcal{O}(\log n)$ -regret polynomial-time online learning algorithm guaranteeing that the overall cost (connection + moving) is at most  $\mathcal{O}(\log n)$  times the time-averaged connection cost of the best fixed solution. Our work improves on the recent result of Fotakis et al. [31] establishing  $\mathcal{O}(k)$ -regret guarantees only on the connection cost.
Due to their various applications in diverse fields (e.g. machine learning, operational research, data science etc.), clustering problems have been extensively studied. In the well-studied  $k$ -median problem, given a set of clients,  $k$  facilities should be placed on a metric with the objective to minimize the sum of the distance of each client from its closest center [55, 14, 13, 67, 6, 44, 52, 65, 51, 15, 54, 3].
In many modern applications (e.g., epidemiology, social media, conference, etc.) the positions of the clients are not static but rather evolve over time [57, 56, 64, 59, 23, 5]. For example the geographic distribution of the clients of an online store or the distribution of Covid-19 cases may drastically change from year to year or respectively from day to day [31]. In such settings it is desirable to update/change the positions of the facilities (e.g., compositions of warehouses or Covid test-units) so as to better serve the time-evolving trajectory of the clients.
The clients' positions may change in complex and unpredictable ways and thus an a priori knowledge on their trajectory is not always available. Motivated by this, a recent line of research studies clustering problems under the online learning framework by assuming that the sequence of clients' positions is unknown and adversarially selected [18, 28, 16, 31]. More precisely, a learner must place  $k$  facilities at each round  $t \geq 1$  without knowing the positions of clients at round  $t$  which are revealed to the learner only after its facility-selection. The learner can use this information to update its decision in the next round; however, moving a facility comes with an additional moving cost that should be taken into account in the learner's updating decision, e.g. moving Covid-19 test-units comes with a cost [18, 28].
Building on this line of works, we consider the following online learning problem:
Problem 1 (Online  $k$ -Clustering with Moving Costs). Let  $G(V, E, w)$  be a weighted graph with  $|V| = n$  vertices and  $k$  facilities. At each round  $t = 1, \ldots, T$ :
1. The learner selects  $F_{t} \subseteq V$ , with  $|F_{t}| = k$ , at which facilities are placed.
*First author contribution.
2. The adversary selects the clients' positions,  $R_{t} \subseteq V$ .
3. The learner learns the clients' positions  $R_{t}$  and suffers
$$
\text{cost} = \sum_{j\in R_{t}}\underbrace{\min_{i\in F_{t}}d_{G}(j,i)}_{\substack{\text{connection cost of client $j$}}} + \underbrace{\gamma\cdot M_{G}(F_{t - 1},F_{t})}_{\substack{\text{moving cost of facilities}}}
$$
where  $d_G(j,i)$  is the distance between vertices  $i,j\in V$ ;  $M_G(F_{t - 1},F_t)$  is the minimum overall distance required to move  $k$  facilities from  $F_{t - 1}$  to  $F_t$ ; and  $\gamma \geq 0$  is the facility-weight.
An online learning algorithm for Problem 1 tries to minimize the overall (connection + moving) cost by placing  $k$  facilities at each round  $t \geq 1$  based only on the previous positions of clients  $R_{1}, \ldots, R_{t-1}$ . To the best of our knowledge, Problem 1 was first introduced in [18] $^2$ . If for any sequence of clients, the overall cost of the algorithm is at most  $\alpha$  times the overall connection cost of the optimal fixed placement of facilities  $F^{*}$  then the algorithm is called  $\alpha$ -regret, while in the special case of  $\alpha = 1$  the algorithm is additionally called no-regret.
Problem 1 comes as a special case of the well-studied Metrical Task System by considering each of the possible  $\binom{n}{k}$  facility placements as a different state. In their seminal work, [11] guarantee that the famous Multiplicative Weights Update algorithm (MWU) achieves  $(1 + \epsilon)$ -regret in Problem 1 for any  $\epsilon > 0$ . Unfortunately, running the MWU algorithm for Problem 1 is not really an option since it requires  $\mathcal{O}(n^k)$  time and space complexity. As a result, the following question naturally arises:
Q. Can we achieve  $\alpha$ -regret for Problem 1 with polynomial-time online learning algorithms?
Answering the above question is a challenging task. Even in the very simple scenario of time-invariant clients, i.e.  $R_{t} = R$  for all  $t \geq 1$ , an  $\alpha$ -regret online learning algorithm must essentially compute an  $\alpha$ -approximate solution of the  $k$ -median problem. Unfortunately the  $k$ -median problem cannot be approximated with ratio  $\alpha < 1 + 2 / e \simeq 1.71$  (unless NP  $\subseteq \mathrm{DTIME}[n^{\log \log n}]$  [43]) which excludes the existence of an  $(1 + 2 / e)$ -regret polynomial-time online learning algorithm for Problem 1. Despite the fact that many  $\mathcal{O}(1)$ -approximation algorithms have been proposed for the  $k$ -median problem (the best current ratio is  $1 + \sqrt{3}$  [54]), these algorithms crucially rely on the (offline) knowledge of the whole sequence of clients and most importantly are not designed to handle the moving cost of the facilities [55, 14, 13, 67, 6, 44, 52, 65, 51, 15, 54, 3].
In their recent work, Fotakis et al. [31] propose an  $\mathcal{O}(k)$ -regret polynomial-time online learning algorithm for Problem 1 without moving costs (i.e. the special case of  $\gamma = 0$ ). Their approach is based on designing a no-regret polynomial-time algorithm for a fractional relaxation of Problem 1 and then using an online client-oblivious rounding scheme in order to convert a fractional solution to an integral one. Their analysis is based on the fact that the connection cost of any possible client is at most  $\mathcal{O}(k)$  times its fractional connection cost. However in order to establish the latter guarantee their rounding scheme performs abrupt changes on the facilities leading to huge moving cost.
Our Contribution and Techniques. In this work, we provide a positive answer to question (Q), by designing the first polynomial-time online learning algorithm for Online  $k$ -Clustering with Moving Costs that achieves  $\mathcal{O}(\log n)$ -regret for any  $\gamma \geq 0$ . The cornerstone idea of our work was to realize that  $\mathcal{O}(1)$ -regret can be established with a polynomial-time online learning algorithm in the special case of  $G$  being a Hierarchical Separation Tree (HST). Then, by using the standard metric embedding result of [25], we can easily convert such an algorithm to an  $\mathcal{O}(\log n)$ -regret algorithm for general graphs. Our approach for HSTs consists of two main technical steps:
1. We introduce a fractional relaxation of Problem 1 for HSTs. We then consider a specific regularizer on the fractional facility placements, called Dilated Entropic Regularizer [26], that takes into account the specific structure of the HST. Our first technical contribution is to establish that the famous Follow the Leader algorithm [35] with dilated entropic regularization admits  $\mathcal{O}(1)$ -regret for any  $\gamma \geq 0$ .
$^2$ In [18], an easier version of Problem 1 with 1-lookahead is considered, meaning that the learner learns the positions of the clients  $R_{t}$  before selecting  $F_{t}$ . Moreover,  $G$  is considered to be the line graph and  $\gamma = 1$ .
2. Our second technical contribution is the design of a novel online client-oblivious rounding scheme, called Cut&Round, that converts a fractional solution for HSTs into an integral one. By exploiting the specific HST structure we establish that Cut&Round, despite not knowing the clients' positions  $R_{t}$ , simultaneously guarantees that (i) the connection cost of each client  $j \in R_{t}$  is upper bounded by its fractional connection cost, and (ii) the expected moving cost of the facilities is at most  $\mathcal{O}(1)$  times the fractional moving cost.
Experimental Evaluation. In Section F of the Appendix we experimentally compare our algorithm with the algorithm of Fotakis et al. [31]. Our experiments verify that our algorithm is robust to increases of the facility weight  $\gamma$  while the algorithm of [31] presents a significant cost increase. We additionally experimentally evaluate our algorithm in the MNIST and CIFAR10 datasets. Our experimental evaluations suggest that the  $\mathcal{O}(\log n)$ -regret bound is a pessimistic upper bound and that in practise our algorithm performs significantly better. Finally, we evaluate our algorithm both in the random arrival case (where the requested vertices are drawn uniformly at random from the graph) as well as in adversarial settings, where the request sequences are constructed through some arbitrary deterministic process.
Related Work. As already mentioned, our work most closely relates with the work of Fotakis et al. [31] that provides an  $\mathcal{O}(k)$ -regret algorithm running in polynomial-time for  $\gamma = 0$ . [16] also consider Problem 1 for  $\gamma = 0$  with the difference that the connection cost of clients is captured through the  $k$ -means objective i.e. the sum of the squared distances. They provide an  $(1 + \epsilon)$ -regret algorithm with  $\mathcal{O}\left((k^2 / \epsilon^2)^{2k}\right)$  time-complexity that is still exponential in  $k$ . [18, 28] study the special case of Problem 1 in which  $G$  is the line graph and  $\gamma = 1$  while assuming 1-lookahead on the request  $R_t$ . For  $k = 1$ , [18] provide an  $(1 + \epsilon)$ -competitive online algorithm meaning that its cost is at most  $(1 + \epsilon)$  times the cost of the optimal dynamic solution and directly implies  $(1 + \epsilon)$ -regret. [28] extended the previous result by providing a 63-competitive algorithm for  $k = 2$  on line graphs. Our work also relates with the works of [23] and [4] that study offline approximation algorithms for clustering problems with time-evolving metrics. Finally our work is closely related with the research line of online learning in combinatorial domains and other settings of online clustering. Due to space limitations, we resume this discussion in Section A of the Appendix.
2 Preliminaries and Our Results
Let  $G(V,E,w)$  be a weighted undirected graph where  $V$  denotes the set of vertices and  $E$  the set of edges among them. The weight  $w_{e}$  of an edge  $e = (i,j)\in E$  denotes the cost of traversing  $e$ . Without loss, we assume that  $w_{e}\in \mathbb{N}$  and  $w_{e}\geq 1$  for all edges  $e\in E$ . The distance between vertices  $i,j\in V$  is denoted with  $d_G(i,j)$  and equals the cost of the minimum cost path from  $i\in V$  to  $j\in V$ . We use  $n\coloneqq |V|$  to denote the cardinality of  $G$  and  $D_{G}\coloneqq \max_{i,j\in V}d_{G}(i,j)$  to denote its diameter.
Given a placement of facilities  $F \subseteq V$ , with  $|F| = k$ , a client placed at vertex  $j \in V$  connects to the closest open facility  $i \in F$ . This is formally captured in Definition 1.
Definition 1. The connection cost of a set of clients  $R \subseteq V$  under the facility-placement  $F \subseteq V$  with  $|F| = k$  equals
$$
C _ {R} (F) := \sum_ {j \in R} \min  _ {i \in F} d _ {G} (j, i)
$$
Next, consider any pair of facility-placements  $F$ ,  $F' \subseteq V$  such that  $|F| = |F'| = k$ . The moving distance between  $F$  and  $F'$  is the minimum overall distance needed to transfer the  $k$  facilities from  $F$  to  $F'$ , formally defined in Definition 2.
Definition 2. Fix any facility-placements  $F, F' \subseteq V$  where  $|F| = |F'| = k$ . Let  $\Sigma$  be the set of all possible matchings from  $F$  to  $F'$ , i.e., each  $\sigma \in \Sigma$  is a one-to-one mapping  $\sigma : F \mapsto F'$  with  $\sigma(i) \in F'$  denoting the mapping of facility  $i \in F$ . The moving cost between  $F$  and  $F'$  equals
$$
M _ {G} (F, F ^ {\prime}) := \min  _ {\sigma \in \Sigma} \sum_ {i \in F} d _ {G} (i, \sigma (i))
$$
At each round  $t \geq 1$ , an online learning algorithm  $\mathcal{A}$  for Problem 1 takes as input all the previous positions of the clients  $R_{1},\ldots ,R_{t - 1} \subseteq V$  and outputs a facility-placement  $F_{t} \coloneqq \mathcal{A}(R_{1},\dots ,R_{t - 1})$
such that  $F_{t} \subseteq V$  and  $|F_{t}| = k$ . The performance of an online learning algorithm is measured by the notion of regret, which we formally introduce in Definition 3.
Definition 3. An online learning algorithm  $\mathcal{A}$  for Problem 1 is called  $\alpha$ -regret with additive regret  $\beta$  if and only if for any sequence of clients  $R_{1},\ldots,R_{T} \subseteq V$ ,
$$
\mathbb {E} \left[ \sum_ {t = 1} ^ {T} C _ {R _ {t}} (F _ {t}) + \gamma \cdot \sum_ {t = 2} ^ {T} M _ {G} (F _ {t - 1}, F _ {t}) \right] \leq \alpha \cdot \min  _ {| F ^ {*} | = k} \sum_ {t = 1} ^ {T} C _ {R _ {t}} (F ^ {*}) + \beta \cdot \sqrt {T}
$$
where  $F_{t} = \mathcal{A}(R_{1},\ldots ,R_{t - 1})$  and  $\alpha ,\beta$  are constants independent of  $T$
An online learning algorithm  $\mathcal{A}$  selects the positions of the  $k$  facilities at each round  $t\geq 1$  solely based on the positions of the clients in the previous rounds,  $R_{1},\ldots ,R_{t - 1}$ . If  $\mathcal{A}$  is  $\alpha$ -regret then Definition 3 implies that its time-averaged overall cost (connection + moving cost) is at most  $\alpha$  times the time-averaged cost of the optimal static solution! Furthermore, the dependency on  $\sqrt{T}$  is known to be optimal [11] and  $\beta$  is typically only required to be polynomially bounded by the size of the input, as for  $T\to \infty$  the corresponding term in the time-averaged cost vanishes.
As already mentioned, the seminal work of [11] implies the existence of an  $(1 + \epsilon)$ -regret algorithm for Problem 1; however, this algorithm requires  $\mathcal{O}(n^k)$  time and space complexity. Prior to this work, the only polynomial-time $^4$  online learning algorithm for Problem 1 was due to Fotakis et al. [31], for the special case of  $\gamma = 0$ . Specifically, in their work the authors design an online learning algorithm with the following guarantee:
Theorem (Fotakis et al. [31]). There exists a randomized online learning algorithm for Problem 1 that runs in polynomial time (w.r.t.  $T$ ,  $n$  and  $\log D_G$ ) such that
$$
\mathbb {E} \left[ \sum_ {t = 1} ^ {T} C _ {R _ {t}} (F _ {t}) \right] \leq \mathcal {O} (k) \cdot \min _ {| F ^ {*} | = k} \sum_ {t = 1} ^ {T} C _ {R _ {t}} (F ^ {*}) + \mathcal {O} (k \cdot n \cdot \sqrt {\log n} \cdot D _ {G}) \cdot \sqrt {T}
$$
Clearly, the algorithm of [31] has not been designed to account for charging the moving of facilities, as indicated by the absence of the moving cost in the above regret guarantee. The main contribution of this work is to obtain (for the first time) regret guarantees that also account for the moving cost.
Theorem 1. There exists a randomized online learning algorithm for Problem 1 (Algorithm 2) that runs in polynomial time (w.r.t.  $T$ ,  $n$  and  $\log D_G$ ) and admits the following regret guarantee:
$$
\mathbb {E} \left[ \sum_ {t = 1} ^ {T} C _ {R _ {t}} \left(F _ {t}\right) + \gamma \cdot \sum_ {t = 2} ^ {T} M _ {G} \left(F _ {t - 1}, F _ {t}\right) \right] \leq \mathcal {O} (\log n) \cdot \min  _ {| F ^ {*} | = k} \sum_ {t = 1} ^ {T} C _ {R _ {t}} \left(F ^ {*}\right) + \beta \cdot \sqrt {T}
$$
for  $\beta = \mathcal{O}(k\cdot n^{3 / 2}\cdot D_G\cdot \max (\gamma ,1))$  and any  $\gamma \geq 0$
Remark 1. We remark that while our additive regret  $\beta$  is larger than the corresponding term in [31] by a factor of  $o(\sqrt{n})$ , our results apply to any  $\gamma \geq 0$  while the algorithm of [31] can generally suffer unbounded moving cost for  $\gamma \to \infty$ , as our experimental results verify.
2.1 HSTs and Metric Embeddings
In this section we provide some preliminary introduction to Hierarchical Separation Trees (HSTs), as they consist a key technical tool towards proving Theorem 1. A weighted tree  $\mathcal{T}(V,E,w)$  is a weighted graph with no cycles. Equivalently, for any pair of vertices  $i,j\in V$  there exists a unique path that connects them. In Definition 4, we establish some basic notation for tree graphs.
Definition 4. Fix any tree  $\mathcal{T}(V,E,w)$ . For every vertex  $u\in V$ ,  $\operatorname{cld}(u)\subseteq V$  denotes the set children vertices of  $u$  and  $p(u)$  denotes its unique parent, i.e.  $u\in \operatorname{cld}(p(u))$ . The root  $r\in V$  of  $\mathcal{T}$  is the unique node with  $p(r) = \emptyset$  and the set  $L(\mathcal{T})\coloneqq \{u\in V:\operatorname{cld}(u) = \emptyset \}$  denotes the leaves of  $\mathcal{T}$ . We use  $\mathrm{dpt}(u)$  to denote the depth of a vertex  $u\in V$ , i.e. the length of the (unique) path from the root  $r$  to  $u$ , and  $h(\mathcal{T})\coloneqq \max_{u\in L(\mathcal{T})}\mathrm{dpt}(u)$  to denote the height of  $\mathcal{T}$ . We use  $\mathrm{lev}(u)\coloneqq h(\mathcal{T}) - \mathrm{dpt}(u)$  to denote the level of a vertex  $u\in V$ . Finally,  $T(u)\subseteq V$  denotes the set of vertices on the sub-tree rooted at  $u$ , i.e. the set of vertices that are descendants of  $u$ .
${}^{3}$  Specifically,the time-averaged overall cost of  $\mathcal{A}$  approaches this upper bound with rate  $\beta  \cdot  {T}^{-1/2}$  .
4Polynomial-time with respect to the input parameters, namely  $T$ ,  $n$  and  $\log D_G$ .
Next, we proceed to define a family of well-structured tree graphs that constitute one of the primary technical tools used in our analysis.
Definition 5. A Hierarchical Separation Tree (HST) is a weighted tree  $\mathcal{T}(V,E,w)$  such that (i) for any node  $u$  and any of its children  $v\in cld(u)$ , the edge  $e = (u,v)$  admits weight  $w_{e} = 2^{\mathrm{lev}(v)}$ , and (ii) the tree is balanced, namely  $\operatorname{lev}(u) = 0$  for all leaves  $u\in L(\mathcal{T})$ .
In their seminal works, [10] and later [24] showed that HSTs can approximately preserve the distances of any graph  $G(V,E,w)$  within some logarithmic level of distortion.
Theorem 2. For any graph  $G(V, E, w)$  with  $|V| = n$  and diameter  $D$ , there exists a polynomial-time randomized algorithm that given as input  $G$  produces an HST  $\mathcal{T}$  with height  $h(\mathcal{T}) \leq \lceil \log D \rceil$  s.t.
1.  $L(\mathcal{T}) = V$ , meaning that the leaves of  $\mathcal{T}$  correspond to the vertices of  $G$ .
2. For any  $u, v \in V$ ,  $d_G(u, v) \leq d_{\mathcal{T}}(u, v)$  and  $\mathbb{E}[d_{\mathcal{T}}(u, v)] \leq \mathcal{O}(\log n) \cdot d_G(u, v)$ .
Theorem 2 states that any weighted graph  $G(V,E,w)$  can be embedded into an HST  $\mathcal{T}$  with  $\mathcal{O}(\log n)$ -distortion. This means that the distance  $d_G(u,v)$  between any pair of vertices  $u,v\in V$  can be approximated by their respective distance  $d_{\mathcal{T}}(u,v)$  in  $\mathcal{T}$  within an (expected) factor of  $\mathcal{O}(\log n)$ .
Remark 2. We note that traditionally HSTs are neither balanced nor are required to have weights that are specifically powers of 2. However, we can transform any general HST into our specific definition, and this has been accounted for in the statement of the above theorem. The details are deferred to Section B of the Appendix.
3 Overview of our approach
In this section we present the key steps of our approach towards designing the  $\mathcal{O}(\log n)$ -regret online learning algorithm for Problem 1. Our approach can be summarized in the following three pillars:
1. In Section 3.1 we introduce a fractional relaxation of Problem 1 in the special case of HSTs (Problem 2). Problem 2 is an artificial problem at which the learner can place a fractional amount of facility to the leaves of an HST so as to fractionally serve the arrived clients. Since the optimal static solution of Problem 2 lower bounds the optimal static solution of Problem 1 in the special case of HSTs, the first step of our approach is to design an  $\mathcal{O}(1)$ -regret algorithm for Problem 2.
2. In Section 3.2 we present the formal guarantees of a novel randomized rounding scheme, called Cut&Round, that is client-oblivious and converts any fractional solution for Problem 2 into an actual placement of  $k$  facilities on the leaves of the HST with just an  $\mathcal{O}(1)$ -overhead in the connection and the moving cost.
3. In Section 3.3 we present how the fractional algorithm for Problem 2 together with the Cut&Round rounding naturally lead to an  $\mathcal{O}(1)$ -regret online learning algorithm for Problem 1 in the special case of HSTs (Algorithm 1). Our main algorithm, presented in Algorithm 2, then consists of running Algorithm 1 into an  $\mathcal{O}(\log n)$  HST embedding of input graph.
3.1 A Fractional Relaxation for HSTs
In this section we introduce a fractional relaxation for Problem 1, called Fractional  $k$ -Clustering with Moving Costs on HSTs (Problem 2). Fix any HST  $\mathcal{T}(V,E,w)$  (in this section,  $V$  denotes the nodes of the HST). We begin by presenting a fractional extension of placing  $k$  facilities on the leaves of  $\mathcal{T}$ .
Definition 6. The set of fractional facility placements  $\mathcal{FP}(\mathcal{T})$  consists of all vectors  $y\in \mathbb{R}^{|V|}$  such that
1.  $y_{v}\in [0,1]$  for all leaves  $v\in L(\mathcal{T})$
2.  $y_{v} = \sum_{u\in \operatorname {cld}(v)}y_{u}$  for all non-leaves  $v\notin L(\mathcal{T})$
3.  $\sum_{v\in L(\mathcal{T})}y_v = k$  , i.e. the total amount of facility on the leaves equals  $k$
For a leaf vertex  $v \in L(\mathcal{T})$ ,  $y_v$  simply denotes the fractional amount of facilities that are placed on it. For all non-leaf vertices  $v \notin L(\mathcal{T})$ ,  $y_v$  denotes the total amount of facility placed in the leaves of the sub-tree  $T(v)$ . Thus, any integral vector  $y \in \mathcal{FP}(\mathcal{T}) \cap \mathbb{N}$  corresponds to a placement of  $k$  facilities on the leaves of  $\mathcal{T}$ .
In Definitions 7 and 8 we extend the notion of connection and moving cost for fractional facility placements. In the special case of integral facility placements, Definitions 7 and 8 respectively collapse to Definitions 1 and 2 (a formal proof is given in Claims 1 and 2 of Section C of the Appendix).
Definition 7. The fractional connection cost of a set of clients  $R \subseteq L(\mathcal{T})$  under  $y \in \mathcal{FP}(\mathcal{T})$  is defined as
$$
f _ {R} (y) := \sum_ {j \in R} \sum_ {v \in P (j, r)} 2 ^ {\operatorname {l e v} (v) + 1} \cdot \max  \left(0, 1 - y _ {v}\right)
$$
where  $P(j,r)$  denotes the set of vertices in the (unique) path from the leaf  $j\in L(\mathcal{T})$  to the root  $r$ .
Definition 8. The fractional moving cost between any  $y, y' \in \mathcal{FP}(\mathcal{T})$  is defined as
$$
| | y - y ^ {\prime} | | _ {\mathcal {T}} := \gamma \cdot \sum_ {v \in V (\mathcal {T})} 2 ^ {l e v (v)} \cdot | y _ {v} - y _ {v} ^ {\prime} |
$$
We are now ready to present our fractional generalization of Problem 1 in the special case of HSTs.
Problem 2 (Fractional  $k$ -Clustering with Moving Costs on HSTs). Fix any HST  $\mathcal{T}$ . At each round  $t = 1, \ldots, T$ :
1. The learner selects a vector  $y^{t}\in \mathcal{FP}(\mathcal{T})$
2. The adversary selects a set of clients  $R_{t} \subseteq L(\mathcal{T})$ .
3. The learner suffers cost  $f_{R_t}(y^t) + ||y^t - y^{t-1}||\mathcal{T}$ .
In Section 4, we develop and present an  $\mathcal{O}(1)$ -regret algorithm for Problem 2 (see Algorithm 3). To this end, we present its formal regret guarantee established in Theorem 3.