SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Before proving Claim 17, let us first show why it suffices to argue about the general case and prove Lemma 7. Let  $y, y' \in \mathcal{FP}(\mathcal{T})$  be any two fractional placements. Recall that  $\|y - y'\|_{\mathcal{T}}$  captures precisely the minimum transportation cost from  $y$  to  $y'$  on  $\mathcal{T}$ . If we break down this transportation plan into small movements of masses between leaves, then we can view it as a sequence of transitions between strictly  $\epsilon$ -neighboring placements. This is formalized in the following claim:
Claim 18. For any  $y, y' \in \mathcal{FP}(\mathcal{T})$ , there exists a finite sequence  $y_0, y_1, \ldots, y_m \in \mathcal{FP}(\mathcal{T})$  of fractional facility placements with  $y = y_0$  and  $y' = y_m$  such that
1.  $y_{j}, y_{j + 1}$  are strictly  $\epsilon$ -neighboring for some  $\epsilon \geq 0$  for  $j = 0, 1, \ldots, m - 1$ .
2.  $||y - y^{\prime}||\tau = \sum_{j = 1}^{m}||y_{j} - y_{j - 1}||\tau .$
We will now prove Lemma 7. Let  $F_{j} = \mathrm{Cut\&Round}(\mathcal{T},y_{j},\alpha)$  be the corresponding output of Cut\&Round on  $y_{j}$  using the same (uniformly random) thresholds  $\alpha_v$ . Then,
$$
\begin{array}{l} \gamma \cdot \mathbb {E} _ {\alpha} \left[ M _ {\mathcal {T}} (F, F ^ {\prime}) \right] \leq \gamma \cdot \mathbb {E} _ {\alpha} \left[ \sum_ {j = 0} ^ {m - 1} M _ {\mathcal {T}} (F _ {j}, F _ {j + 1}) \right] \\ = \gamma \cdot \sum_ {j = 0} ^ {m - 1} \mathbb {E} _ {\alpha} \left[ M _ {\mathcal {T}} \left(F _ {j}, F _ {j + 1}\right) \right] \\ \leq 4 \cdot \sum_ {j = 0} ^ {m - 1} \left| \left| y _ {j} - y _ {j + 1} \right| \right| \tau \\ = 4 \cdot | | y - y ^ {\prime} | | \tau \\ \end{array}
$$
In the above calculation, the first inequality holds from the fact that the minimum transportation cost satisfies the triangular inequality. The first equality holds from linearity of expectation. The second inequality holds from Claim 17 and the second equality holds from Claim 18.
Thus, we have shown that proving Lemma 7 for the special case of strictly  $\epsilon$ -neighboring fractional facility placements  $y, y'$  suffices to prove Lemma 7 for the general case of any  $y, y' \in \mathcal{FP}(\mathcal{T})$  and conclude this section. The rest of this section is dedicated to proving Claim 17, which is the main technical challenge towards proving Lemma 7.
Proof of Claim 17. Fix any pair of strictly  $\epsilon$ -neighboring fractional facility placements  $y, y' \in \mathcal{FP}(\mathcal{T})$  and let the corresponding outputs of  $\mathrm{Cut}\& \mathrm{Round}$  be  $F = \mathrm{Cut}\& \mathrm{Round}(\mathcal{T}, y, \alpha)$  and  $F' = \mathrm{Cut}\& \mathrm{Round}(\mathcal{T}, y', \alpha)$ . In section E.1 we have already shown that  $F, F' \subseteq L(\mathcal{T})$  are valid facility placements since  $|F| = |F'| = k$ . Let  $Y, Y' \in \mathcal{FP}(\mathcal{T})$  be used to denote the corresponding integral placements, i.e.
$Y_{v}:= |L(\mathcal{T})\cap F|=$  number of facilities in  $F$  placed on the leaves of the sub-tree rooted at  $\upsilon$
$Y_{v}^{\prime} \coloneqq |L(\mathcal{T}) \cap F^{\prime}| =$  number of facilities in  $F^{\prime}$  placed on the leaves of the sub-tree rooted at  $v$
Recall that  $Y$  and  $Y'$  are precisely the values of the  $Y$ -variables that algorithm Cut&Round sets. As shown in Claim 2 of Section E.4, we know that  $\gamma \cdot M_{\mathcal{T}}(F, F') = ||Y - Y'||_{\mathcal{T}}$ . Thus, in order to prove Claim 17, we need to show that
$$
\mathbb {E} _ {\alpha} \left[ | | Y - Y ^ {\prime} | | _ {\mathcal {T}} \right] \leq 4 \cdot | | y - y ^ {\prime} | | _ {\mathcal {T}}
$$
Since  $y, y'$  are strictly  $\epsilon$ -neighboring fractional facility placements, we know that there exist two leaves  $s, t \in L(\mathcal{T})$  with lowest common ancestor  $p \in V(\mathcal{T})$  such that  $|y_v - y'_v|$  is  $\epsilon$  among vertices on the (unique) path from  $s$  to  $t$  (excluding vertex  $p$ ) and is 0 otherwise. Let  $L = \mathrm{lev}(p)$ . Then, by definition of  $||\cdot||_{\mathcal{T}}$  we have
$$
\left| \left| y - y ^ {\prime} \right| \right| _ {\mathcal {T}} = \sum_ {v \in V (\mathcal {T})} 2 ^ {\operatorname {l e v} (v)} \cdot \left| y _ {v} - y _ {v} ^ {\prime} \right| = 2 \epsilon \cdot \sum_ {l = 0} ^ {L - 1} 2 ^ {l} = 2 \epsilon \cdot \left(2 ^ {L} - 1\right) \tag {5}
$$
Furthermore, recall that from Lemma 5, Cut&Round rounds  $y_v$  to either  $Y_v = \lfloor y_v \rfloor + 1$  with probability  $\delta(y_v)$  or to  $\lfloor y_v \rfloor$  with probability  $1 - \delta(y_v)$ . Since  $\epsilon$  is sufficiently small so that either  $\lfloor y_v \rfloor = \lfloor y_v' \rfloor$  or  $|y_v - y_v'| \leq 1$  and at lowest one of the two is integral (and it is thus always rounded to itself), we get that  $|Y_v - Y_v'| \leq 1$  for all  $v \in V(\mathcal{T})$ . This implies that
$$
\begin{array}{l} \mathbb {E} _ {\alpha} \left[ | | Y - Y ^ {\prime} | | _ {\mathcal {T}} \right] = \mathbb {E} _ {\alpha} \left[ \sum_ {v \in V (\mathcal {T})} 2 ^ {\operatorname {l e v} (v)} \cdot | Y _ {v} - Y _ {v} ^ {\prime} | \right] \\ = \sum_ {v \in V (\mathcal {T})} 2 ^ {\operatorname {l e v} (v)} \cdot \mathbb {E} _ {\alpha} \left[ \left| Y _ {v} - Y _ {v} ^ {\prime} \right| \right] \\ = \sum_ {v \in V (\mathcal {T})} 2 ^ {\operatorname {l e v} (v)} \cdot \mathbb {P} _ {\alpha} \left[ \left| Y _ {v} - Y _ {v} ^ {\prime} \right| = 1 \right] \\ \end{array}
$$
Let  $l \in [0, h(\mathcal{T})]$  be any level on the HST  $\mathcal{T}$  and let  $C_l$  be used to denote the expected number of vertices at level  $l$  that are rounded to different values, i.e.
$$
C _ {l} := \mathbb {E} _ {\alpha} \left[ | \left\{v \in V (\mathcal {T}): \operatorname {l e v} (v) = l \text {a n d} Y _ {v} \neq Y _ {v} ^ {\prime} \right\} | \right]
$$
Then, the above imply that
$$
\mathbb {E} _ {\alpha} \left[ | | Y - Y ^ {\prime} | | _ {\mathcal {T}} \right] = \sum_ {l = 0} ^ {h (\mathcal {T})} 2 ^ {l} \cdot C _ {l} \tag {6}
$$
It remains to compute  $C_l$  for all  $l \in [0, h(\mathcal{T})]$ . This is done in Claim 19, where we prove that  $C_l = 0$  for  $l \geq L$  (the level of  $s$  and  $t$ 's lowest common ancestor) and  $C_l \leq 4\epsilon \cdot (L - l)$  otherwise. Combining this claim with equations (5) and (6) immediately implies that
which completes the proof of Claim 17.
Claim 19. For any  $l \geq L$ ,  $C_l = 0$ . For any  $l < L$ ,  $C_l \leq 4\epsilon \cdot (L - l)$ .
Proof. Recall that for fixed thresholds  $\alpha_{v}$ , the output of Cut&Round is deterministic. Since  $L$  is the level of vertex  $p$  (the lowest common ancestor of leaves  $s, t$ ) and by definition of strictly  $\epsilon$ -neighboring placements  $y, y'$  we know  $y_{v} = y_{v}'$  for any vertex  $v$  such that  $\mathrm{lev}(v) \geq L$ , we immediately get that  $C_{l} = 0$  for any  $l \geq L$ .
We will now proceed to analyze  $C_l$  for any  $l < L$ . We partition the set of vertices  $v \in V(\mathcal{T})$  with  $\operatorname{lev}(v) = l$  into three sets:
- A vertex  $v$  is called active if it lies on the (unique) path between leaves  $s$  and  $t$ .
- A vertex  $v$  is called inactive if it is not a descendant of  $p$  (the lowest common ancestor of leaves  $s$  and  $t$ ).
- A vertex  $v$  is called affected if it is not active and is a descendant of  $p$ .
Obviously, each vertex  $v$  with  $\operatorname{lev}(v) = l$  must lie in exactly one of these sets.
Inactive Vertices. We will prove that for every inactive vertex  $v$ ,  $\mathbb{P}_{\alpha}[Y_v \neq Y_v'] = 0$ . Since the same set of thresholds  $\alpha$  is used to round both  $y$  and  $y'$ , the output of Cut&Round is deterministic. Furthermore, if a vertex  $v$  is inactive, then we know that  $y_v = y_v'$  and also  $y_u = y_u'$  for any ancestor vertex of  $u$  of  $v$  (by Definition 10 of neighboring facility placements). Thus, this immediately implies that  $Y_v = Y_v'$  with probability 1 and thus we do not need to account for inactive vertices when computing  $C_l$ .
Active Vertices. We will prove that for every active vertex  $v$ ,  $\mathbb{P}_{\alpha}[Y_v \neq Y_v'] = \epsilon$ . Recall that any active vertex is either an ancestor of leaf  $s$  or leaf  $t$ . We will only prove the claim in the case when  $v$  is an ancestor of  $t$ ; the other case is completely analogous. A formal proof by induction is given in Claim 20, presented at the end of this section. As a direct corollary, since there are only two active vertices per level, the expected number of active vertices in level  $l$  that are rounded two different values is precisely  $2\epsilon$ .
Affected Vertices. Finally, we will now analyze the affected vertices. By definition, we know that each affected vertex  $v$  will have a unique active ancestor (also counting  $p$ ). We partition the set of affected vertices on level  $l$  into  $2(L - l - 1) + 1$  groups, based on their corresponding active ancestor. The main argument we need to establish is that by definition of Round&Cut, at most one vertex in each of these groups can be rounded to a different value.
To see this, observe that Round&Cut is monotone, in the sense that if  $y_v' \geq y_v$  and also  $y_u' \geq y_u$  for all ancestors  $u$  of  $v$ , then (assuming the same set of thresholds is used), we know that  $Y_v' \geq Y_v$ . Using this fact on the vertices of a group, since all of them can either only increase or decrease, in order to maintain balance at most one of them can change, otherwise we would get a change of 2 or more on the parent node which cannot happen.
Furthermore, for a specific group, if both the common active ancestor and its child  $u$  with  $y_u \neq y_u'$  end up rounded to the same value, we get (from the fact that the same thresholds are used) that all the vertices in the group will be rounded to the same value. Thus, in order for a (unique) vertex in any group to change, at least one of two active vertices must change, which happens with probability at most  $2\epsilon$ . Since there are  $2(L - l - 1) + 1$  groups, we get as a corollary that the expected number of affected vertices at level  $l$  that get rounded to a different value is at most  $2\epsilon \cdot (2L - 2l - 1)$ .
Combining everything, we get that  $C_l \leq 0 + 2\epsilon + 2\epsilon \cdot (2L - 2l - 1) = 4\epsilon \cdot (L - l)$ .
Claim 20. Let  $v$  be any active vertex that is an ancestor of  $t$ . Then,  $\mathbb{P}_{\alpha}[Y_v \neq Y_v'] = \epsilon$ .
Proof. In fact, we will in fact prove the following stronger claim,
-  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor \text{ and } Y_v' = \lfloor y_v \rfloor] = 1 - \delta(y_v) - \epsilon.$
-  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor$  and  $Y_v' = \lfloor y_v \rfloor + 1] = \epsilon$ .
-  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor + 1$  and  $Y_v' = \lfloor y_v \rfloor] = 0$
-  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor + 1$  and  $Y_v' = \lfloor y_v \rfloor + 1] = \delta(y_v)$
which clearly implies Claim 20.
Once again, we will prove the claim via induction, starting from the highest active ancestor of  $t$  at level  $l = L - 1$  and moving towards the leaf  $t$  at level  $l = 0$ . We begin by mentioning that for vertex  $p$  ( $s$  and  $t$ 's lowest common ancestor at level  $L$ ) we know for sure that  $Y_{p} = Y_{p}^{\prime}$  since  $y_{p} = y_{p}^{\prime}$  and  $y_{u} = y_{u}^{\prime}$  for any  $u$  such that  $\mathrm{lev}(u) \geq L$ ; thus, since the same set of thresholds  $\alpha$  is used, the execution of Cut&Round will be identical up to this point.
We assume that the first child of any vertex  $v$  visited by Alloc is always the active child; this can be done without loss of generality as the order that Alloc visits the vertices hasn't played any part on our analysis yet.
Base of the induction. For the base of the induction, let  $v$  be the (unique) child of  $p$  that is an ancestor of  $t$ ; i.e. let  $v$  be the highest active ancestor of  $t$ . We have already mentioned that  $Y_{p} = Y_{p}^{\prime}$  with probability 1. Thus, it can either be the case that  $Y_{p} = Y_{p}^{\prime} = \lfloor y_{p} \rfloor$  of  $Y_{p} = Y_{p}^{\prime} = \lfloor y_{p} \rfloor + 1$ . From Lemma 5 we know that the first happens with probability  $1 - \delta(y_{p})$  and the latter with probability  $\delta(y_{p})$ . We distinguish between the following cases:
- Let  $\delta(y_v) < \delta(y_p)$ . Then, if  $Y_p = Y_p' = \lfloor y_p \rfloor$  we know from the description of Alloc that  $Y_v = Y_v' = \lfloor y_v \rfloor$  with probability 1. On the other hand, if  $Y_p = Y_p' = \lfloor y_p \rfloor + 1$ , we know that  $Y_v = \lfloor y_v \rfloor + 1$  if  $\alpha_v \leq \delta(y_v) / \delta(y_p)$  and likewise  $Y_v' = \lfloor y_v \rfloor + 1$  if  $\alpha_v \leq (\delta(y_v) + \epsilon) / (\delta(y_p))$ . Thus, by conditioning on the values of  $Y_p$  and  $Y_p'$ , we get
1.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor$  and  $Y_v' = \lfloor y_v \rfloor] = (1 - \delta(y_p)) \cdot 1 + \delta(y_p) \cdot (1 - \frac{\delta(y_v) + \epsilon}{\delta(y_p)}) = 1 - \delta(y_v) - \epsilon.$
2.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor$  and  $Y_v' = \lfloor y_v \rfloor + 1] = (1 - \delta(y_p)) \cdot 0 + \delta(y_p) \cdot \left( \frac{\delta(y_v) + \epsilon}{\delta(y_p)} - \frac{\delta(y_v)}{\delta(y_p)} \right) = \epsilon.$
3.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v\rfloor +1$  and  $Y_{v}^{\prime} = \lfloor y_{v}\rfloor ] = (1 - \delta (y_{p}))\cdot 0 + \delta (y_{p})\cdot 0 = 0$
4.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor + 1$  and  $Y_v' = \lfloor y_v \rfloor + 1] = (1 - \delta(y_p)) \cdot 0 + \delta(y_p) \cdot \frac{\delta(y_v)}{\delta(y_p)} = \delta(y_v)$ .
- Let  $\delta(y_v) \geq \delta(y_p)$ . Then, if  $Y_p = Y_p' = \lfloor y_p \rfloor + 1$  we know from the description of Alloc that  $Y_v = Y_v' = \lfloor y_v \rfloor + 1$  with probability 1. On the other hand, if  $Y_p = Y_p' = \lfloor y_p \rfloor$ , we know that  $Y_v = \lfloor y_v \rfloor + 1$  if  $\alpha_v \leq (\delta(y_v) - \delta(y_p)) / (1 - \delta(y_p))$  and likewise  $Y_v' = \lfloor y_v \rfloor + 1$  if  $\alpha_v \leq (\delta(y_v) + \epsilon - \delta(y_p)) / (1 - \delta(y_p))$ . Thus, by conditioning on the values of  $Y_p$  and  $Y_p'$ , we get
1.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor \text{ and } Y_v' = \lfloor y_v \rfloor] = (1 - \delta(y_p)) \cdot (1 - \frac{\delta(y_v) + \epsilon - \delta(y_p)}{1 - \delta(y_p)}) + \delta(y_p) \cdot 0 = 1 - \delta(y_v) - \epsilon.$
2.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v\rfloor$  and  $Y_{v}^{\prime} = \lfloor y_{v}\rfloor ] = (1 - \delta (y_{p}))\cdot (\frac{\delta(y_{v}) + \epsilon - \delta(y_{p})}{1 - \delta(y_{p})} -\frac{\delta(y_{v}) - \delta(y_{p})}{1 - \delta(y_{p})}) + \delta (y_{p})\cdot$ $0 = \epsilon$
3.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor + 1$  and  $Y_v' = \lfloor y_v \rfloor] = (1 - \delta(y_p)) \cdot 0 + \delta(y_p) \cdot 0 = 1 - \delta(y_v) = 0.$
4.  $\mathbb{P}_{\alpha}[Y_v = \lfloor y_v \rfloor + 1$  and  $Y_v' = \lfloor y_v \rfloor] = (1 - \delta(y_p)) \cdot \frac{\delta(y_v) - \delta(y_p)}{1 - \delta(y_p)} + \delta(y_p) \cdot 1\delta(y_v)$ .
So in both cases, the base of the induction holds.
Inductive Step. Using the exact same approach, we can prove the claim for any active ancestor  $u$  of  $t$ , assuming that the claim holds for  $u$ 's father  $v = p(u)$ . The only difference, is that now we can't claim that  $Y_v = Y_v'$  with probability 1. Instead, there are three different cases that we need to consider; namely
1.  $Y_{v} = Y_{v}^{\prime} = |y_{v}|$  with probability  $1 - \epsilon - \delta(y_{v})$ .
2.  $Y_{v} = Y_{v}^{\prime} = \lfloor y_{v}\rfloor +1$  with probability  $\delta (y_v)$
3.  $Y_{v} = \lfloor y_{v}\rfloor$  and  $Y_{v}^{\prime} = \lfloor y_{v}\rfloor +1$  with probability  $\epsilon$
where the probabilities hold from the inductive hypothesis on the parent vertex  $v$ . Next, we will need to once again consider the case of whether  $\delta(y_u) < \delta(y_v)$  or not (notice that the same relation will hold for  $y_u'$  and  $y_v'$ ) and use the description of Alloc to get the assignment probabilities. Since this is a simple matter of arithmetic, the details are omitted.
F Experimental Evaluation
In this section we experimentally evaluate the performance of Algorithm 2 with respect to the best fixed facility placement and compare it with the respective performance of the algorithm proposed by [31]. In all the following experiments the step-size of Algorithm 3 (subroutine of Algorithm 2) is set to  $\eta := \max(\gamma, 1)\sqrt{nT}$ .
Periodically Moving Clients. We first present a simple setting to indicate the inefficiency of the online learning algorithm of [31] in handling moving costs. In this experiment the underlying graph is the 0.01-discretization of  $[0,1]\times [0,1]$ . At each round  $t\geq 1$ , we periodically select one of four balls of radius  $R = 0.2$  depicted in Figure 1 and then a client arrives uniformly at random on the selected ball. In Figure 1 and Table 1 we present the overall cost of Algorithm 2 and the algorithm of [31] for different values of facility-weight  $\gamma$ ,  $k = 3$  facilities and  $T = 4000$  time-steps. In all cases, the facilities of Algorithm 2 eventually converge to three of the four ball-centers, which is the optimal fixed facility placement. As the experiment reveals, the algorithm of [31] admits significantly larger cost as the facility-weight increases while Algorithm 2 is robust to the increase.
Figure 1: We plot the evolution of the approximation ratio for Algorithm 2 (red curve) and the algorithm from [31] (blue curve) compared to the hindsight optimal facility placement for facility weights  $\gamma = 0, \gamma = 1$  and  $\gamma = 10$ . Both scales are logarithmic. The bottom-right plot depicts the facilities eventually placed by our Algorithm 2 which coincides with the optimal configuration
Table 1: Ratio of the overall cost of both algorithms with respect to to the hindsight optimal (20 runs).
MovingClients	γ = 0	γ = 1	γ = 10
[31]	1.297 ± 0.045	1.943 ± 0.466	3.388 ± 1.335
Algorithm 2	1.083 ± 0.001	1.091 ± 0.001	1.343 ± 0.014
Real-World Datasets. We evaluate the performance of Algorithm 2 on the MNIST and CIFAR10 datasets. We randomly sample  $n = 10000$  images and construct a graph where each image corresponds to a vertex with the edge weights given by the Euclidean distance of the respective images. At each round  $t$ , an image is sampled uniformly at random and a client arrives in the corresponding vertex. We then evaluate Algorithm 2 in the latter setting for  $T = 3000$  rounds and  $k = 10$  facilities. In Table 2 we present the ratio of the overall cost of Algorithm 2 over the ratio cost of the fractional hindsight optimal<sup>7</sup>. As our experiments indicate, the sub-optimality of Algorithm 2 is way smaller than the theoretical  $\mathcal{O}(\log n)$  upper bound on the regret.
Table 2: The ratio of the cost of Algorithm 2 with respect to the cost of the fractional hindsight optimal facility placement (20 runs).
Algorithm 2	γ = 0	γ = 1	γ = 10
MNIST	1.118 ± 0.01	1.403 ± 0.04	1.5631 ± 0.03
CIFAR10	1.113 ± 0.01	1.189 ± 0.04	1.59 ± 0.31
Beyond unit batch sizes and random arrivals. Finally, we once again evaluate the performance of Algorithm 2 on the MNIST and CIFAR10 datasets. This time, the requests arrive in batches of size  $R = 10$  for  $T = 3000$  rounds. In order to go beyond the random arrival model, we first sample the  $R \cdot T$  requested vertices uniformly at random from  $[n]$  and then we proceed to order them based on their respective categories, using the lexicographical vector order to break ties. Then, we partition these requests into  $T$  batches of size  $R$  and sequentially reveal them to the algorithm as usual. As a result, all images/vertices from the first category are requested first, then the second etc.
In Table 3, we present our experimental evaluations on the above constructed sequence. As our experiments indicate, our algorithm admits way better performance than the theoretical  $O(\log n)$  guarantees even in sequences with higher batch sizes and non-random arrivals.
Table 3: The ratio of the cost of our Algorithm with respect to the cost of the fractional hindsight optimal facility placement.
Our Algorithm	γ = 0	γ = 1	γ = 10
CIFAR10	1.050	1.048	1.051
MNIST	1.082	1.045	1.12
The cost of the fractional hindsight optimal can be efficiently computed [31] and lower bounds the cost of the optimal facility placement. As a result, the presented ratios in Tables 2 and 3 are upper bounds on the actual ratio of Algorithm 2 and the optimal facility-placement.