SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} B ^ {\prime} - B = \sum_ {v \in I} \left(\mu_ {v} ^ {\prime} - \mu_ {v}\right) \\ = \sum_ {v \in I} w _ {v} \cdot (y _ {p (v)} ^ {\prime} + \delta_ {p (v)}) \cdot (\frac {y _ {v} ^ {\prime} + \delta_ {v}}{y _ {p (v)} ^ {\prime} + \delta_ {p (v)}} - \frac {y _ {v} + \delta_ {v}}{y _ {p (v)} + \delta_ {p (v)}}) \\ = \sum_ {v \notin L (\mathcal {T})} \frac {w _ {v}}{2} \cdot \left(y _ {v} ^ {\prime} + \delta_ {v}\right) \cdot \sum_ {u \in \operatorname {c l d} (v) \cap I} \left(\frac {y _ {u} ^ {\prime} + \delta_ {u}}{y _ {v} ^ {\prime} + \delta_ {v}} - \frac {y _ {u} + \delta_ {u}}{y _ {v} + \delta_ {v}}\right) \\ = \frac {1}{2} \cdot \sum_ {v \notin L (\mathcal {T})} \frac {w _ {v}}{2} \cdot \left(y _ {v} ^ {\prime} + \delta_ {v}\right) \cdot \sum_ {u \in \operatorname {c l d} (u)} \left| \frac {y _ {u} ^ {\prime} + \delta_ {u}}{y _ {v} ^ {\prime} + \delta_ {v}} - \frac {y _ {u} + \delta_ {u}}{y _ {v} + \delta_ {v}} \right| \\ \end{array}
$$
where the last equality follows from the fact that the ratio in the inner sum always sum to 1, and thus by only summing over the ones with positive difference we get half of the total sum of absolute differences. By swapping the summation order once again, we get
$$
\begin{array}{l} B ^ {\prime} - B = \frac {1}{2} \cdot \sum_ {v \neq r} w _ {v} \cdot (y _ {p (v)} ^ {\prime} + \delta_ {p (v)}) \cdot | \frac {y _ {v} ^ {\prime} + \delta_ {v}}{y _ {p (v)} ^ {\prime} + \delta_ {p (v)}} - \frac {y _ {v} + \delta_ {v}}{y _ {p (v)} + \delta_ {p (v)}} | \\ = \frac {1}{2} \cdot h (y ^ {\prime}, y) \\ \end{array}
$$
and from inequality (3) we finally get
$$
f (y ^ {\prime}, y) \geq \frac {1}{2 k D} \cdot \left(h (y ^ {\prime}, y)\right) ^ {2}
$$
Claim 5. For any  $y, y' \in \mathcal{FP}(\mathcal{T})$ , it holds that  $||y' - y||_{\mathcal{T}} \leq 2\gamma \cdot h(y', y)$ .
Proof. To prove the claim, we first need to establish some extra notation. For any  $y \in \mathcal{FP}(\mathcal{T})$  and  $v \neq r$ , let
$$
\lambda_ {v} (y) := \frac {y _ {v} + \delta_ {v}}{y _ {p (v)} + \delta_ {p (v)}}
$$
Furthermore, for any vertex  $v$  and any integer  $i \in [0, h - lev(v)]$ , we use  $p(v, i)$  to denote the  $i$ -th ancestor of  $v$  on  $\mathcal{T}$ , for example  $p(v, 0) = v$ ,  $p(v, 1) = p(v)$  and  $p(v, h - lev(v)) = r$ .
Recall that by definition,  $y_{r} = \delta_{r} = k$ . Thus, if we telescope these terms and let  $m_v = h - lev(v) - 1$ , we clearly have that
$$
y _ {v} + \delta_ {v} = 2 k \cdot \Pi_ {i = 0} ^ {m _ {v}} \lambda_ {p (v, i)} (y)
$$
$$
\begin{array}{l} y _ {v} ^ {\prime} - y _ {v} = 2 k \cdot \Pi_ {i = 0} ^ {m _ {v}} \lambda_ {p (v, i)} \left(y ^ {\prime}\right) - 2 k \cdot \Pi_ {i = 0} ^ {m _ {v}} \lambda_ {p (v, i)} (y) \\ = 2 k \cdot \sum_ {i = 0} ^ {m _ {v}} \lambda_ {p (v, 0)} \left(y ^ {\prime}\right) \cdot \dots \cdot \left(\lambda_ {p (v, i)} \left(y ^ {\prime}\right) - \lambda_ {p (v, i)} (y)\right) \cdot \dots \cdot \lambda_ {p (v, m _ {v})} (y) \\ = 2 k \cdot \sum_ {i = 0} ^ {m _ {v}} \frac {y _ {v} ^ {\prime} + \delta_ {v}}{y _ {p (v , i)} ^ {\prime} + \delta_ {p (v , i)}} \cdot \left(\lambda_ {p (v, i)} \left(y ^ {\prime}\right) - \lambda_ {p (v, i)} (y)\right) \cdot \frac {y _ {p (v , i + 1)} + \delta_ {p (v , i + 1)}}{2 k} \\ = \left(y _ {v} ^ {\prime} + \delta_ {v}\right) \cdot \sum_ {i = 0} ^ {m _ {v}} \frac {y _ {p (v , i + 1)} + \delta_ {p (v , i + 1)}}{y _ {p (v , i)} ^ {\prime} + \delta_ {p (v , i)}} \cdot \left(\lambda_ {p (v, i)} \left(y ^ {\prime}\right) - \lambda_ {p (v, i)} (y)\right) \\ \end{array}
$$
and from the triangular inequality
$$
\left| y _ {v} ^ {\prime} - y _ {v} \right| \leq \left(y _ {v} ^ {\prime} + \delta_ {v}\right) \cdot \sum_ {i = 0} ^ {m _ {v}} \frac {y _ {p (v , i + 1)} + \delta_ {p (v , i + 1)}}{y _ {p (v , i)} ^ {\prime} + \delta_ {p (v , i)}} \cdot \left| \lambda_ {p (v, i)} \left(y ^ {\prime}\right) - \lambda_ {p (v, i)} (y) \right| \tag {4}
$$
Plugging inequality (4) into the definition of norm  $||\cdot ||_{\mathcal{T}}$  , we get
$$
\left. \right.\left|\left| y ^ {\prime} - y \right|\right| _ {\mathcal {T}} \leq \gamma \cdot \sum_ {v \neq r} w _ {v} \cdot \left(y _ {v} ^ {\prime} + \delta_ {v}\right) \cdot \left(\sum_ {i = 0} ^ {m _ {v}} \frac {y _ {p (v , i + 1)} + \delta_ {p (v , i + 1)}}{y _ {p (v , i)} ^ {\prime} + \delta_ {p (v , i)}} \cdot \left| \lambda_ {p (v, i)} \left(y ^ {\prime}\right) - \lambda_ {p (v, i)} (y) \right|\right)
$$
and by carefully exchanging the summation order, we obtain
$$
\left| \left| y ^ {\prime} - y \right| \right| \tau \leq \gamma \cdot \sum_ {v \neq r} \frac {y _ {p (v)} + \delta_ {p (v)}}{y _ {v} ^ {\prime} + \delta_ {v}} \cdot \left| \lambda_ {v} \left(y ^ {\prime}\right) - \lambda_ {v} (y) \right| \cdot \left(\sum_ {u \in T (v)} w _ {u} \left(y _ {u} ^ {\prime} + \delta_ {u}\right)\right)
$$
Finally, observe that  $\sum_{u\in T(v)}w_uy_u'\leq 2w_vy_v'$ . To see this, fix the sub-tree  $T(v)$  rooted at vertex  $v$  and recall that since  $y^\prime \in \mathcal{FP}(\mathcal{T})$ , the total amount of facilities at each level is  $y_{v}^{\prime}$ . Furthermore, the weights  $w_{v}$  decrease by a factor of 2 at every level. Using the same arguments, we obtain  $\sum_{u\in T(v)}w_u\delta_u\leq 2w_v\delta_v$ . Combining everything, we finally get
$$
\left| \left| y ^ {\prime} - y \right| \right| _ {\mathcal {T}} \leq 2 \gamma \cdot \sum_ {v \neq r} w _ {v} \cdot \left(y _ {p (v)} + \delta_ {p (v)}\right) \cdot \left| \lambda_ {v} \left(y ^ {\prime}\right) - \lambda_ {v} (y) \right|
$$
or equivalently,  $||y' - y||_{\mathcal{T}} \leq 2\gamma \cdot h(y', y)$ .
D.2 Bounding the Moving Cost (Proof of Lemma 2)
In this section we will upper bound the moving cost of FTRL by its connection cost. Fix any sequence of client requests  $R_{1}, R_{2}, \ldots, R_{T} \subseteq L(\mathcal{T})$ . Recall that at each step  $t$ , FTRL selects a fractional facility placement  $y^{t}$  given by
$$
y^{t} = \operatorname *{arg  min}_{y\in \mathcal{FP}(\mathcal{T})}\Phi_{t}(y)
$$
where  $\Phi_t(y) = \sum_{s=1}^{t-1} f_{R_s}(y) + \frac{1}{\eta} \cdot R_{\mathcal{T}}(y)$  is the objective that FTRL minimizes over at step  $t$  for  $\eta = (\gamma \cdot \sqrt{nT})^{-1}$ . In this section, we prove Lemma 2, by arguing that
$$
\sum_ {t = 2} ^ {T} \left| \left| y ^ {t} - y ^ {t - 1} \right| \right| \tau \leq \frac {1}{2} \cdot \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t}\right) + \frac {\eta}{2 \alpha} \cdot T
$$
since the proof follows easily by the definitions of  $\eta$  and  $\alpha$ .
From Lemma 1 we already know that  $R_{\mathcal{T}}$  is  $\alpha$ -strongly convex with respect to  $||\cdot||_{\mathcal{T}}$  for  $\alpha = (8kD\gamma^2)^{-1}$ . Furthermore, by definition the fractional connection cost
$$
f _ {R} (y) = \sum_ {j \in R} \sum_ {v \in P (j, r)} 2 ^ {\operatorname {l e v} (v) + 1} \cdot \max  \left(0, 1 - y _ {v}\right)
$$
is clearly convex for any client request  $R \subseteq L(\mathcal{T})$ . Thus, it is straightforward to argue that at any step  $t$ , the FTRL objective  $\Phi_t$  is  $\frac{\alpha}{\eta}$ -strongly convex with respect to  $|| \cdot ||_{\mathcal{T}}$ . Unfortunately,  $f_R(y)$  is not differentiable on  $\mathcal{FP}(\mathcal{T})$ , but its sub-gradients are well-defined on any  $y \in \mathcal{FP}(\mathcal{T})$ . Thus, the strong convexity of  $\Phi_t$  provides us with the following guarantee:
Claim 6. Fix any pair of fractional facility placements  $y, y' \in \mathcal{FP}(\mathcal{T})$  and any time step  $t \in [T]$ . Let  $g_t \in \partial \Phi_t(y)$  be any sub-gradient of  $\Phi_t$  at  $y$ . Then, it holds that
$$
\Phi_ {t} \left(y ^ {\prime}\right) \geq \Phi_ {t} (y) + \langle g _ {t}, y ^ {\prime} - y \rangle + \frac {\alpha}{\eta} \cdot | | y - y ^ {\prime} | | _ {T} ^ {2}
$$
Furthermore, since by definition  $y^{t}$  is the (unique) minimizer of  $\Phi_t$ , the first order optimality conditions on  $\Phi_t$  imply that there exists some  $g_{t}^{*}\in \partial \Phi_{t}(y^{t})$  such that  $\langle g_t^*,y - y^t\rangle \geq 0$  for any  $y\in \mathcal{FP}(\mathcal{T})$ . Claim 6 for  $y = y^{t}$ ,  $y^\prime = y^{t - 1}$  and  $g_{t} = g_{t}^{*}$  gives us
$$
\Phi_ {t} \left(y ^ {t - 1}\right) \geq \Phi_ {t} \left(y ^ {t}\right) + \frac {\alpha}{\eta} \cdot \left| \left| y ^ {t} - y ^ {t - 1} \right| \right| _ {T} ^ {2}
$$
$$
\begin{array}{l} \left\| y ^ {t} - y ^ {t - 1} \right\| _ {\mathcal {T}} ^ {2} \leq \frac {\eta}{\alpha} \cdot \left(\Phi_ {t} \left(y ^ {t - 1}\right) - \Phi_ {t} \left(y ^ {t}\right)\right) \\ = \frac {\eta}{\alpha} \cdot \left(\Phi_ {t - 1} \left(y ^ {t - 1}\right) + f _ {R _ {t - 1}} \left(y ^ {t - 1}\right) - \Phi_ {t - 1} \left(y ^ {t}\right) - f _ {R _ {t - 1}} \left(y ^ {t}\right)\right) \\ \leq \frac {\eta}{\alpha} \cdot \left(f _ {R _ {t - 1}} \left(y ^ {t - 1}\right) - f _ {R _ {t - 1}} \left(y ^ {t}\right)\right) \\ \end{array}
$$
where for the equality we used the fact that  $\Phi_t(y) = \Phi_{t-1}(y) + f_{R_{t-1}}(y)$  and for the second inequality we used the fact that  $y^{t-1}$  is by definition the minimizer of  $\Phi_{t-1}$ . Finally, since  $f_R(y) \geq 0$  for any client request  $R \subseteq L(\mathcal{T})$ , we have
$$
\begin{array}{l} \left| \left| y ^ {t} - y ^ {t - 1} \right| \right| \tau \leq \sqrt {\frac {\eta}{\alpha} \cdot f _ {R _ {t - 1}} \left(y ^ {t - 1}\right)} \\ \leq \frac {\eta}{2 \alpha} + \frac {1}{2} \cdot f _ {R _ {t - 1}} (y ^ {t - 1}) \\ \end{array}
$$
where the last inequality follows from the Arithmetic Mean - Geometric Mean inequality. Summing over all  $t$  completes the proof of Lemma 2.
D.3 Bounding the Connection Cost (Proof of Lemma 3)
In this section we will upper bound the connection cost of FTRL by the connection cost of the optimal fractional facility placement in hindsight. This is a standard analysis found in many textbooks, and we present it just for the sake of completeness.
Fix any sequence of client requests  $R_{1}, R_{2}, \ldots, R_{T} \subseteq L(\mathcal{T})$ . Recall that at each step  $t$ , FTRL selects a fractional facility placement  $y^{t}$  given by
where  $\Phi_t(y) = \sum_{s=1}^{t-1} f_{R_s}(y) + \frac{1}{\eta} \cdot R_{\mathcal{T}}(y)$  is the objective that FTRL minimizes over at step  $t$  for  $\eta = (\gamma \cdot \sqrt{nT})^{-1}$ . Let  $y^*$  be the optimal facility placement in hindsight, i.e.
$$
y ^ {*} = \underset {y \in \mathcal {F P} (\mathcal {T})} {\arg \min } \sum_ {t = 1} ^ {T} f _ {R _ {t}} (y)
$$
In this section we prove Lemma 3, by arguing that
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t}\right) \leq \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \frac {k n D}{\eta} + 3 2 k n ^ {2} D \eta \cdot T
$$
and then the proof follows easily by definition of  $\eta$ .
In the standard analysis of FTRL, the following quantities are of special interest as they appear in the final regret guarantees of the algorithm:
- Let  $\mathrm{diam}(R_{\mathcal{T}}) \coloneqq \max_{y, y' \in \mathcal{FP}(\mathcal{T})} |R_{\mathcal{T}}(y) - R_{\mathcal{T}}(y')|$  be the diameter of the regularizer.
- Let  $G_{f}$  be an upper bound on the dual norm of the sub-gradient of the fractional connection cost for any client request, i.e. for any  $R \subseteq L(\mathcal{T})$  and any  $y \in \mathcal{FP}(\mathcal{T})$ , there exists some sub-gradient  $g \in \partial f_{R}(y)$  such that  $\| g\|_{\mathcal{T}}^{*} \leq G_{f}$ . Here,  $\| \cdot \|_{\mathcal{T}}^{*}$  denotes the dual norm of  $\| \cdot \|_{\mathcal{T}}$ .
We begin by presenting the standard analysis of FTRL and deriving an expression for the regret guarantee that depends on the above quantities. Recall that at any step  $t$ , the FTRL objective  $\Phi_t$  doesn't include  $f_{R_t}$  since the client request  $R_t$  is not revealed to the algorithm at the time of decision. We begin by bounding the connection cost of a theoretical algorithm that has access to this information and thus at time  $t$  can pick facility placement  $y^{t+1}$ .
Claim 7. The output of FTRL satisfies
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t + 1}\right) \leq \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \frac {\operatorname {d i a m} \left(R _ {\mathcal {T}}\right)}{\eta}
$$
$$
\begin{array}{l} \Phi_ {t} (y ^ {t}) = \Phi_ {t - 1} (y ^ {t}) + f _ {R _ {t - 1}} (y ^ {t}) \\ \geq \Phi_ {t - 1} (y ^ {t - 1}) + f _ {R _ {t - 1}} (y ^ {t}) \\ \end{array}
$$
where the equality holds by definition of  $\Phi_t$  and the inequality holds from the optimality of  $y^{t-1}$  on  $\Phi_{t-1}$ . Similarly, we obtain
$$
\Phi_ {t - 1} \left(y ^ {t - 1}\right) \geq \Phi_ {t - 2} \left(y ^ {t - 2}\right) + f _ {R _ {t - 2}} \left(y ^ {t - 1}\right)
$$
If we keep applying this rule, we finally get that
$$
\Phi_ {t} (y ^ {t}) \geq \sum_ {s = 1} ^ {t - 1} f _ {R _ {s}} (y ^ {s + 1}) + \Phi_ {1} (y ^ {1})
$$
Furthermore, we have  $\Phi_1(y^1) = R_{\mathcal{T}}(y_1) / \eta$  and  $\Phi_t(y^*) \geq \Phi_t(y^t)$  for all  $t$ . Thus, we get
$$
\Phi_ {T + 1} (y ^ {*}) \geq \sum_ {t = 1} ^ {T} f _ {R _ {t}} (y ^ {t + 1}) + \frac {1}{\eta} \cdot R _ {\mathcal {T}} (y ^ {1})
$$
or equivalently (by substituting  $\Phi_{T + 1}$  's definition) we have
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t + 1}\right) \leq \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \frac {R _ {\mathcal {T}} \left(y ^ {*}\right) - R _ {\mathcal {T}} \left(y ^ {1}\right)}{\eta}
$$
The claim follows from the definition of  $\mathrm{diam}(R_{\mathcal{T}})$ .
Next, we proceed by bounding the increase in the connection cost that we suffer by choosing  $y^{t}$  instead of  $y^{t + 1}$  at time  $t$ .
Claim 8. For any  $t \geq 0$ , it holds that  $f_{R_t}(y^t) \leq f_{R_t}(y^{t+1}) + \eta G_f^2 / \alpha$ .
Proof. For any client request  $R \subseteq L(\mathcal{T})$ , the fractional connection cost function  $f_{R}(y)$  is clearly convex and its sub-gradients are well-defined on  $\mathcal{FP}(\mathcal{T})$ . By definition of  $G_{f}$ , we know that there exists some sub-gradient  $g \in \partial f_{R_t}(y^t)$  such that  $||g||_{\mathcal{T}}^* \leq G_f$ . Using this sub-gradient, we get
$$
\begin{array}{l} f _ {R _ {t}} \left(y ^ {t}\right) \leq f _ {R _ {t}} \left(y ^ {t + 1}\right) + \langle g, y ^ {t} - y ^ {t + 1} \rangle \\ \leq f _ {R _ {t}} \left(y ^ {t + 1}\right) + \left\| g \right\| _ {T} ^ {*} \cdot \left\| y ^ {t} - y ^ {t + 1} \right\| _ {T} \\ \leq f _ {R _ {t}} \left(y ^ {t + 1}\right) + G _ {f} \cdot \left| \left| y ^ {t} - y ^ {t + 1} \right| \right| \tau \\ \end{array}
$$
where the first inequality is derived from the convexity of the fractional connection cost, the second inequality is an application of Holder's inequality and the third inequality is from  $G_{f}$ 's definition.
As we have already argued in section D.2, we know that for any step  $t$ , the FTRL objective  $\Phi_t$  is  $\alpha/\eta$ -strongly convex with respect to  $\|\cdot\|_{\mathcal{T}}$ . Using the definition of strong convexity, this implies that
$$
\Phi_ {t + 1} (y ^ {t}) \geq \Phi_ {t + 1} (y ^ {t + 1}) + \langle g, y ^ {t} - y ^ {t + 1} \rangle + \frac {\alpha}{\eta} \cdot | | y ^ {t} - y ^ {t + 1} | | _ {7} ^ {2}
$$
for any sub-gradient  $g \in \partial \Phi_{t+1}(y^{t+1})$ . Furthermore, since  $y^{t+1}$  is the minimizer of  $\Phi_{t+1}$ , we know from the first order optimality conditions that we can select  $g \in \partial \Phi_{t+1}(y^{t+1})$  such that  $\langle g, y - y^{t+1} \rangle \geq 0$  for any  $y \in \mathcal{FP}(\mathcal{T})$ . Using such a sub-gradient, we get
$$
\begin{array}{l} \left\| y ^ {t} - y ^ {t + 1} \right\| _ {\mathcal {T}} ^ {2} \leq \frac {\eta}{\alpha} \cdot \left(\Phi_ {t + 1} (y ^ {t}) - \Phi_ {t + 1} (y ^ {t + 1})\right) \\ = \frac {\eta}{\alpha} \cdot \left(\Phi_ {t} \left(y ^ {t}\right) + f _ {R _ {t}} \left(y ^ {t}\right) - \Phi_ {t} \left(y ^ {t + 1}\right) - f _ {R _ {t}} \left(y ^ {t + 1}\right)\right) \\ \leq \frac {\eta}{\alpha} \cdot \left(f _ {R _ {t}} \left(y ^ {t}\right) - f _ {R _ {t}} \left(y ^ {t + 1}\right)\right) \\ \end{array}
$$
where we just expanded  $\Phi_{t + 1}$ 's definition and used the fact that  $y^{t}$  is the minimizer of  $\Phi_t$ .
Combining everything, we finally obtain
$$
f _ {R _ {t}} (y ^ {t}) - f _ {R _ {t}} (y ^ {t + 1}) \leq G _ {f} \cdot \sqrt {\frac {\eta}{\alpha} \cdot (f _ {R _ {t}} (y ^ {t}) - f _ {R _ {t}} (y ^ {t + 1}))}
$$
and the claim follows.
We complete the analysis of FTRL by combining Claims 7 and 8 in order to obtain the following regret guarantee:
Claim 9. The output of FTRL satisfies
$$
\sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {t}\right) \leq \sum_ {t = 1} ^ {T} f _ {R _ {t}} \left(y ^ {*}\right) + \frac {\operatorname {d i a m} \left(R _ {\mathcal {T}}\right)}{\eta} + \frac {\eta G _ {f} ^ {2}}{\alpha} \cdot T
$$
It remains to substitute the specific values of the parameters that appear in the regret guarantee. We have already proven in section D.1 that  $R_{\mathcal{T}}$  is  $\alpha$ -strongly convex with respect to  $||\cdot||_{\mathcal{T}}$  for  $\alpha = (8kD\gamma^{2})^{-1}$ . Next, we provide an upper bound for the diameter of the regularizer.
Claim 10. It holds that  $\mathrm{diam}(R_{\mathcal{T}}) \leq knD$ .
Proof. Fix any  $y \in \mathcal{FP}(\mathcal{T})$ . Be definition, we know that  $y_v \leq y_{p(v)}$  and  $\delta_v \leq \delta_{p(v)}$  for any  $v \neq r$ . Thus, the expressions inside the logarithms of the regularizer are always at most 1, which implies that  $R_{\mathcal{T}}(y) \leq 0$ . Furthermore, for any  $\alpha, \beta > 0$  it holds that  $\alpha - \beta \leq \alpha \cdot \ln (\alpha / \beta)$ . Using this inequality, we get that
$$
R _ {\mathcal {T}} (y) \geq \sum_ {v \neq r} w _ {v} \cdot \left(y _ {v} + \delta_ {v} - y _ {p (v)} - \delta_ {p (v)}\right)
$$
Fix any level  $l \in [0, h - 1]$  and let  $V_{l} = \{v \in V(\mathcal{T}) : lev(v) = l\}$  denote the set of vertices of the HST at level  $l$ . Since  $y \in \mathcal{FP}(\mathcal{T})$ , we know that  $\sum_{v \in V_{l}} y_{v} = k$ , and by definition of  $\delta$ 's we know that  $\sum_{v \in V_{l}} \delta_{v} = k$  as well. Furthermore, we know that  $\sum_{v \in V_{l}} y_{p(v)} \leq n \cdot \sum_{v \in V_{l + 1}} y_{v} = n \cdot k$  since any vertex  $v$  can have at most  $n$  (i.e. the total number of leaves) children. Using the same argument,
we have  $\sum_{v\in V_l}\delta_{p(v)}\leq n\cdot \sum_{v\in V_{l + 1}}y_v = n\cdot k$ . Thus, combining everything we obtain
$$
\begin{array}{l} R _ {\mathcal {T}} (y) \geq \sum_ {v \neq r} w _ {v} \cdot \left(y _ {v} + \delta_ {v} - y _ {p (v)} - \delta_ {p (v)}\right) \\ = \sum_ {l = 0} ^ {h - 1} \sum_ {v \in V _ {l}} 2 ^ {l} \cdot \left(y _ {v} + \delta_ {v} - y _ {p (v)} - \delta_ {p (v)}\right) \\ \geq \sum_ {l = 0} ^ {h - 1} 2 ^ {l} \cdot (2 k - 2 k n) \\ = 2 k (1 - n) \left(2 ^ {h} - 1\right) \\ = k (1 - n) D. \\ \end{array}
$$
which proves our claim.
Finally, we only need to find an upper bound for  $G_{f}$ . We begin by computing a set of sub-gradients for the fractional connection cost function.
Claim 11. Fix any client request  $R \subseteq L(\mathcal{T})$  and any  $y \in \mathcal{FP}(\mathcal{T})$ . Define the vector  $g^{R,y} \in \mathbb{R}^{|V(\mathcal{T})|}$  such that
$$
g _ {v} ^ {R, y} = \left\{ \begin{array}{l l} 0 & \text {i f y _ {v} \geq 1} \\ - 2 ^ {l e v (v) + 1} \cdot | T (v) \cap R | & \text {i f y _ {v} <   1} \end{array} \right.
$$
Then,  $g^{R,y} \in \partial f_R(y)$ , i.e.  $g^{R,y}$  is a sub-gradient of  $f_{R}$  on point  $y$ .
Proof. Fix any client request  $R \subseteq L(\mathcal{T})$ . By definition of the fractional connection cost on facility placement  $y \in \mathcal{FP}(\mathcal{T})$ , we have
where  $P(j,r)$  denotes the unique path from leaf  $j \in L(\mathcal{T})$  to the root  $r$ . This is clearly a convex function on  $\mathcal{FP}(\mathcal{T})$  and thus the sub-gradients of  $f_{R}$  are well-defined. Fix any  $v \in V(\mathcal{T})$ . We distinguish between two cases.
- If  $y_v < 1$ , then the partial derivative of  $f_R(y)$  is well-defined and given by
$$
\frac {\partial f _ {R} (y)}{\partial y _ {v}} = - 2 ^ {l e v (v) + 1} \cdot | T (v) \cap R |
$$
where  $T(v)$  is the set of vertices on the sub-tree rooted at vertex  $v$ .
- If  $y_v \geq 1$ , then clearly it doesn't contribute to  $f_R(y)$ . Using standard calculus, it is not hard to argue that in this case there exists a sub-gradient of  $f_R(y)$  whose coordinate corresponding to  $v$  is 0. Thus, we have argued that that  $g^{R,y}$  is a valid sub-gradient of  $f_R$  on point  $y$ .
Finally, we provide an upper bound on the dual-norm of the sub-gradients that we computed on Claim 11.
Claim 12. For any  $y \in \mathcal{FP}(\mathcal{T})$  and any  $R \subseteq L(\mathcal{T})$ , it holds that  $\| g^{R,v} \|_{\mathcal{T}}^* \leq \frac{2n}{\gamma}$ .
Proof. Recall that we have defined the moving cost norm as
$$
\left\| y \right\| _ {\mathcal {T}} = \gamma \cdot \sum_ {v \in V (\mathcal {T})} w _ {v} \cdot y _ {v}
$$
which is basically a weighted  $l_{1}$ -norm with weights  $\gamma \cdot w_v$ . It is well-known that the dual of the  $l_{1}$ -norm is the  $l_{\infty}$  norm. Similarly, the dual of the weighted  $l_{1}$ -norm is a weighted  $l_{\infty}$  norm with inverse weights, i.e.  $||\cdot||^* = l_{\infty}((\gamma w)^{-1})$ . Thus, we have
$$
| | x | | _ {\mathcal {T}} ^ {*} = \max _ {v} \frac {| x _ {v} |}{\gamma \cdot w _ {v}}
$$
Using the calculation of the sub-gradients from Claim 11 and that  $R \subseteq L(\mathcal{T})$  and thus  $|R| \leq n$ , we immediately get the claim.
Claim 10 provides us with an expression for  $\mathrm{diam}(R_{\mathcal{T}})$  and Claim 12 provides us with an expression for  $G_{f}$ . Plugging everything in into Claim 9, we complete the proof of Lemma 3.
D.4 Incorporating approximation errors (Proof of Lemma 4)
where  $\Phi_t(y) = \sum_{s=1}^{t-1} f_{R_s}(y) + \frac{1}{\eta} \cdot R_{\mathcal{T}}(y)$  is the objective that FTRL minimizes over at step  $t$  for  $\eta = (\gamma \cdot \sqrt{nT})^{-1}$ .
Now, assume that instead of minimizing  $\Phi_t(y)$  over  $\mathcal{FP}(\mathcal{T})$  to compute  $y^t$ , we are only able to compute a fractional facility placement  $z^t \in \mathcal{FP}(\mathcal{T})$  such that  $\Phi_t(z^t) \leq \Phi_t(y^t) + \epsilon$  for some  $\epsilon > 0$ .
Claim 13. For any step  $t$ , it holds that
$$
| | z ^ {t} - y ^ {t} | | \tau \leq \sqrt {\epsilon \cdot \frac {\eta}{\alpha}}
$$
Proof. As we have already argued in section D.2, we know that for any step  $t$ , the FTRL objective  $\Phi_t$  is  $\alpha/\eta$ -strongly convex with respect to  $||\cdot||_{\mathcal{T}}$ . Combining this with the first order optimality condition for  $\Phi_t$  on  $y_t$ , we get
$$
\Phi_ {t} (z ^ {t}) \geq \Phi_ {t} (y ^ {t}) + \frac {\alpha}{\eta} \cdot | | z ^ {t} - y ^ {t} | | _ {T} ^ {2}
$$
Using Claim 13, we can easily bound both the connection and the moving cost of the approximated FTRL solutions.
- For the connection cost, recall that the fractional connection cost function  $f_{R_t}$  at step  $t$  is convex, which implies that
$$
f _ {R _ {t}} (z ^ {t}) \leq f _ {R _ {t}} (y ^ {t}) + \langle g, z ^ {t} - y ^ {t} \rangle
$$
for some  $g \in \partial f_{R_t}(z^t)$ . Using Holder's inequality to upper bound the inner-product and using the upper bound of Claim 12 for the dual norm of the sub-gradients of  $f_{R_t}$ , we get that
$$
f _ {R _ {t}} \left(z ^ {t}\right) \leq f _ {R _ {t}} \left(y ^ {t}\right) + \frac {2 n}{\gamma} \cdot \left| \left| z ^ {t} - y ^ {t} \right| \right| \tau
$$
and finally from Claim 13 we get that
$$
f _ {R _ {t}} (z ^ {t}) \leq f _ {R _ {t}} (y ^ {t}) + \frac {2 n}{\gamma} \cdot \sqrt {\epsilon \cdot \frac {\eta}{\alpha}}
$$
- For the moving cost, recall it suffices to use the triangular inequality that  $||\cdot ||_{\mathcal{T}}$  (as a norm) satisfies:
$$
\begin{array}{l} \left| \left| z ^ {t} - z ^ {t - 1} \right| \right| _ {\mathcal {T}} \leq \left| \left| z ^ {t} - y ^ {t} \right| \right| _ {\mathcal {T}} + \left| \left| y ^ {t} - y ^ {t - 1} \right| \right| _ {\mathcal {T}} + \left| \left| y ^ {t - 1} - z ^ {t - 1} \right| \right| _ {\mathcal {T}} \\ \leq | | y ^ {t} - y ^ {t - 1} | | \tau + 2 \cdot \sqrt {\epsilon \cdot \frac {\eta}{\alpha}} \\ \end{array}
$$