SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 5: FIND-LEAF  $(T,C_1)$
Input:  $T,C_1$    
Initialization:  $C_0 = \emptyset$    
for each node  $v$  in  $C_1$  do while  $v$  is not a leaf node do  $\begin{array}{rl} & v\gets \arg_{w}\max \{N_{w},w\in ch(v)\} ,\mathrm{where~}ch(v)\mathrm{~denotes~the~children~nodes~of~}v\\ & \mathrm{Add~}v\mathrm{~to~}C_{0} \end{array}$    
Output: Initial center set  $C_0\subseteq U$
equivalently that the edge weight of the  $i$ -th level is  $2^{i - 1}$ . The crucial step of our analysis is to examine the approximation error in terms of the 2-HST metric, after which the error can be adapted to the general metrics by the following qwll-known result.
Lemma 3.4 (Bartal (1996)). In a metric space  $(U,\rho)$  with  $|U| = n$  and diameter  $\triangle$ , it holds that  $\forall x,y\in U$ ,  $E[\rho^T (x,y)] = O(\min \{\log n,\log \triangle \})\rho (x,y)$ . In the Euclidean space  $\mathbb{R}^d$ ,  $E[\rho^T (x,y)] = O(d)\rho (x,y)$ ,  $\forall x,y\in U$ .
Recall  $C_0, C_1$  from Algorithm 4. We define
$$
\operatorname {c o s t} _ {k} ^ {T} (U) = \sum_ {y \in U} \min  _ {x \in C _ {0}} \rho^ {T} (x, y), \tag {3}
$$
$$
\operatorname {c o s t} _ {k} ^ {T ^ {\prime}} (U, C _ {1}) = \min  _ {| F \cap T (v) | = 1, \atop \forall v \in C _ {1}} \sum_ {y \in U} \min  _ {x \in F} \rho^ {T} (x, y), \tag {4}
$$
$$
O P T _ {k} ^ {T} (U) = \min  _ {F \subset U, | F | = k} \sum_ {y \in U} \min  _ {x \in F} \rho^ {T} (x, y) \equiv \min  _ {C _ {1} ^ {\prime}} c o s t _ {k} ^ {T \prime} (U, C _ {1} ^ {\prime}). \tag {5}
$$
For simplicity, we will use  $cost_k^{T'}(U)$  to denote  $cost_k^{T'}(U, C_1)$ . Here,  $OPT_k^T$  (5) is the cost of the global optimal solution with the 2-HST metric. The last equivalence in (5) holds because the optimal centers can always be located in  $k$  disjoint subtrees, as each leaf only contains one point. (3) is the  $k$ -median cost with 2-HST metric of the output  $C_0$  of Algorithm 4. (4) is the optimal cost after the subtrees are chosen. That is, it represents the minimal cost to pick one center from each subtree in  $C_1$ . We first bound the error of the subtree search step and the leaf search step, respectively.
Lemma 3.5 (Subtree search).  $cost_{k}^{T^{\prime}}(U)\leq 5OPT_{k}^{T}(U)$
Lemma 3.6 (Leaf search).  $\text{cost}_k^T(U) \leq 2\text{cost}_k^{T'}(U)$ .
Combining Lemma 3.5 and Lemma 3.6, we obtain:
Theorem 3.7 (2-HST error). Running Algorithm 4, we have  $\text{cost}_k^T(U) \leq 10OPT_k^T(U)$ .
Thus, HST-initialization produces an  $O(1)$  approximation to the optimal cost in the 2-HST metric. Define  $cost_{k}(U)$  as (1) for our HST centers, and the optimal cost w.r.t.  $\rho$  as
$$
O P T _ {k} (U) = \min  _ {| F | = k} \sum_ {y \in U} \min  _ {x \in F} \rho (x, y). \tag {6}
$$
We have the following result based on Lemma 3.4.
Theorem 3.8. In the general metric space, the expected  $k$ -median cost of NDP-HST (Algorithm 4) is  $E[\mathrm{cost}_k(U)] = O(\min\{\log n, \log \Delta\}) OPT_k(U)$ .
Remark 3.9. In the Euclidean space, Makarychev et al. (2019) showed that using  $O(\log k)$  random projections suffices for  $k$ -median to achieve  $O(1)$  error. Thus, if  $\log \triangle = O(\log d)$ , by Lemma 3.4, HST initialization is able to achieve  $O(\log (\min \{d,k\}))$  error, which is better than  $O(\log k)$  of  $k$ -median  $++$  (Arthur and Vassilvitskii, 2007) when  $d$  is small.
NDP-HST Local Search. We are interested in the approximation quality of standard local search (Algorithm 1), when the initial centers are produced by our NDP-HST.
Theorem 3.10. When initialized by NDP-HST, local search achieves  $O(1)$  approximation error in expected  $O(k\log \log \min \{n,\triangle \})$  number of iterations for input in general metric space.
We remark that the initial centers found by NDP-HST can be used for  $k$ -means clustering analogously. For general metrics,  $E[\text{cost}_{km}(U)] = O(\min\{\log n, \log \Delta\})^2 OPT_{km}(U)$  where  $\text{cost}_{km}(U)$  is the optimal  $k$ -means cost. See Appendix C for more detaileds.
4 HST Initialization with Differential Privacy
In this section, we consider initialization and clustering with differential privacy (DP). Recall (2) that in this problem,  $U$  is the universe of data points, and  $D \subset U$  is a demand set that needs to be clustered with privacy. Since  $U$  is public, simply running initialization algorithms on  $U$  would
Algorithm 6: DP-HST initialization
Input:  $U, D, \Delta, k, \epsilon$   
Build a level- $L$  2-HST  $T$  based on input  $U$   
for each node  $v$  in  $T$  do  
 $\mid N_v \leftarrow |D \cap T(v)|$ ,  $\hat{N}_v \leftarrow N_v + \text{Lap}(2^{(L - h_v)} / \epsilon)$ ,  $\text{score}(v) \leftarrow \hat{N}(v) \cdot 2^{h_v}$   
Based on  $\hat{N}_v$ , apply the same strategy as Algorithm 4: find  $C_1$ ;  $C_0 = \text{FIND-LEAF}(C_1)$   
Output: Private initial center set  $C_0 \subseteq U$
Algorithm 7: DP-HST local search
Input:  $U$  demand points  $D\subseteq U$  , parameter  $k,\epsilon ,T$    
Initialization:  $F_{1}$  the private initial centers generated by Algorithm 6 with privacy  $\epsilon /2$    
Set parameter  $\epsilon^{\prime} = \frac{\epsilon}{4\Delta(T + 1)}$    
for  $i = 1$  to  $T$  do Select  $(x,y)\in F_i\times (V\setminus F_i)$  with prob. proportional to  $\exp (-\epsilon^{\prime}\times (cost(F_i - \{x\} +\{y\}))$  Let  $F_{i + 1}\gets F_i - \{x\} +\{y\}$    
Select  $j$  from  $\{1,2,\dots,T + 1\}$  with probability proportional to  $\exp (-\epsilon^{\prime}\times cost(F_j))$    
Output:  $F = F_{j}$  the private center set
preserve the privacy of  $D$ . However, 1) this might be too expensive; 2) in many cases one would probably want to incorporate some information about  $D$  in the initialization, since  $D$  could be a very imbalanced subset of  $U$ . For example,  $D$  may only contain data points from one cluster, out of tens of clusters in  $U$ . In this case, initialization on  $U$  is likely to pick initial centers in multiple clusters, which would not be helpful for clustering on  $D$ .
Next, we show how our proposed HST initialization can be easily combined with differential privacy and at the same time contains useful information about the demand set  $D$ , leading to improved approximation error (Theorem 4.3). Again, suppose  $T$  is an  $L = \log \triangle$ -level 2-HST of universe  $U$  in a general metric space. Denote  $N_v = |T(v) \cap D|$  for a node point  $v$ . Our private HST initialization (DP-HST) is similar to the non-private Algorithm 4. To gain privacy, we perturb  $N_v$  by adding i.i.d. Laplace noise:  $\hat{N}_v = N_v + \text{Lap}(2^{(L - h_v)} / \epsilon)$ , where  $\text{Lap}(2^{(L - h_v)} / \epsilon)$  is a Laplace random number with rate  $2^{(L - h_v)} / \epsilon$ . We will use the perturbed  $\hat{N}_v$  for node sampling instead of the true value  $N_v$ , as described in Algorithm 6. The DP guarantee of this initialization scheme is straightforward by the composition theory (Dwork, 2006).
Theorem 4.1. Algorithm 6 is  $\epsilon$ -differentially private.
Proof. For each level  $i$ , the subtrees  $T(v,i)$  are disjoint to each other. The privacy budget used in  $i$ -th level is  $\epsilon / 2^{(L - i)}$ , so by composition the total privacy budget is  $\sum_{i}\epsilon / 2^{(L - i)} < \epsilon$ .
Theorem 4.2. Algorithm 6 finds initial centers such that
$$
E \left[ \operatorname {c o s t} _ {k} (D) \right] = O (\log n) \left(O P T _ {k} (D) + k \epsilon^ {- 1} \triangle \log n\right).
$$
DP-HST Local Search. Similarly, we can use private HST initialization to improve the performance of private  $k$ -median local search, which is presented in Algorithm 7. After DP-HST initialization, the DP local search procedure follows Gupta et al. (2010) using the exponential mechanism.
Theorem 4.3. Algorithm 7 achieves  $\epsilon$ -differential privacy. The output centers achieve  $\mathrm{cost}_k(D) \leq 6OPT_k(D) + O(\epsilon^{-1}k^2\triangle (\log \log n)\log n)$  in  $O(k\log \log n)$  iterations with probability  $(1 - \frac{1}{poly(n)})$ .
In prior literature, the DP local search with random initialization (Gupta et al., 2010) has 6 multiplicative error and  $O(\epsilon^{-1} \triangle k^2 \log^2 n)$  additive error. Our result improves the log  $n$  term to  $\log \log n$  in the additive error. Meanwhile, the number of iterations needed is improved from  $T = O(k \log n)$  to  $O(k \log \log n)$  (see Appendix A for an empirical justification). Notably, it has been shown in Gupta et al. (2010) that for  $k$ -median problem, the lower bounds on the multiplicative and additive error of any  $\epsilon$ -DP algorithm are  $O(1)$  and  $O(\epsilon^{-1} \triangle k \log (n / k))$ , respectively. Our result matches the lower bound on the multiplicative error, and the additive error is only worse than the bound by a factor of  $O(k \log \log n)$ . To our knowledge, Theorem 4.3 is the first result in the literature to improve the error of DP local search in general metric space.
5.1 Datasets and Algorithms
Discrete Euclidean space. Following previous work, we test  $k$ -median clustering on the MNIST hand-written digit dataset (LeCun et al., 1998) with 10 natural clusters (digit 0 to 9). We set  $U$  as 10000 randomly chosen data points. We choose the demand set  $D$  using two strategies: 1) "balance", where we randomly choose 500 samples from  $U$ ; 2) "imbalance", where  $D$  contains 500 random samples from  $U$  only from digit "0" and "8" (two clusters). We note that, the imbalanced  $D$  is a very practical setting in real-world scenarios, where data are typically not uniformly distributed. On this dataset, we test clustering with both  $l_{1}$  and  $l_{2}$  distance as the underlying metric.
Metric space induced by graph. Random graphs have been widely considered in testing  $k$ -median methods (Balcan et al., 2013; Todo et al., 2019). Our construction of graphs follows a similar approach as the synthetic pmedinfo graphs provided by the popular OR-Library (Beasley, 1990). The metric  $\rho$  for this experiment is the (weighted) shortest path distance. To generate a size- $n$  graph, we first randomly split the nodes into 10 clusters. Within each cluster, each pair of nodes is connected with probability 0.2, and with weight drawn from uniform [0, 1]. For every pair of clusters, we randomly connect some nodes from each cluster, with weights following uniform  $[0.5, r]$ . A larger  $r$  makes the graph more separable, i.e., clusters are farther from each other (see Appendix A for example graphs). For this task,  $U$  has 3000 nodes, and the private set  $D$  (500 nodes) is chosen using the similar "balanced" and "imbalanced" approaches as described above. In the imbalanced case, we choose the demand set  $D$  randomly from only two clusters.
Algorithms. We compare the following clustering algorithms in both non-DP and DP setting: (1) NDP-rand: Local search with random initialization; (2) NDP-kmedian++. Local search with  $k$ -median++. initialization (Algorithm 2); (3) NDP-HST: Local search with NDP-HST initialization (Algorithm 4), as described in Section 3; (4) DP-rand: Standard DP local search algorithm (Gupta et al., 2010), which is Algorithm 7 with initial centers randomly chosen from  $U$ ; (5) DP-kmedian++. DP local search with  $k$ -median++. initialization run on  $U$ ; (6) DP-HST: DP local search with HST-initialization (Algorithm 7). For non-DP tasks, we set  $L = 6$ . For DP clustering, we use  $L = 8$ .
For non-DP methods, we set  $\alpha = 10^{-3}$  in Algorithm 1 and the maximum number of iterations as 20. To examine the quality of initialization as well as the final centers, We report both the cost at initialization and the cost of the final output. For DP methods, we run the algorithms for  $T = 20$  steps and report the results with  $\epsilon = 1$  (comparisons/results with other  $T$  and  $\epsilon$  are similar). We test  $k\in \{2,5,10,15,20\}$ . The average cost over  $T$  iterations is reported for robustness. All the results are averaged over 10 independent repetitions.
The results on MNIST and graph data are given in Figure 2. Here we present the  $l_{2}$ -clustering on MNIST, and the simulated graph with  $r = 1$  (clusters are less separable). The comparisons are similar for both  $l_{1}$  metric on MNIST and  $r = 100$  graph (see Figure 4 in Appendix A):
- From the left column, the initial centers found by HST has lower cost than  $k$ -median++ and random initialization, for both non-DP and DP setting, and for both balanced and imbalanced demand set  $D$ . This confirms that the proposed HST initialization is more powerful than  $k$ -median++ in finding good initial centers.
- From the right column, we also observe lower final cost of HST followed by local search in DP clustering. In the non-DP case, the final cost curves overlap, which means that despite HST offers better initial centers, local search can always find a good solution eventually.
- The advantage of DP-HST, in terms of both the initial and the final cost, is more significant when  $D$  is an imbalanced subset of  $U$ . As mentioned before, this is because our DP-HST initialization approach also privately incorporates the information of  $D$ .
To sum up, the proposed HST initialization scheme could perform better with various metrics and data patterns, in both non-private and private setting—in all cases, HST finds better initial centers with smaller cost than  $k$ -median++. HST considerably outperforms  $k$ -median++. in the private and imbalanced  $D$  setting, for MNIST with both  $l_{2}$  and  $l_{1}$  metric, and for graph with both  $r = 100$  (highly separable) and  $r = 1$  (less separable).
Figure 2:  $k$ -median cost on the MNIST ( $l_2$ -metric) and graph dataset ( $r = 1$ ). 1st column: initial cost. 2nd column: final output cost.
We develop a new initialization framework for the  $k$ -median problem in the general metric space. Our approach, called HST initialization, is built upon the HST structure from metric embedding theory. We propose a novel and efficient tree search approach which provably improves the approximation error of the  $k$ -median++ method, and has lower complexity (higher efficiency) than  $k$ -median++ when  $k$  is not too small, which is a common practice. Moreover, we propose differentially private (DP) HST initialization algorithm, which adapts to the private demand point set, leading to better clustering performance. When combined with subsequent DP local search heuristic, our algorithm is able to improve the additive error of DP local search, which is close to the theoretical lower bound within a small factor. Experiments with Euclidean metrics and graph metrics verify the effectiveness of our methods, which improve the cost of both the initial centers and the final  $k$ -median output.
$k$ -Median Clustering via Metric Embedding: Towards Better Initialization with Differential Privacy (Supplementary Material)
A More Details on Experiments
A.1 Examples of Graph Data
In Figure 3, we plot two example graphs (subgraphs of 50 nodes) with  $r = 100$  and  $r = 1$ . When  $r = 100$ , the graph is highly separable (i.e., clusters are far from each other). When  $r = 1$ , the clusters are harder to be distinguished from each other.
Figure 3: Example of synthetic graphs: subgraph of 50 nodes. Upper  $r = 1$ . Bottom:  $r = 100$ . Darker and thicker edged have smaller distance. When  $r = 100$ , the graph is more separable.
Figure 4:  $k$ -median cost on the MNIST ( $l_1$ -metric) and graph dataset ( $r = 100$ ) 1st column: initial cost. 2nd column: final output cost.
A.2 More Experiments
A.3 Improved Iteration Cost of DP-HST
In Theorem 4.3, we show that under differential privacy constraints, the proposed DP-HST (Algorithm 7) improves both the approximation error and the number of iterations required to find a good solution of classical DP local search (Gupta et al., 2010). In this section, we provide some numerical results to justify the theory.
First, we need to properly measure the iteration cost of DP local search. This is because, unlike the non-private clustering, the  $k$ -median cost after each iteration in DP local search is not decreasing monotonically, due to the probabilistic exponential mechanism. To this end, for the cost sequence with length  $T = 20$ , we compute its moving average sequence with window size 5. Attaining the
minimal value of the moving average indicates that the algorithm has found a "local optimum", i.e., it has reached a "neighborhood" of solutions with small clustering cost. Thus, we use the number of iterations to reach such local optimum as the measure of iteration cost. The results are provided in Figure 5. We see that on all the tasks (MNIST with  $l_{1}$  and  $l_{2}$  distance, and graph dataset with  $r = 1$  and  $r = 100$ ), DP-HST has significantly smaller iterations cost. In Figure 6, we further report the  $k$ -median cost of the best solution in  $T$  iterations found by each DP algorithm. We see that DP-HST again provide the smallest cost. This additional set of experiments again validates the claims of Theorem 4.3, that DP-HST is able to find better initial centers in fewer iterations.
Figure 5: Iteration cost to reach a locally optimal solution, on MNIST and graph datasets with different  $k$ . The demand set is an imbalanced subset of the universe.
Figure 6: The  $k$ -median cost of the best solution found by each differentially private algorithm. The demand set is an imbalanced subset of the universe. Same comparison holds on graph data.
The following composition result of differential privacy will be used in our proof.
Theorem B.1 (Composition Theorem (Dwork, 2006)). If Algorithms  $\mathbb{A}_1,\mathbb{A}_2,\dots,\mathbb{A}_m$  are  $\epsilon_1,\epsilon_2,\ldots ,\epsilon_m$  differentially private respectively, then the union  $(\mathbb{A}_1(D),\mathbb{A}_2(D),\ldots ,\mathbb{A}_m(D))$  is  $\sum_{i = 1}^{m}\epsilon_{i}$  -DP.
B.1 Proof of Lemma 3.5
Proof. Consider the intermediate output of Algorithm 4,  $C_1 = \{v_1, v_2, \dots, v_k\}$ , which is the set of roots of the minimal subtrees each containing exactly one output center  $C_0$ . Suppose one of the optimal "root set" that minimizes (4) is  $C_1^* = \{v_1', v_2', \dots, v_k'\}$ . If  $C_1 = C_1^*$ , the proof is done. Thus, we prove the case for  $C_1 \neq C_1^*$ . Note that  $T(v), v \in C_1$  are disjoint subtrees. We have the following reasoning.
- Case 1: for some  $i, j'$ ,  $v_i$  is a descendant node of  $v_j'$ . Since the optimal center point  $f^*$  is a leaf node by the definition of (4), we know that there must exist one child node of  $v_j'$  that expands a subtree which contains  $f^*$ . Therefore, we can always replace  $v_j'$  by one of its child nodes. Hence, we can assume that  $v_i$  is not a descendant of  $v_j'$ .
Note that, we have  $score(v_j') \leq score(v_i)$  if  $v_j' \notin C_1^* \cap C_1$ . Algorithm 4 sorts all the nodes based on cost value, and it would have more priority to pick  $v_j'$  than  $v_i$  if  $score(v_j') > score(v_i)$  and  $v_i$  is not a child node of  $v_j'$ .
- Case 2: for some  $i, j'$ ,  $v_j'$  is a descendant of  $v_i$ . In this case, optimal center point  $f^*$ , which is a leaf of  $T(v_i)$ , must also be a leaf node of  $T(v_j')$ . We can simply replace  $C_1$  with the swap  $C_1 \setminus \{v_i\} + \{v_j'\}$  which does not change  $\text{cost}_k^{T'}(U)$ . Hence, we can assume that  $v_j'$  is not a descendant of  $v_i$ .
- Case 3: Otherwise. By the construction of  $C_1$ , we know that  $score(v_j') \leq \min\{score(v_i), i = 1, \dots, k\}$  when  $v_j' \in C_1^* \setminus C_1$ . Consider the swap between  $C_1$  and  $C_1^*$ . By the definition of tree distance, we have  $OPT_k^T(U) \geq \sum_{v_i \in C_1 \setminus C_1^*} N_{v_i} 2^{h_{v_i}}$ , since  $\{T(v_i), v_i \in C_1 \setminus C_1^*\}$  does not contain any center of the optimal solution determined by  $C_1^*$  (which is also the optimal "root set" for  $OPT_k^T(U)$ ).
Thus, we only need to consider Case 3. Let us consider the optimal clustering with center set be  $C^* = \{c_1^*, c_2^*, \dots, c_k^*\}$  (each center  $c_j^*$  is a leaf of subtree whose root be  $c_j'$ ), and  $S_j'$  be the leaves assigned to  $c_j^*$ . Let  $S_j$  denote the set of leaves in  $S_j'$  whose distance to  $c_j^*$  is strictly smaller than its distance to any centers in  $C_1$ . Let  $P_j$  denote the union of paths between leaves of  $S_j$  to its closest center in  $C_1$ . Let  $v_j''$  be the nodes in  $P_j$  with highest level satisfying  $T(v_j'') \cap C_1 = \emptyset$ . The score of  $v_j''$  is  $2^{h_{v_j''}} N(v_j'')$ . That means the swap with a center  $v_j'$  into  $C_1$  can only reduce  $4 \cdot 2^{h_{v_j''}} N(v_j'')$  to  $cost_k^{T'}(U)$  (the tree distance between any leaf in  $S_j$  and its closest center in  $C_1$  is at most  $4 \cdot 2^{h_{v_j''}}$ ). We just use  $v_j'$  to represent  $v_j''$  for later part of this proof for simplicity. By our reasoning, summing all the swaps over  $C_1^* \setminus C_1$  gives
$$
\operatorname{cost}_{k}^{T^{\prime}}(U) - \operatorname{OPT}_{k}^{T}(U)\leq 4\sum_{v^{\prime}_{j}\in C_{1}^{*}\setminus C_{1}}N_{v^{\prime}_{j}}2^{h_{v^{\prime}_{j}}},
$$
$$
O P T _ {k} ^ {T} (U) \geq \sum_ {v _ {i} \in C _ {1} \backslash C _ {1} ^ {*}} N _ {v _ {i}} 2 ^ {h _ {v _ {i}}}.
$$
Also, based on our discussion on Case 1, it holds that
$$
N _ {v _ {j} ^ {\prime}} 2 ^ {h _ {v _ {j} ^ {\prime}}} - N _ {v _ {i}} 2 ^ {h _ {v _ {i}}} \leq 0.
$$
Summing them together, we have  $\text{cost}_k^{T'}(U) \leq 5OPT_k^T(U)$ .
B.2 Proof of Lemma 3.6
Proof. Since the subtrees in  $C_1$  are disjoint, it suffices to consider one subtree with root  $v$ . With a little abuse of notation, let  $cost_1^{T'}(v, U)$  denote the optimal  $k$ -median cost within the point set  $T(v)$  with one center in 2-HST:
$$
\operatorname {c o s t} _ {1} ^ {T ^ {\prime}} (v, U) = \min  _ {x \in T (v)} \sum_ {y \in T (v)} \rho^ {T} (x, y), \tag {7}
$$
which is the optimal cost within the subtree. Suppose  $v$  has more than one children  $u, w, \ldots$ , otherwise the optimal center is clear. Suppose the optimal solution of  $cost_1^{T'}(v, U)$  chooses a leaf node in  $T(u)$ , and our HST initialization algorithm picks a leaf of  $T(w)$ . If  $u = w$ , then HST chooses the optimal one where the argument holds trivially. Thus, we consider  $u \neq w$ . We have the following two observations:
- Since one needs to pick a leaf of  $T(u)$  to minimize  $cost_1^{T'}(v, U)$ , we have  $cost_1^{T'}(v, U) \geq \sum_{x \in ch(v), x \neq u} N_x \cdot 2^{h_x}$  where  $ch(u)$  denotes the children nodes of  $u$ .
- By our greedy strategy,  $\text{cost}_1^T(v, U) \leq \sum_{x \in \text{ch}(u)} N_x \cdot 2^{h_x} \leq \text{cost}_1^{T'}(v, U) + N_u \cdot 2^{h_u}$ .
Since  $h_u = h_w$ , we have
$$
2 ^ {h _ {u}} \cdot (N _ {u} - N _ {w}) \leq 0,
$$
since our algorithm picks subtree roots with highest scores. Then we have  $cost_1^T(v, U) \leq cost_1^{T'}(v, U) + N_w \cdot 2^{h_w} \leq 2cost_1^{T'}(v, U)$ . Since the subtrees in  $C_1$  are disjoint, the union of centers for  $OPT_1^T(v, U)$ ,  $v \in C_1$  forms the optimal centers with size  $k$ . Note that, for any data point  $p \in U \setminus C_1$ , the tree distance  $\rho^T(p, f)$  for  $\forall f$  that is a leaf node of  $T(v)$ ,  $v \in C_1$  is the same. That is, the choice of leaf in  $T(v)$  as the center does not affect the  $k$ -median cost under 2-HST metric. Therefore, union bound over  $k$  subtree costs completes the proof.
B.3 Proof of Proposition 3.2
Proof. It is known that the 2-HST can be constructed in  $O(dn\log n)$  (Bartal, 1996). The subtree search in Algorithm 4 involves at most sorting all the nodes in the HST based on the score, which takes  $O(n\log n)$ . We use a priority queue to store the nodes in  $C_1$ . When we insert a new node  $v$  into queue, its parent node (if existing in the queue) would be removed from the queue. The number of nodes is  $O(n)$  and each operation (insertion, deletion) in a priority queue based on score has  $O(\log n)$  complexity. Lastly, the total time to obtain  $C_0$  is  $O(n)$ , as the FIND-LEAF only requires a top down scan in  $k$  disjoint subtrees of  $T$ . Summing parts together proves the claim.
B.4 Proof of Theorem 4.2