SlowGuess/NeurIPS_2023_OCR huggingface dataset:
converges to zero at rates given by the r.h.s of (6) and (7). To gain intuition into what (8) tells us about the resemblance between  $\hat{\mathcal{D}}$  and  $\mathcal{D}$ , it is useful to consider an intermediate dendrogram illustrated in figure 1(b) which conveys the realized values of  $Z_{1},\ldots ,Z_{n}$ . This dendrogram is constructed from  $\mathcal{D}$  by adding a leaf vertex corresponding to each observation  $\mathbf{Y}_i$ , with parent  $Z_{i}$  and height  $p^{-1}\mathbb{E}[\| \mathbf{Y}_i\|^2 |Z_1,\dots,Z_n] = h(Z_i) + p^{-1}\mathrm{tr}[\mathbf{S}(Z_i)^\top \mathbf{S}(Z_i)]$ , and deleting any  $v\in \mathcal{Z}$  such that  $Z_{i}\neq v$  for all  $i\in [n]$  (e.g., vertex  $c$  in figure 1(b)). The resulting merge height between the vertices corresponding to  $\mathbf{Y}_i$  and  $\mathbf{Y}_j$  is  $m(Z_{i},Z_{j})$ . (8) being small implies this must be close to  $\hat{m} (i,j)$  in  $\hat{\mathcal{D}}$  as in figure 1(c). Moreover, in the case  $\hat{\alpha} = \hat{\alpha}_{\mathrm{data}}$ , the height  $\hat{h} (\{i\})$  of leaf vertex  $\{i\}$  in  $\hat{\mathcal{D}}$  is upper bounded by  $p^{-1}\| \mathbf{Y}_i\|^2$ , which under our statistical assumptions is concentrated around  $p^{-1}\mathbb{E}[\| \mathbf{Y}_i\|^2 |Z_1,\dots,Z_n]$ , i.e., the heights of the leaves in figure 1(c) approximate those of the corresponding leaves in figure 1(b).
Overall we see that  $\hat{\mathcal{D}}$  in figure 1(c) approximates the dendrogram in figure 1(b), and in turn  $\mathcal{D}$ . However even if  $m(Z_{i},Z_{j}) = \hat{m} (i,j)$  for all  $i,j$ , the tree output from algorithm 1,  $\hat{\mathcal{T}}$ , may not be isomorphic (i.e., equivalent up to relabelling of vertices) to the tree in figure 1(b);  $\hat{\mathcal{T}}$  is always binary and has  $2n - 1$  vertices, whereas the tree in figure 1(b) may not be binary, depending on the underlying  $\mathcal{T}$  and the realization of  $Z_{1},\ldots ,Z_{n}$ . This reflects the fact that merge distortion, in general, is a pseudometric on dendrograms. However, if one restricts attention to specific classes of true dendrograms  $\mathcal{D}$ , for instance binary trees with non-zero branch lengths, then asymptotically algorithm 1 can recover them exactly. We explain this point further in appendix C.3.
We explore the numerical performance of algorithm 1 in the setting of five data sets summarised below. The real datasets used are open source, and full details of data preparation and sources are given in appendix D.
Simulated data. A simple tree structure with vertices  $\mathcal{V} = \{1,2,3,4,5,6,7,8\}$ , edge set  $\mathcal{E} = \{6\to 1,6\to 2,6\to 3,7\to 4,7\to 5,8\to 6,8\to 7\}$  and  $\mathcal{Z} = \{1,2,3,4,5\}$  (the leaf vertices).  $Z_{1},\ldots ,Z_{n}$  are drawn from the uniform distribution on  $\mathcal{Z}$ . The  $X_{j}(v)$  are Gaussian random variables, independent across  $j$ . Full details of how these variables are sampled are in appendix D. The elements of  $\mathbf{E}_i$  are standard Gaussian, and  $\mathbf{S}(v) = \sigma \mathbf{I}_p$  with  $\sigma = 1$ .
20 Newsgroups. We used a random subsample of  $n = 5000$  documents from the well-known 20 Newsgroups data set [29]. Each data vector corresponds to one document, capturing its  $p = 12818$  Term Frequency Inverse Document Frequency features. The value of  $n$  was chosen to put us in the regime  $p \geq n$ , to which our theory is relevant - see section 3.2. Some ground-truth labelling of documents is known: each document is associated with 1 of 20 newsgroup topics, organized at two hierarchical levels.
Zebrafish gene counts. These data comprise gene counts in zebrafish embryo cells taken from their first day of development [47]. As embryos develop, cells differentiate into various types with specialised, distinct functions, so the data are expected to exhibit tree-like structure mapping these changes. We used a subsample such that  $n = 5079$  and  $p = 5498$  to put us in the  $p \geq n$  regime. Each cell has two labels: the tissue that the cell is from and a subcategory of this.
Amazon reviews. This dataset contains customer reviews on Amazon products [1]. A random sample of  $n = 5000$  is taken and each data vector corresponds to one review with  $p = 5594$  Term Frequency Inverse Document Frequency features. Each product reviewed has labels which make up a three-level hierarchy of product types.
S&P 500 stock returns. The data are  $p = 1259$  daily returns between for  $n = 368$  stocks which were constituents of the S&P 500 market index [2] between 2013 to 2018. The two-level hierarchy of stock sectors by industries and sub-industries follows the Global Industry Classification Standard [3].
4.1 Comparing algorithm 1 to existing methods
We numerically compare algorithm 1 against three very popular variants of agglomerative clustering: UPGMA with Euclidean distance, Ward's method, and UPGMA with cosine distance. These are natural comparators because they work by iteratively merging clusters in a manner similar to algorithm 1, but using different criteria for choosing which clusters to merge. In appendix E we complement our numerical results with mathematical insights into how these methods perform under our modelling assumptions. Numerical results for other linkage functions and distances are given in appendix D. Several popular density-based clustering methods use some hierarchical structure, such as CURE [22], OPTICS [7] and BIRCH [52] but these have limitations which prevent direct comparisons: they aren't equipped with a way to simplify the structure into a tree, which it is our aim to recover, and only suggest extracting a flat partition based on a density threshold. HDBSCAN [9] is a density-based method that doesn't have these limitations, and we report numerical comparisons against it.
Kendall  $\tau_{b}$  ranking correlation. For real data some ground-truth hierarchical labelling may be available but ground-truth merge heights usually are not. We need a performance measure to quantitatively compare methods operating on such data. Commonly used clustering performance measures such as the Rand index [40] and others [23, 21] allow pairwise comparisons between partitions, but do not capture information about hierarchical structure. The cophenetic correlation coefficient [45] is commonly used to compare dendrograms, but relies on an assumption that points close in Euclidean distance should be considered similar which is incompatible with our notion of dot product affinity. To overcome these obstacles we formulate a performance measure as follows. For each of  $n$  data points, we rank the other  $n - 1$  data points according to the order in which they merge with it in the ground-truth hierarchy. We then compare these ground truth rankings to those obtained from a given hierarchical clustering algorithm using the Kendall  $\tau_{b}$  correlation coefficient [26]. This outputs a value in the interval  $[-1, 1]$ , with  $-1, 1$  and  $0$  corresponding to negative, positive and lack of association between the ground-truth and algorithm-derived rankings. We report the mean association value across all  $n$  data points as the overall performance measure. Table 1 shows results with raw data vectors  $\mathbf{Y}_{1:n}$  or PC scores  $\zeta_{1:n}$  taken as input to the various algorithms. For all the data sets except S&P 500, algorithm 1 is found to recover hierarchy more accurately than other methods. We include the results for the S&P 500 data to give a balanced scientific view, and in appendix E we discuss why our modelling assumptions may not be appropriate for these data, thus explaining the limitations of algorithm 1.
4.2 Simulation study of dot product estimation with and without PCA dimension reduction
For high-dimensional data, reducing dimension with PCA prior to clustering may reduce overall computational cost. Assuming  $\zeta_{1:n}$  are obtained from, e.g., a partial SVD, in time  $O(npr)$ , the time complexity of evaluating  $\hat{\alpha}_{\mathrm{PCA}}$  is  $O(npr + n^2 r)$ , versus  $O(n^2 p)$  for  $\hat{\alpha}_{\mathrm{data}}$ , although this ignores the cost of choosing  $r$ . In table 1 we see for algorithm 1, the results for input  $\mathbf{Y}_{1:n}$  are very similar to those for  $\zeta_{1:n}$ . To examine this more closely and connect our findings to theorem 2, we now compare  $\hat{\alpha}_{\mathrm{data}}$  and  $\hat{\alpha}_{\mathrm{PCA}}$  as estimates of  $\alpha$  through simulation. The model is as described at the start of section 4. In figure 2(a)-(b), we see that when  $p$  is growing with  $n$ , and when  $p$  is constant, the  $\hat{\alpha}_{\mathrm{PCA}}$  error is very slightly smaller than the  $\hat{\alpha}_{\mathrm{data}}$  error. By contrast, in figure 2(c), when  $n = 10$  is
Table 1: Kendall  $\tau_{b}$  ranking performance measure. For the dot product method, i.e., algorithm 1,  $\mathbf{Y}_{1:n}$  as input corresponds to using  $\hat{\alpha}_{\mathrm{data}}$ , and  $\zeta_{1:n}$  corresponds to  $\hat{\alpha}_{\mathrm{pca}}$ . The mean Kendall  $\tau_{b}$  correlation coefficient is reported alongside the standard error (numerical value shown is the standard error  $\times 10^{3}$ ).
Data	Input	Dot product	UPGMA w/ cos. dist.	HDBSCAN	UPGMA w/ Eucl. dist.	Ward
Newsgroups	Y1:n	0.26 (2.9)	0.26 (2.9)	-0.010 (0.65)	0.23 (2.7)	0.18 (2.5)
ζ1:n	0.24 (2.6)	0.18 (1.9)	-0.016 (1.9)	0.038 (1.5)	0.19 (2.7)
Zebrafish	Y1:n	0.34 (3.4)	0.25 (3.1)	0.023 (2.9)	0.27 (3.2)	0.30 (3.8)
ζ1:n	0.34 (3.4)	0.27 (3.2)	0.11 (2.8)	0.16 (2.5)	0.29 (3.8)
Reviews	Y1:n	0.15 (2.5)	0.12 (1.9)	0.014 (1.1)	0.070 (1.5)	0.10 (1.8)
ζ1:n	0.14 (2.4)	0.14 (2.4)	-0.0085 (0.78)	0.14 (2.6)	0.12 (2.4)
S&P 500	Y1:n	0.34 (10)	0.34 (10)	0.14 (9.3)	0.34 (1)	0.35 (10)
ζ1:n	0.36 (9.4)	0.42 (11)	0.33 (13)	0.39 (11)	0.39 (11)
Simulated	Y1:n	0.86 (1)	0.81 (2)	0.52 (8)	0.52 (8)	0.52 (8)
ζ1:n	0.86 (1)	0.81 (2)	0.52 (8)	0.52 (8)	0.52 (8)
fixed, we see that the  $\hat{\alpha}_{\mathrm{pca}}$  error is larger than that for  $\hat{\alpha}_{\mathrm{data}}$ . This inferior performance of  $\hat{\alpha}_{\mathrm{pca}}$  for very small and fixed  $n$  is explained by  $n$  appearing in the denominator of the second term in the rate  $O_{\mathbb{P}}(\sqrt{nr / p} +\sqrt{r / n})$  for  $\hat{\alpha}_{\mathrm{pca}}$  in theorem 2 versus  $n$  appearing only in the numerator of  $O_{\mathbb{P}}(n^{2 / q} / \sqrt{p})$  for  $\hat{\alpha}_{\mathrm{data}}$ . Since it is Gaussian, this simulation model has finite exponential-of-quadratic moments, which is a much stronger condition than A4; we conjecture the convergence rate in this Gaussian case is  $O_{\mathbb{P}}(\sqrt{\log n / p})$  for  $\hat{\alpha}_{\mathrm{data}}$ , which would be consistent with figure 2(a). These numerical results seem to suggest the rate for  $\hat{\alpha}_{\mathrm{pca}}$  is similar, thus the second result of theorem 2 may not be sharp.
Figure 2: Simulation study of  $\hat{\alpha}_{\mathrm{data}}$  and  $\hat{\alpha}_{\mathrm{pca}}$  as estimators of  $\alpha$ . All three subplots display the maximum error,  $\max_{i,j\in [n],i\neq j}|\alpha (Z_i,Z_j) - \hat{\alpha} (i,j)|$ , for  $\hat{\alpha} = \hat{\alpha}_{\mathrm{data}}$  (black in all subplots (a)-(c)) and  $\hat{\alpha} = \hat{\alpha}_{\mathrm{pca}}$  (red). Error bars showing the standard deviation from 100 simulations are present for all data points, but in some cases are so small they are barely visible.
4.3 Comparing dot product affinities and Euclidean distances for the 20 Newsgroups data
In this section we expand on the results in table 1 for the 20 Newsgroups data, by exploring how inter-topic and intra-topic dot product affinities and Euclidean distances relate to ground-truth labels. Most existing agglomerative clustering techniques quantify dissimilarity using Euclidean distance. To compare dot products and Euclidean distances, figures 3(a)-(b) show, for each topic, the top five topics with the largest average dot product and smallest average Euclidean distance respectively. We see that clustering of semantically similar topic classes is apparent when using dot products but not when using Euclidean distance. Note that the average dot product affinity between comp.windows.x and itself is not shown in figure 3(b), but is shown in 3(a), by the highest dark blue square in the comp.windows.x column. In appendix D we provide additional numerical results illustrating that with  $n$  fixed, performance in terms of  $\tau_b$  correlation coefficient increases with  $p$ .
(e) Dendrogram of topics using dot products
(f) Dendrogram of topics using UPGMA
Figure 3: Analysis of the 20 Newsgroups data. Marker shapes correspond to newsgroup classes and marker colours correspond to topics within classes. The first/second columns show results for dot products/Euclidean distances respectively. First row: for each topic ( $x$ -axis), the affinity/distance ( $y$ -axis) to the top five best-matching topics, calculated using average linkage of PC scores between documents within topics. Second row: average affinity/distance between documents labelled 'comp.windows.x' and all other topics. Third row: dendrograms output from algorithm 1 and UPGMA applied to cluster topics.
For one topic ('comp.windows.x') the results are expanded in figures 3(c)-(d) to show the average dot products and average Euclidean distances to all other topics. Four out of the five topics with the largest dot product affinity belong to the same 'comp' topic class and other one is a semantically similar 'sci.crypt' topic. Whereas, the other topics in the same 'comp' class are considered dissimilar in terms of Euclidean distance.
In order to display visually compact estimated dendrograms, we applied algorithm 1 and UPGMA in a semi-supervised setting where each topic is assigned its own PC score, taken to be the average of the PC scores of the documents in that topic, and then the algorithms are applied to cluster the topics. The results are shown in figures 3(e)-(f) (for ease of presentation, leaf vertex 'heights' are fixed to be equal).
5 Limitations and opportunities
Our algorithm is motivated by modelling assumptions. If these assumptions are not appropriate for the data at hand, then the algorithm cannot be expected to perform well. A notable limitation of our model is that  $\alpha(u,v) \geq 0$  for all  $u,v \in \mathcal{V}$  (see lemma 3 in appendix C). This is an inappropriate assumption when there are strong negative cross-correlations between some pairs of data vectors, and may explain why our algorithm has inferior performance on the S&P 500 data in table 1. Further discussion is given in appendix E. A criticism of agglomerative clustering algorithms in their basic form is that their computational cost scales faster than  $O(n^{2})$ . Approximations to standard agglomerative methods which improve computational scalability have been proposed [33, 5, 35]. Future research could investigate analogous approximations and speed-up of our method. Fairness in hierarchical clustering has been recently studied in cost function-based settings by [6] and in greedy algorithm settings by [13]. Future work could investigate versions of our algorithm which incorporate fairness measures.
[41] Chandan K Reddy and Bhanukiran Vinzamuri. A survey of partitional and hierarchical clustering algorithms. In Data clustering, pages 87-110. Chapman and Hall/CRC, 2018.
[42] Alessandro Rinaldo, Aarti Singh, Rebecca Nugent, and Larry Wasserman. Stability of density-based clustering. Journal of Machine Learning Research, 13:905, 2012.
[43] Aurko Roy and Sebastian Pokutta. Hierarchical clustering via spreading metrics. Advances in Neural Information Processing Systems, 29, 2016.
[44] R. R. Sokal and C. D. Michener. A statistical method for evaluating systematic relationships. University of Kansas Science Bulletin, 38:1409-1438, 1958.
[45] Robert R Sokal and F James Rohlf. The comparison of dendrograms by objective methods. Taxon, pages 33-40, 1962.
[46] Baris Sumengen, Anand Rajagopalan, Gui Citovsky, David Simcha, Olivier Bachem, Pradipta Mitra, Sam Blasiak, Mason Liang, and Sanjiv Kumar. Scaling hierarchical agglomerative clustering to billion-sized datasets. arXiv preprint arXiv:2105.11653, 2021.
[47] Daniel E Wagner, Caleb Weinreb, Zach M Collins, James A Briggs, Sean G Megason, and Allon M Klein. Single-cell mapping of gene expression landscapes and lineage in the zebrafish embryo. Science, 360(6392):981-987, 2018.
[48] Joe H Ward Jr. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 58(301):236-244, 1963.
[49] Nick Whiteley, Annie Gray, and Patrick Rubin-Delanchy. Discovering latent topology and geometry in data: a law of large dimension. arXiv preprint arXiv:2208.11665, 2022.
[50] F Alexander Wolf, Philipp Angerer, and Fabian J Theis. Scanpy: large-scale single-cell gene expression data analysis. Genome biology, 19:1-5, 2018.
[51] Wang Xuejun, Hu Shuhe, Yang Wenzhi, and Shen Yan. On complete convergence for weighted sums of-mixing random variables. Journal of Inequalities and Applications, 2010:1-13, 2010.
[52] Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustering method for very large databases. ACM sigmoid record, 25(2):103-114, 1996.
A Wasserstein dimension selection
In order to compute the principal component scores  $\zeta_1, \ldots, \zeta_n$ , one must choose the dimension  $r$ . Traditionally and somewhat heuristically, this is done by finding the "elbow" in the scree plot of eigenvalues. The bias-variance tradeoff associated with choosing  $r$  was explored by [49], who suggested a data-splitting method of dimension selection for high-dimensional data using Wasserstein distances. Whilst this method may be more costly than the traditional "elbow" approach, it was empirically demonstrated in [49] to have superior performance.
Algorithm 2 Wasserstein PCA dimension selection [49]
Input: data vectors  $\mathbf{Y}_1,\dots ,\mathbf{Y}_n\in \mathbb{R}^p$    
1: for  $r\in \{1,\ldots ,\min (n,p)\}$  do   
2: Let  $\mathbf{V}\in \mathbb{R}^{p\times r}$  denote the matrix whose columns are orthonormal eigenvectors associated with the  $r$  largest eigenvalues of  $\sum_{i = 1}^{\lceil n / 2\rceil}\mathbf{Y}_i\mathbf{Y}_i^\top$    
3: Orthogonally project  $\mathbf{Y}_1,\dots ,\mathbf{Y}_{\lceil n / 2\rceil}$  onto the column space of  $\mathbf{V},\hat{\mathbf{Y}}_i\coloneqq \mathbf{V}\mathbf{V}^\top \mathbf{Y}_i$    
4: Compute Wasserstein distance  $d_r$  between  $\hat{\mathbf{Y}}_i,\dots ,\hat{\mathbf{Y}}_{\lceil n / 2\rceil}$  and  $\mathbf{Y}_{\lceil n / 2\rceil +1},\dots ,\mathbf{Y}_n$  (as point sets in  $\mathbb{R}^p$    
5: end for   
Output: selected dimension  $\hat{r} =$  argmin  $\{d_r\}$
We note that in practice, the eigenvectors appearing in this procedure could be computed sequentially as  $r$  grows, and to limit computational cost one might consider  $r$  only up to some  $r_{\mathrm{max}} < \min (n,p)$ .
B Implementation using scikit-learn
Algorithm 1 can be implemented using the Python module scikit-learn [38] via their AgglomerativeClustering class, using the standard 'average' linkage criterion and a custom metric function to compute the desired affinities. However, AgglomerativeClustering merges clusters based on minimum distance metric, whereas algorithm 1 merges according to maximum dot product. Therefore, the custom metric function we used in our implementation calculates all the pairwise dot products and subtracts them from the maximum. This transformation needs to then be rectified if accessing the merge heights.
All code released as part of this paper is under the MIT License and can be found at https: //github.com/anniegray52/dot_product_hierarchical
C Proofs and supporting theoretical results
Lemma 2. The height function  $h(v) \coloneqq \alpha(v, v)$  satisfies  $h(v) \geq h(\mathrm{Pa}_v)$  for all  $v \in \mathcal{V}$  except the root.
$$
\begin{array}{l} h (v) = \frac {1}{p} \mathbb {E} [ \| \mathbf {X} (v) - \mathbf {X} (\mathrm {P a} _ {v}) + \mathbf {X} (\mathrm {P a} _ {v}) \| ^ {2} ] \\ = \frac {1}{p} \mathbb {E} [ \| \mathbf {X} (v) - \mathbf {X} (\mathrm {P a} _ {v}) \| ^ {2} ] + 2 \frac {1}{p} \mathbb {E} [ \langle \mathbf {X} (v) - \mathbf {X} (\mathrm {P a} _ {v}), \mathbf {X} (\mathrm {P a} _ {v}) \rangle ] + h (\mathrm {P a} _ {v}) \geq h (\mathrm {P a} _ {v}), \\ \end{array}
$$
where  $\mathbf{A2}$  combined with the tower property and linearity of conditional expectation implies  $\mathbb{E}[\langle \mathbf{X}(v) - \mathbf{X}(\mathrm{Pa}_v),\mathbf{X}(\mathrm{Pa}_v)\rangle ] = 0$
Lemma 3. For all  $u,v\in \mathcal{V},\alpha (u,v)\geq 0$
Proof. If  $u = v$ ,  $\alpha(u, v) \geq 0$  holds immediately from the definition of  $\alpha$  in (2). For  $u \neq v$  with most recent common ancestor  $w$ ,
$$
\begin{array}{l} \mathbb {E} [ \langle \mathbf {X} (u), \mathbf {X} (v) \rangle ] = \sum_ {j = 1} ^ {p} \mathbb {E} \left[ \mathbb {E} [ X _ {j} (u) X _ {j} (v) | X _ {j} (w) ] \right] \\ = \sum_ {j = 1} ^ {p} \mathbb {E} [ \mathbb {E} [ X _ {j} (u) | X _ {j} (w) ] \mathbb {E} [ X _ {j} (v) | X _ {j} (w) ] ] \\ = \sum_ {j = 1} ^ {p} \mathbb {E} [ | X _ {j} (w) | ^ {2} ] \geq 0, \\ \end{array}
$$
where the second equality uses A1 together with standard conditional independence arguments, and the third equality uses A2.
Proof of lemma 1. Let  $w$  be the most recent common ancestor of  $u$  and  $v$ . For each  $j = 1, \dots, p$ , the property A1 together with standard conditional independence arguments imply that  $X_{j}(u)$  and  $X_{j}(v)$  are conditionally independent given  $X_{j}(w)$ , and the property A2 implies that  $\mathbb{E}[X_{j}(u)|X_{j}(w)] = \mathbb{E}[X_{j}(v)|X_{j}(w)] = X_{j}(w)$ . Therefore, by the tower property of conditional expectation,
$$
\begin{array}{l} \mathbb {E} \left[ X _ {j} (u) X _ {j} (v) \right] = \mathbb {E} \left[ \mathbb {E} \left[ X _ {j} (u) X _ {j} (v) \mid X _ {j} (w) \right] \right] \\ = \mathbb {E} \left[ \mathbb {E} \left[ X _ {j} (u) \mid X _ {j} (w) \right] \mathbb {E} \left[ X _ {j} (v) \mid X _ {j} (w) \right] \right] \\ = \mathbb {E} \left[ X _ {j} (w) ^ {2} \right]. \\ \end{array}
$$
Hence, using the definitions of the merge height  $m$ , the height  $h$  and the affinity  $\alpha$ ,
$$
m (u, v) = h (w) = \alpha (w, w) = \frac {1}{p} \sum_ {j = 1} ^ {p} \mathbb {E} [ X _ {j} (w) ^ {2} ] = \frac {1}{p} \sum_ {j = 1} ^ {p} \mathbb {E} [ X _ {j} (u) X _ {j} (v) ] = \alpha (u, v),
$$
which proves the first equality in the statement. The second equality is the definition of  $\alpha$ .
For the third equality in the statement, we have
$$
\begin{array}{l} d (u, v) = h (u) + h (v) - 2 h (w) \\ = \alpha (u, u) + \alpha (v, v) - 2 \alpha (u, v) \\ = \frac {1}{p} \mathbb {E} [ \langle \mathbf {X} (u), \mathbf {X} (u) \rangle ] + \frac {1}{p} \mathbb {E} [ \langle \mathbf {X} (v), \mathbf {X} (v) \rangle ] - 2 \frac {1}{p} \mathbb {E} [ \langle \mathbf {X} (u), \mathbf {X} (v) \rangle ] \\ = \frac {1}{p} \mathbb {E} \left[ \| \mathbf {X} (u) - \mathbf {X} (v) \| ^ {2} \right], \\ \end{array}
$$
where the first equality uses the definition of  $d$ , and the second equality uses the definition of  $h$  and  $h(w) = m(u,v) = \alpha (u,v)$ .
The following lemma establishes an identity concerning the affinities computed in algorithm 1 which will be used in the proof of theorem 1.
Lemma 4. Let  $P_{m}$ ,  $m \geq 0$ , be the sequence of partitions of  $[n]$  constructed in algorithm 1. Then for any  $m \geq 0$ ,
$$
\hat {\alpha} (u, v) = \frac {1}{| u | | v |} \sum_ {i \in u, j \in v} \hat {\alpha} (i, j), \quad f o r a l l d i s t i n c t p a i r s u, v \in P _ {m}. \tag {9}
$$
Proof. The proof is by induction on  $m$ . With  $m = 0$ , (9) holds immediately since  $P_0 = \{\{1\}, \ldots, \{n\}\}$ . Now suppose (9) holds at step  $m$ . Then for any distinct pair  $w, w' \in P_{m+1}$ ,
either  $w$  or  $w'$  is the result of merging two elements of  $P_{m}$ , or  $w$  and  $w'$  are both elements of  $P_{m}$ . In the latter case the induction hypothesis immediately implies:
$$
\hat {\alpha} (w, w ^ {\prime}) = \frac {1}{| w | | w ^ {\prime} |} \sum_ {i \in w, j \in w ^ {\prime}} \hat {\alpha} (i, j).
$$