SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Peter Macgregor
School of Informatics
peter.macgregor@ed.ac.uk
Spectral clustering is a popular and effective algorithm designed to find  $k$  clusters in a graph  $G$ . In the classical spectral clustering algorithm, the vertices of  $G$  are embedded into  $\mathbb{R}^k$  using  $k$  eigenvectors of the graph Laplacian matrix. However, computing this embedding is computationally expensive and dominates the running time of the algorithm. In this paper, we present a simple spectral clustering algorithm based on a vertex embedding with  $O(\log (k))$  vectors computed by the power method. The vertex embedding is computed in nearly-linear time with respect to the size of the graph, and the algorithm provably recovers the ground truth clusters under natural assumptions on the input graph. We evaluate the new algorithm on several synthetic and real-world datasets, finding that it is significantly faster than alternative clustering algorithms, while producing results with approximately the same clustering accuracy.
Graph clustering is an important problem with numerous applications in machine learning, data science, and theoretical computer science. Spectral clustering is a popular graph clustering algorithm with strong theoretical guarantees and excellent empirical performance. Given a graph  $G$  with  $n$  vertices and  $k$  clusters, the classical spectral clustering algorithm consists of the following two high-level steps [26, 33].
1. Embed the vertices of  $G$  into  $\mathbb{R}^k$  according to  $k$  eigenvectors of the graph Laplacian matrix.
2. Apply a  $k$ -means clustering algorithm to partition the vertices into  $k$  clusters.
Recent work shows that if the graph has a well-defined cluster structure, then the clusters are well-separated in the spectral embedding and the  $k$ -means algorithm will return clusters which are close to optimal [21, 28].
The main downside of this algorithm is the high computational cost of computing  $k$  eigenvectors of the graph Laplacian matrix. In this paper, we address this computational bottleneck and propose a new fast spectral clustering algorithm which avoids the need to compute eigenvectors while maintaining excellent theoretical guarantees. Moreover, our proposed algorithm is simple, fast, and effective in practice.
1.1 Sketch of Our Approach
In this section, we introduce the high-level idea of this paper, which is also illustrated in Figure 1. We begin by considering a recent result of Makarychev et al. [23] who show that a random projection of data into  $O(\log (k))$  dimensions preserves the  $k$ -means objective function for all partitions of the
Figure 1: An illustration of the steps of the spectral clustering algorithm, and the contribution of this paper. We are given a graph as input. In classical spectral clustering we follow the top path: we compute the spectral embedding and apply a  $k$ -means algorithm to find clusters. Through the recent result of Makarychev et al. [23], we can project the embedded points into  $O(\log (k))$  dimensions and obtain approximately the same clustering. In this paper, we show that it is possible to compute the low-dimensional embedding directly with the power method, skipping the computationally expensive step of computing  $k$  eigenvectors.
data, with high probability. Since the final step of spectral clustering is to apply  $k$ -means, we might consider the following alternative spectral clustering algorithm which will produce roughly the same output as the classical algorithm.
1. Embed the vertices of  $G$  into  $\mathbb{R}^k$  according to  $k$  eigenvectors of the graph Laplacian matrix.
2. Randomly project the embedded points into  $O(\log (k))$  dimensions.
3. Apply a  $k$ -means clustering algorithm to partition the vertices into  $k$  clusters.
Of course, this does not avoid the expensive eigenvector computation and so it is not immediately clear that this random projection can be used to improve the spectral clustering algorithm.
The key technical element of our paper is a proof that we can efficiently approximate a random projection of the spectral embedding without computing the spectral embedding itself. For this, we use the power method, which is a well-known technique in numerical linear algebra for approximating the dominant eigenvalue of a matrix [11]. We propose the following simple algorithm (formally described in Algorithm 2).
1. Embed the vertices of  $G$  into  $O(\log (k))$  dimensions using  $O(\log (k))$  random vectors computed with the power method.
2. Apply a  $k$ -means clustering algorithm to partition the vertices into  $k$  clusters.
We prove that the projection obtained using the power method is approximately equivalent to a random projection of the spectral embedding. Then, by carefully applying the techniques developed by Makarychev et al. [23] and Macgregor and Sun [21], we obtain a theoretical bound on the number of vertices misclassified by our proposed algorithm. Moreover, the time complexity of step 1 is nearly linear in the size of the graph, and the algorithm is fast in practice. The formal theoretical guarantee is given in Theorem 3.1.
This paper is closely related to a sequence of recent results which prove an upper bound on the number of vertices misclassified by the classical spectral clustering algorithm [15, 21, 24, 28]. While we will directly compare our result with these in a later section, our proposed algorithm has a much faster running time than the classical spectral clustering algorithm, and has similar theoretical guarantees.
Boutsidis et al. [5] also study spectral clustering using the power method. Our result improves on theirs in two respects. Firstly, our algorithm is faster since we compute  $O(\log (k))$  vectors rather than  $k$  vectors and their algorithm includes an additional singular value decomposition step. Secondly, we give a theoretical upper bound on the total number of vertices misclassified by our algorithm.
Makarychev et al. [23] generalise the well-known Johnson-Lindenstrauss lemma [13] to show that random projections of data into  $O(\log (k))$  dimensions preserve the  $k$ -means objective, and we make use of their result in our analysis.
Macgregor and Sun [21] show that for graphs with certain structures of clusters, spectral clustering with fewer than  $k$  eigenvectors performs better than using  $k$  eigenvectors. In this paper, we present the first proof that embedding with  $O(\log (k))$  vectors is sufficient to find  $k$  clusters with spectral clustering.
Other proposed methods for fast spectral clustering include the Nystrom method [6] and using a 'pre-clustering' step to reduce the number of data points to be clustered [35]. These methods lack rigorous theoretical guarantees on the accuracy of the returned clustering. Moreover, our proposed algorithm is significantly simpler to implement that the alternative methods.
Let  $G = (V, E, w)$  be a graph with  $n = |V|$  and  $m = |E|$ . For any  $v \in V$ , the degree of  $v$  is given by  $d(v) = \sum_{u \neq v} w(u, v)$ . For any  $S \subset V$ , the volume of  $S$  is given by  $\operatorname{vol}(S) = \sum_{u \in S} d(u)$ . The Laplacian matrix of  $G$  is  $\mathbf{L} = \mathbf{D} - \mathbf{A}$  where  $\mathbf{D}$  is the diagonal matrix with  $\mathbf{D}(i, i) = d(i)$  and  $\mathbf{A}$  is the adjacency matrix of  $G$ . The normalised Laplacian is given by  $\mathbf{N} = \mathbf{D}^{-\frac{1}{2}} \mathbf{L} \mathbf{D}^{-\frac{1}{2}}$ . We always use  $\lambda_1 \leq \lambda_2 \leq \ldots \leq \lambda_n$  to be the eigenvalues of  $\mathbf{N}$  and the corresponding eigenvectors are  $f_1, \ldots, f_n$ . For any graph, it holds that  $\lambda_1 = 0$  and  $\lambda_n \leq 2$  [7]. We will also use the signless Laplacian matrix  $\mathbf{M} = \mathbf{I} - (1/2) \mathbf{N}$  and will let  $\gamma_1 \geq \ldots \geq \gamma_n$  be the eigenvalues of  $\mathbf{M}$ . Notice that by the definition of  $\mathbf{M}$ , we have  $\gamma_i = 1 - (1/2) \lambda_i$  and the eigenvectors of  $\mathbf{M}$  are also  $f_1, \ldots, f_n$ . For an integer  $k$ , we let  $[k] = \{1, \ldots, k\}$  be the set of all positive integers less than or equal to  $k$ . Given two sets  $A$  and  $B$ , their symmetric difference is given by  $A \triangle B = (A \setminus B) \cup (B \setminus A)$ . We call  $\{S_i\}_{i=1}^k$  a  $k$ -way partition of  $V$  if  $S_i \cap S_j = \emptyset$  for  $i \neq j$  and  $\bigcup_{i=1}^k S_i = V$ .
Throughout the paper, we use big-O notation to hide constants. For example, we use  $l = O(n)$  to mean that there exists a universal constant  $c$  such that  $l \leq cn$ . We sometimes use  $\widetilde{O}(n)$  in place of  $O(n\log^c (n))$  for some constant  $c$ . Following [21], we say that a partition  $\{S_i\}_{i=1}^k$  of  $V$  is almost-balanced if  $\mathrm{vol}(S_i) = \Theta(\mathrm{vol}(V)/k)$  for all  $i \in [k]$ .
2.1 Conductance and the Graph Spectrum
Given a graph  $G = (V, E)$ , and a cluster  $S \subset V$ , the conductance of  $S$  is given by
$$
\Phi (S) \triangleq \frac {w (S , \overline {{S}})}{\min \{\mathrm {v o l} (S) , \mathrm {v o l} (\overline {{S}}) \}}
$$
where  $w(S, \overline{S}) = \sum_{u \in S} \sum_{v \in \overline{S}} w(u, v)$ . Then, the  $k$ -way expansion of  $G$  is defined to be
$$
\rho (k)\triangleq \min_{\text{partition} C_{1},\ldots C_{k}}\max_{i}\Phi (C_{i}).
$$
Notice that  $\rho(k)$  is small if and only if  $G$  can be partitioned into  $k$  clusters of low conductance. There is a close connection between the  $k$ -way expansion of  $G$  and the eigenvalues of the graph Laplacian matrix, as shown in the following higher-order Cheeger inequality.
The signless Laplacian is usually defined to be  $2 \cdot \mathbf{I} - \mathbf{N}$ . We divide this by 2 so that the eigenvalues of  $\mathbf{M}$  lie between 0 and 1.
Lemma 2.1 (Higher-Order Cheeger Inequality, [19]). For a graph  $G$ , let  $\lambda_1 \leq \ldots \leq \lambda_n$  be the eigenvalues of the normalised Laplacian matrix. Then, for any  $k$ ,
$$
\frac {\lambda_ {k}}{2} \leq \rho (k) \leq O \big (k ^ {3} \big) \sqrt {\lambda_ {k}}.
$$
From this lemma, we can see that an upper bound on  $\rho(k)$  and a lower bound on  $\lambda_{k+1}$  are sufficient conditions to guarantee that  $G$  can be partitioned into  $k$  clusters of low conductance, and cannot be partitioned into  $k+1$  clusters. This condition is commonly used in the analysis of graph clustering [15, 21, 24, 28].
2.2 The  $k$ -means Objective
For any matrix  $\mathbf{B} \in \mathbb{R}^{n \times d}$ , the  $k$ -means cost of a  $k$ -way partition  $\{A_i\}_{i=1}^k$  of the data points is given by
$$
\mathrm {C O S T} _ {\mathbf {B}} (A _ {1}, \ldots , A _ {k}) \triangleq \sum_ {i = 1} ^ {k} \sum_ {u \in A _ {i}} \| \mathbf {B} (u,:) - \mu_ {i} \| _ {2} ^ {2},
$$
where  $\mathbf{B}(u, :)$  is the  $u$ th row of  $\mathbf{B}$  and  $\mu_{i} = (1 / |A_{i}|) \sum_{u \in A_{i}} \mathbf{B}(u, :)$  is the mean of the points in  $A_{i}$ . Although optimising the  $k$ -means objective is NP-hard, there is a polynomial-time constant factor approximation algorithm [14], and an approximation scheme which is polynomial in  $n$  and  $d$  with an exponential dependency on  $k$  [8, 16]. Lloyds algorithm [20], with  $k$ -means++ initialisation [3] is an  $O(\log(n))$ -approximation algorithm which is fast and effective in practice.
2.3 The Power Method
The power method is an algorithm which is most often used to approximate the dominant eigenvalue of a matrix [11, 25]. Given some matrix  $\mathbf{M} \in \mathbb{R}^{n \times n}$ , a vector  $\boldsymbol{x}_0 \in \mathbb{R}^n$ , and a positive integer  $t$ , the power method computes the value of  $\mathbf{M}^t \boldsymbol{x}_0$  by repeated multiplication by  $\mathbf{M}$ . The formal algorithm is given in Algorithm 1.
Algorithm 1: POWERMETHOD  $(\mathbf{M}\in \mathbb{R}^{n\times n},\pmb{x}_0\in \mathbb{R}^n,t\in \mathbb{Z}_{\geq 0})$
1 for  $i\in \{1,\dots ,t\}$  do   
2  $\begin{array}{rl}{\pmb{x}_i} & {= \mathbf{M}\pmb{x}_{i - 1}} \end{array}$    
3 end   
4 return  $x_{t}$
3 Algorithm Description and Analysis
In this section, we present our newly proposed algorithm and sketch the proof of our main result. Omitted proofs can be found in the Appendix. We first prove that if  $\mathbf{M}$  has  $k$  eigenvalues  $\gamma_{1},\ldots ,\gamma_{k}$  close to 1, then the power method can be used to compute a random vector in the span of the eigenvectors corresponding to  $\gamma_{1},\dots,\gamma_{k}$ .
We then apply this result to the signless Laplacian matrix of a graph to develop a fast spectral clustering algorithm and we bound the number of misclassified vertices when the algorithm is applied to a well-clustered graph.
3.1 Approximating a Random Vector with the Power Method
Suppose we are given some matrix  $\mathbf{M} \in \mathbb{R}^{n \times n}$  with eigenvalues  $1 \geq \gamma_1 \geq \dots \geq \gamma_n \geq 0$  and corresponding eigenvectors  $\pmb{f}_1, \dots, \pmb{f}_n$ . Typically, the power method is used to approximate the dominant eigenvector,  $\pmb{f}_1$ . In this section, we show that when  $\gamma_k$  is sufficiently close to 1, the power method can be used to compute a random vector in the space spanned by  $\pmb{f}_1, \dots, \pmb{f}_k$ .
Let  $\pmb{x}_0 \in \mathbb{R}^n$  be a random vector chosen according to the  $n$ -dimensional Gaussian distribution  $\mathrm{N}(\mathbf{0},\mathbf{I})$ . We can write  $\pmb{x}_0$  as a linear combination of the eigenvectors:
$$
\boldsymbol {x} _ {0} = \sum_ {i = 1} ^ {n} a _ {i} \boldsymbol {f} _ {i},
$$
where  $a_{i} = \langle \pmb{x}_{0},\pmb{f}_{i}\rangle$ . Then, the vector  $\pmb{x}_t$  computed by POWERMETHOD  $(\mathbf{M},\pmb{x}_0,t)$  can be written as
$$
\boldsymbol {x} _ {t} = \sum_ {i = 1} ^ {n} a _ {i} \gamma_ {i} ^ {t} \boldsymbol {f} _ {i}.
$$
Informally, if  $\gamma_{k} \geq 1 - c_{1}$  and  $\gamma_{k + 1} \leq 1 - c_{2}$  for sufficiently small  $c_{1}$  and sufficiently large  $c_{2}$ , then for a carefully chosen value of  $t$  we have  $\gamma_{i}^{t} \approx 1$  for  $i \leq k$  and  $\gamma_{i}^{t} \approx 0$  for  $i \geq k + 1$ . This implies that
$$
\boldsymbol {x} _ {t} \approx \sum_ {i = 1} ^ {k} a _ {i} \boldsymbol {f} _ {i} = \left(\sum_ {i = 1} ^ {k} \boldsymbol {f} _ {i} \boldsymbol {f} _ {i} ^ {\intercal}\right) \boldsymbol {x} _ {0},
$$
and we observe that  $\left(\sum_{i=1}^{k} \boldsymbol{f}_{i} \boldsymbol{f}_{i}^{\top}\right) \boldsymbol{x}_{0}$  is a random vector distributed according to a  $k$ -dimensional Gaussian distribution in the space spanned by  $\boldsymbol{f}_{1}, \ldots, \boldsymbol{f}_{k}$ . We make this intuition formal in Lemma 3.1 and specify the required conditions on  $\gamma_{k}, \gamma_{k+1}$  and  $t$ .
Lemma 3.1. Let  $\mathbf{M} \in \mathbb{R}^{n \times n}$  be a matrix with eigenvalues  $1 \geq \gamma_1 \geq \ldots \geq \gamma_n \geq 0$  and corresponding eigenvectors  $\pmb{f}_1, \ldots, \pmb{f}_n$ . Let  $\pmb{x}_0 \in \mathbb{R}^n$  be drawn from the  $n$ -dimensional Gaussian distribution  $N(\mathbf{0}, \mathbf{I})$ . Let  $\pmb{x}_t = \text{POWERMETHOD}(\mathbf{M}, \pmb{x}_0, t)$  for  $t = \Theta(\log(n / \epsilon^2 k))$ . If  $\gamma_k \geq 1 - O\left(\epsilon \cdot \log(n / \epsilon^2 k)^{-1}\right)$  and  $\gamma_{k+1} \leq 1 - \Omega(1)$ , then with probability at least  $1 - 1/(10k)$ ,
$$
\left\| \boldsymbol {x} _ {t} - \mathbf {P} \boldsymbol {x} _ {0} \right\| _ {2} \leq \epsilon \sqrt {k},
$$
where  $\mathbf{P} = \sum_{i=1}^{k} \mathbf{f}_{i} \mathbf{f}_{i}^{\top}$  is the projection onto the space spanned by the first  $k$  eigenvectors of  $\mathbf{M}$ .
3.2 The Fast Spectral Clustering Algorithm
We now introduce the fast spectral clustering algorithm. The algorithm follows the pattern of the classical spectral clustering algorithm, with an important difference: rather than embedding the vertices according to  $k$  eigenvectors of the graph Laplacian, we embed the vertices with  $\Theta\left(\log(k) \cdot \epsilon^{-2}\right)$  random vectors computed with the power method for the signless graph Laplacian  $\mathbf{M}$ . Algorithm 2 formally specifies the algorithm, and the theoretical guarantees are given in Theorem 3.1.
Theorem 3.1. Let  $G$  be a graph with  $\lambda_{k+1} = \Omega(1)$  and  $\rho(k) = O\left(\epsilon \cdot \log(n/\epsilon)^{-1}\right)$ . Additionally, let  $\{S_i\}_{i=1}^k$  be the  $k$ -way partition corresponding to  $\rho(k)$  and suppose that  $\{S_i\}_{i=1}^k$  are almost balanced. Let  $\{A_i\}_{i=1}^k$  be the output of Algorithm 2. With probability at least  $0.9 - \epsilon$ , there exists a permutation  $\sigma : [k] \to [k]$  such that
$$
\sum_ {i = 1} ^ {k} \operatorname {v o l} \left(A _ {i} \triangle S _ {\sigma (i)}\right) = O \left(\epsilon \cdot \operatorname {v o l} \left(V _ {G}\right)\right).
$$
Moreover, the running time of Algorithm 2 is
$$
\widetilde {O} \big (m \cdot \epsilon^ {- 2} \big) + T _ {\mathrm {K M}} (n, k, l),
$$
where  $m$  is the number of edges in  $G$  and  $T_{\mathrm{KM}}(n,k,l)$  is the running time of the  $k$ -means approximation algorithm on  $n$  points in  $l$  dimensions.
Remark 3.1. The assumptions on  $\lambda_{k + 1}$  and  $\rho (k)$  in Theorem 3.1 imply that the graph  $G$  can be partitioned into exactly  $k$  clusters of low conductance. This is related to previous results which make an assumption on the ratio  $\lambda_{k + 1} / \rho (k)$  [21, 28, 30]. Macgregor and Sun [21] prove a guarantee like Theorem 3.1 under the assumption that  $\lambda_{k + 1} / \rho (k) = \Omega (1)$ . We achieve a faster algorithm under a slightly stronger assumption.
Remark 3.2. The running time of Theorem 3.1 improves on previous spectral clustering algorithms. Boutsidis et al. [4] describe an algorithm with running time  $\widetilde{O}\big(m\cdot k\cdot \epsilon^{-2}\big) + O\big(k^2\cdot n\big) + T_{\mathrm{KM}}(n,k,k)$ . Moreover, their analysis does not provide any guarantee on the number of misclassified vertices.
Algorithm 2: FASTSPECTRALCLUSTER  $(G = (V,E),k\in \mathbb{Z}_{\geq 0},\epsilon \in [0,1])$
1  $\mathbf{M}\gets \mathbf{I} - (1 / 2)\cdot \mathbf{N}_G$    
2  $l\gets \Theta (\log (k)\cdot \epsilon^{-2})$    
3  $t\gets \Theta (\log (n / \epsilon^2 k))$    
4 for  $i\in \{1,\ldots ,l\}$  do   
5 Let  $x_{i}\in \mathbb{R}^{n}$  be a random vector from Gaussian distribution N(0,I)   
6  $\pmb {y}_i\gets \mathrm{POWERMATHOD}(\mathbf{M},\pmb {x}_i,t)$    
7 end   
8  $\mathbf{Y}\gets [\pmb {y}_1;\dots ;\pmb {y}_l]$    
9  $A_{1},\ldots ,A_{k}\gets \mathrm{KMEANS}(\mathbf{D}_{G}^{-1 / 2}\mathbf{Y},k)$    
10 return  $A_{1},\ldots ,A_{k}$
Remark 3.3. The constants in the definition of  $l$  and  $t$  in Algorithm 2 are based on those in the analysis of Makarychev et al. [23] and this paper. In practice, we find that setting  $l = \log (k)$  and  $t = 10\log (n / k)$  works well.
Throughout the remainder of this section, we will sketch the proof of Theorem 3.1. We assume that  $G = (V,E)$  is a graph with  $k$  clusters  $\{S_i\}_{i=1}^k$  of almost balanced size,  $\lambda_{k+1} = \Omega(1)$ , and  $\rho(k) = O\big(\epsilon \cdot \log(n/\epsilon)^{-1}\big)$ .
In order to understand the behaviour of the  $k$ -means algorithm on the computed vectors, we will analyse the  $k$ -means cost of a given partition under three different embeddings of the vertices. Let  $\pmb{f}_1,\dots ,\pmb{f}_k$  be the eigenvectors of  $\mathbf{M}$  corresponding to the eigenvalues  $\gamma_{1},\ldots ,\gamma_{k}$  and let  $\pmb{y}_1,\dots ,\pmb{y}_l$  be the vectors computed in Algorithm 2. We will also consider the vectors  $\pmb{z}_1,\dots ,\pmb{z}_l$  given by  $\pmb{z}_i = \mathbf{P}\pmb{x}_i$ , where  $\{\pmb{x}_i\}_{i = 1}^k$  are the random vectors sampled in Algorithm 2, and  $\mathbf{P} = \sum_{i = 1}^{k}\pmb {f}_i\pmb{f}_i^{\mathrm{T}}$  is the projection onto the space spanned by  $\pmb{f}_1,\dots ,\pmb{f}_k$ . Notice that each  $\pmb{z}_i$  is a random vector distributed according to the  $k$ -dimensional Gaussian distribution. Furthermore, let
$$
\mathbf {F} = \left[ \begin{array}{c c c} | & & | \\ \boldsymbol {f} _ {1} & \ldots & \boldsymbol {f} _ {k} \\ | & & | \end{array} \right], \quad \mathbf {Y} = \left[ \begin{array}{c c c} | & & | \\ \boldsymbol {y} _ {1} & \ldots & \boldsymbol {y} _ {l} \\ | & & | \end{array} \right] \quad \text {a n d} \quad \mathbf {Z} = \left[ \begin{array}{c c c} | & & | \\ \boldsymbol {z} _ {1} & \ldots & \boldsymbol {z} _ {l} \\ | & & | \end{array} \right].
$$
We will consider the vertex embeddings given by  $\mathbf{D}^{-1/2}\mathbf{F}$ ,  $\mathbf{D}^{-1/2}\mathbf{Z}$  and  $\mathbf{D}^{-1/2}\mathbf{Y}$  and show that the  $k$ -means objective for every  $k$ -way partition is approximately equal in each of them. We will use the following result shown by Makarychev et al. [23].
Lemma 3.2 ([23], Theorem 1.3). Given data  $\mathbf{X} \in \mathbb{R}^{n \times k}$ , let  $\Pi \in \mathbb{R}^{k \times l}$  be a random matrix with each column sampled from the  $k$ -dimensional Gaussian distribution  $\mathrm{N}(\mathbf{0}, \mathbf{I}_k)$  and
$$
l = O \left(\frac {\log (k) + \log (1 / \epsilon)}{\epsilon^ {2}}\right).
$$
Then, with probability at least  $1 - \epsilon$ , it holds for all partitions  $\{A_{i}\}_{i = 1}^{k}$  of  $[n]$  that
$$
\operatorname {C O S T} _ {\mathbf {X}} \left(A _ {1}, \dots , A _ {k}\right) \in (1 \pm \epsilon) \operatorname {C O S T} _ {\mathbf {X} \Pi} \left(A _ {1}, \dots , A _ {k}\right).
$$
Applying this lemma with  $\mathbf{X} = \mathbf{D}^{-\frac{1}{2}}\mathbf{F}$  and  $\boldsymbol{\Pi} = \mathbf{F}^{\intercal}\mathbf{Z}$  shows that the  $k$ -means cost is approximately equal in the embeddings given by  $\mathbf{D}^{-\frac{1}{2}}\mathbf{F}$  and  $\mathbf{D}^{-\frac{1}{2}}\mathbf{Z}$ , since  $\mathbf{F}\mathbf{F}^{\intercal}\mathbf{Z} = \mathbf{Z}$  and each of the entries of  $\mathbf{F}^{\intercal}\mathbf{Z}$  is distributed according to the Gaussian distribution  $\mathrm{N}(0,1)$ . By Lemma 3.1, we can also show that the  $k$ -means objective in  $\mathbf{D}^{-\frac{1}{2}}\mathbf{Y}$  is within an additive error of  $\mathbf{D}^{-\frac{1}{2}}\mathbf{Z}$ . This allows us to prove the following lemma.
Lemma 3.3. With probability at least  $0.9 - \epsilon$ , for any partitioning  $\{A_i\}_{i=1}^k$  of the vertex set  $V$ , we have
$$
\operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {Y}} (A _ {1}, \dots , A _ {k}) \geq (1 - \epsilon) \operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {F}} (A _ {1}, \dots , A _ {k}) - \epsilon k
$$
$$
\operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {Y}} (A _ {1}, \dots , A _ {k}) \leq (1 + \epsilon) \operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {F}} (A _ {1}, \dots , A _ {k}) + \epsilon k.
$$
There is some interesting subtlety in this argument. If we ignore the  $\mathbf{D}^{-\frac{1}{2}}$  matrix, then  $\mathbf{F}$  is the data matrix, and  $\mathbf{F}^{\intercal}\mathbf{Z}$  represents the random projection. After projecting the data, we are left with the projection matrix  $\mathbf{Z}$  itself. This happens only because the data matrix is the orthonormal basis  $\mathbf{F}$  which is also used to project  $\mathbf{Z}$ .
To complete the proof of Theorem 3.1, we will make use of the following results proved by Macgregor and Sun [21].
Lemma 3.4 ([21], Lemma 4.1). There exists a partition  $\{A_i\}_{i=1}^k$  of the vertex set  $V$  such that
$$
\operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {F}} \left(A _ {1}, \dots A _ {k}\right) <   k \cdot \rho (k) / \lambda_ {k + 1}.
$$
Lemma 3.5 ([21], Theorem 2). Given some partition of the vertices,  $\{A_i\}_{i=1}^k$ , such that
$$
\operatorname {C O S T} _ {\mathbf {D} ^ {- 1 / 2} \mathbf {F}} (A _ {1}, \dots A _ {k}) \leq c \cdot k,
$$
then there exists a permutation  $\sigma : [k] \to [k]$  such that
$$
\sum_ {i = 1} ^ {k} \operatorname {v o l} \left(A _ {i} \triangle S _ {\sigma (i)}\right) = O \left(c \cdot \operatorname {v o l} (V)\right).
$$
Proof of Theorem 3.1. By Lemma 3.4 and Lemma 3.3, with probability at least  $0.9 - \epsilon$ , there exists some partition  $\{\widehat{A}_i\}_{i=1}^k$  of the vertex set  $V_G$  such that