SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \widehat {D ^ {(u)}} = \tau_ {1} ^ {2} d i a g \left(\left[ 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 2 \frac {\tau_ {c}}{\tau_ {1}}, 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 2 \frac {\tau_ {c}}{\tau_ {1}}, 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 2 \frac {\tau_ {c}}{\tau_ {1}}, 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 2 \frac {\tau_ {\mathrm {c}}}{\tau_ {1}}, 4, 4 \right]\right). \\ \widehat {D ^ {(u) - \frac {1}{2}}} = \frac {1}{\tau_ {1}} d i a g \left(\left[ 1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {\tau_ {c}}{\tau_ {1}}, 1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {\tau_ {c}}{\tau_ {1}}, 1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {\tau_ {c}}{\tau_ {1}}, 1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {\tau_ {c}}{\tau_ {1}}, 2, 2 \right]\right). \\ \widehat {D ^ {(u) - \frac {1}{2}}} \widehat {\eta_ {u} A ^ {(u)}} \widehat {D ^ {(u) - \frac {1}{2}}} = \left[ \begin{array}{c c c c c c} 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 & 0 \\ 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 0 & 0 \\ 0 & 2 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 0 & 0 & 0 & 0 & \frac {1}{2} & \frac {1}{2} \\ 0 & 0 & 0 & 0 & \frac {1}{2} & \frac {1}{2} \end{array} \right]. \\ \end{array}
$$
Let  $\hat{\lambda}_1^{(u)},\ldots ,\hat{\lambda}_6^{(u)}$  be six eigenvalue of  $\widehat{D^{(u) - \frac{1}{2}}\eta_uA^{(u)}}\widehat{D^{(u) - \frac{1}{2}}}$ , and  $\hat{v}_1^{(u)},\ldots ,\hat{v}_6^{(u)}$  be corresponding eigenvectors. By direct calculation we have
$$
\hat {\lambda} _ {1} ^ {(u)} = 1, \hat {\lambda} _ {2} ^ {(u)} = 1, \hat {\lambda} _ {3} ^ {(u)} = 1 - 4 \frac {\tau_ {s}}{\tau_ {1}}, \hat {\lambda} _ {4} ^ {(u)} = 1 - 4 \frac {\tau_ {c}}{\tau_ {1}}, \hat {\lambda} _ {5} ^ {(u)} = 1 - 4 \frac {\tau_ {s}}{\tau_ {1}} - 4 \frac {\tau_ {c}}{\tau_ {1}}, \hat {\lambda} _ {6} ^ {(u)} = 0
$$
and corresponding eigenvector as
$$
\begin{array}{l} \hat {v} _ {1} ^ {(u)} = [ 0, 0, 0, 0, 1, 1 ], \\ \hat {v} _ {2} ^ {(u)} = [ 1, 1, 1, 1, 0, 0 ], \\ \hat {v} _ {3} ^ {(u)} = [ 1, - 1, 1, - 1, 0, 0 ], \\ \hat {v} _ {4} ^ {(u)} = [ 1, 1, - 1, - 1, 0, 0 ], \\ \hat {v} _ {5} ^ {(u)} = [ 1, - 1, - 1, 1, 0, 0 ], \\ \hat {v} _ {6} ^ {(u)} = [ 0, 0, 0, 0, 1, - 1 ]. \\ \end{array}
$$
Then, by Weyl's Theorem, for  $i \in \{1, 2, 3\}$ , we have
$$
\left| \lambda_ {i} ^ {(u)} - \hat {\lambda} _ {i} ^ {(u)} \right| \leq \left\| D ^ {(u) - \frac {1}{2}} \eta_ {u} A ^ {(u)} D ^ {(u) - \frac {1}{2}} - \widehat {D ^ {(u) - \frac {1}{2}}} \widehat {\eta_ {u} A ^ {(u)}} \widehat {D ^ {(u) - \frac {1}{2}}} \right\| _ {2} \leq O \left(\left(\frac {\tau_ {c}}{\tau_ {1}}\right) ^ {2}\right).
$$
$$
\| \sin (U ^ {(u)}, \hat {U} ^ {(u)}) \| _ {F} \leq \frac {O ((\frac {\tau_ {c}}{\tau_ {1}}) ^ {2})}{4 (\frac {\tau_ {c}}{\tau_ {1}} - \frac {\tau_ {s}}{\tau_ {1}})} \leq O (\frac {\tau_ {c} ^ {2}}{\tau_ {1} (\tau_ {c} - \tau_ {s})}).
$$
C Technical Details for Main Theory
C.1 Notation
We let  $\mathbf{1}_n$ ,  $\mathbf{0}_n$  be the  $n$ -dimensional vector with all 1 or 0 values respectively.  $\mathbf{1}_{m\times n}$ ,  $\mathbf{0}_{m\times n}$  are similarly defined for  $m$ -by- $n$  matrix.  $I_{n}$  is the identity matrix with shape  $n\times n$ . For any matrix  $V$ ,  $V_{(i,j)}$  indicts the value at  $i$ -th row and  $j$ -th column of  $V$ . If the matrix is subscripted like  $V_{k}$ , we use a comma in-between like  $V_{k,(i,j)}$ . Similarly,  $\mathbf{v}_{(i)}$  and  $\mathbf{v}_{k,(i)}$  are the  $i$ -th value for vector  $\mathbf{v}$  and  $\mathbf{v}_k$  respectively.  $[n]$  is used to abbreviate the set  $\{1,2,\dots,n\}$ .
C.2 Matrix Form of K-means and the Derivative
Recall that we defined the K-means clustering measure of features in Sec. 4:
$$
\mathcal {M} _ {k m s} (\Pi , Z) = \sum_ {\pi \in \Pi} \sum_ {i \in \pi} \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| ^ {2} / \sum_ {\pi \in \Pi} | \pi | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\Pi} \| ^ {2}, \tag {12}
$$
where the numerator measures the intra-class distance:
$$
\mathcal {M} _ {\text {i n t r a}} (\Pi , Z) = \sum_ {\pi \in \Pi} \sum_ {i \in \pi} \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| ^ {2}, \tag {13}
$$
and the denominator measures the inter-class distance:
$$
\mathcal {M} _ {\text {i n t e r}} (\Pi , Z) = \sum_ {\pi \in \Pi} | \pi | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\Pi} \| ^ {2}. \tag {14}
$$
We will show next how to convert the intra-class and the inter-class measure into a matrix form, which is desirable for analysis.
Intra-class measure. Note that the  $K$ -means intra-class measure can be rewritten in a matrix form:
$$
\mathcal {M} _ {i n t r a} (\Pi , Z) = \| Z - H _ {\Pi} Z \| _ {F} ^ {2},
$$
where  $H_{\Pi}$  is a matrix to convert  $Z$  to mean vectors w.r.t clusters defined by  $\Pi$ . Without losing the generality, we assume  $Z$  is ordered according to the partition in  $\Pi$  — first  $|\pi_1|$  vectors are in  $\pi_1$ , next  $|\pi_2|$  vectors are in  $\pi_2$ , etc. Then  $H_{\Pi}$  is given by:
$$
H _ {\Pi} = \left[ \begin{array}{c c c c} \frac {1}{| \pi_ {1} |} \mathbf {1} _ {| \pi_ {1} | \times | \pi_ {1} |} & \mathbf {0} & \ldots & \mathbf {0} \\ \mathbf {0} & \frac {1}{| \pi_ {2} |} \mathbf {1} _ {| \pi_ {2} | \times | \pi_ {2} |} & \ldots & \mathbf {0} \\ \ldots & \ldots & \ldots & \ldots \\ \mathbf {0} & \mathbf {0} & \ldots & \frac {1}{| \pi_ {k} |} \mathbf {1} _ {| \pi_ {k} | \times | \pi_ {k} |} \end{array} \right].
$$
Going further, we have:
$$
\begin{array}{l} \mathcal {M} _ {i n t r a} (\Pi , Z) = \| Z - H _ {\Pi} Z \| _ {F} ^ {2} \\ = \operatorname {T r} \left(\left(I - H _ {\Pi}\right) ^ {2} Z Z ^ {\top}\right) \\ = \mathrm {T r} ((I - 2 H _ {\Pi} + H _ {\Pi} ^ {2}) Z Z ^ {\top}) \\ = \operatorname {T r} \left(\left(I - H _ {\mathrm {I I}}\right) Z Z ^ {\top}\right). \\ \end{array}
$$
Inter-class measure. The inter-class measure can be equivalently given by:
$$
\mathcal {M} _ {i n t e r} (\Pi , Z) = \left\| H _ {\Pi} Z - \frac {1}{N} \mathbf {1} _ {N \times N} Z \right\| _ {F} ^ {2},
$$
where  $H_{\Pi}$  is defined as above. And we can also derive:
$$
\begin{array}{l} \mathcal {M} _ {i n t e r} (\Pi , Z) = \| H _ {\Pi} Z - \frac {1}{N} \mathbf {1} _ {N \times N} Z \| _ {F} ^ {2} \\ = \operatorname {T r} \left(\left(H _ {\Pi} - \frac {1}{N} \mathbf {1} _ {N \times N}\right) ^ {2} Z Z ^ {\top}\right) \\ = \operatorname {T r} \left(\left(H _ {\Pi} ^ {2} - \frac {2}{N} H _ {\Pi} \mathbf {1} _ {N \times N} + \frac {1}{N ^ {2}} \mathbf {1} _ {N \times N} ^ {2}\right) Z Z ^ {\top}\right) \\ = \operatorname {T r} \left(\left(H _ {\Pi} - \frac {1}{N} \mathbf {1} _ {N \times N}\right) Z Z ^ {\top}\right). \\ \end{array}
$$
C.3 K-means Measure Has the Same Order as K-means Error
Theorem C.1. (Recap of Theorem 4.1) We define the  $\xi_{\pi \to \pi'}$  as the index of samples that is from class division  $\pi$  however is closer to  $\pmb{\mu}_{\pi'}$  than  $\pmb{\mu}_{\pi}$ . In other words,  $\xi_{\pi \to \pi'} = \{i : i \in \pi, \| \mathbf{z}_i - \pmb{\mu}_{\pi} \|_2 \geq \| \mathbf{z}_i - \pmb{\mu}_{\pi'} \|_2\}$ . Assuming  $|\xi_{\pi \to \pi'}| > 0$ , we define below the clustering error ratio from  $\pi$  to  $\pi'$  as  $\mathcal{E}_{\pi \to \pi'}$  and the overall cluster error ratio  $\mathcal{E}_{\Pi, Z}$  as the Harmonic Mean of  $\mathcal{E}_{\pi \to \pi'}$  among all class pairs:
The  $K$ -means measure  $\mathcal{M}_{kms}(\Pi, Z)$  has the same order as the Harmonic Mean of the cluster error ratio between all cluster pairs:
Proof. We have the following inequality for  $i \in \xi_{\pi \to \pi'}$ :
$$
4 \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| _ {2} ^ {2} \geq 2 \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| _ {2} ^ {2} + 2 \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2} \geq \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}.
$$
$$
\begin{array}{l} \mathcal {M} _ {i n t r a} (\Pi , Z) = \sum_ {\pi \in \Pi} \sum_ {i \in \pi} \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| _ {2} ^ {2} \\ \geq \sum_ {i \in \pi} \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| _ {2} ^ {2} \\ \geq \sum_ {i \in \xi_ {\pi \rightarrow \pi^ {\prime}}} \| \mathbf {z} _ {i} - \boldsymbol {\mu} _ {\pi} \| _ {2} ^ {2} \\ \geq \frac {1}{4} \sum_ {i \in \xi_ {\pi \rightarrow \pi^ {\prime}}} \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2} \\ = \frac {1}{4} | \xi_ {\pi \rightarrow \pi^ {\prime}} | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}. \\ \end{array}
$$
Note that the inter-class measure can be decomposed into the summation of cluster center distances:
$$
\begin{array}{l} \mathcal {M} _ {i n t e r} (\Pi , Z) = \sum_ {\pi \in \Pi} | \pi | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\Pi} \| _ {2} ^ {2} \\ = \sum_ {\pi \in \Pi} \frac {| \pi |}{N ^ {2}} \left\| \left(\sum_ {\pi^ {\prime} \in \Pi} | \pi^ {\prime} |\right) \boldsymbol {\mu} _ {\pi} - \sum_ {\pi^ {\prime} \in \Pi} | \pi^ {\prime} | \boldsymbol {\mu} _ {\pi^ {\prime}} \right\| _ {2} ^ {2} \\ \leq \frac {C}{N ^ {2}} \sum_ {\pi \in \Pi} | \pi | \sum_ {\pi^ {\prime} \in \Pi} | \pi^ {\prime} | ^ {2} \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2} \\ = \frac {C}{N ^ {2}} \sum_ {\pi \neq \pi^ {\prime}} | \pi | | \pi^ {\prime} | (| \pi^ {\prime} | + | \pi |) \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}, \\ \end{array}
$$
where  $\sum_{\pi \neq \pi'}$  is enumerating over any two different class partitions in  $\Pi$ . Combining together, we have:
$$
\begin{array}{l} C (C - 1) / \left(\sum_ {\pi \neq \pi^ {\prime}} \frac {(| \pi^ {\prime} | + | \pi |)}{| \xi_ {\pi \rightarrow \pi^ {\prime}} |}\right) = C (C - 1) / \left(\sum_ {\pi \neq \pi^ {\prime}} \frac {(| \pi^ {\prime} | + | \pi |) \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}}{| \xi_ {\pi \rightarrow \pi^ {\prime}} | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}}\right) \\ \leq C (C - 1) / \left(\sum_ {\pi \neq \pi^ {\prime}} \frac {| \pi^ {\prime} | | \pi | (| \pi^ {\prime} | + | \pi |) \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}}{N ^ {2} | \xi_ {\pi \rightarrow \pi^ {\prime}} | \| \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi^ {\prime}} \| _ {2} ^ {2}}\right) \\ \leq C (C - 1) / \left(\frac {\mathcal {M} _ {i n t e r} (\Pi , Z)}{4 C \mathcal {M} _ {i n t r a} (\Pi , Z)}\right) \\ = O \left(\mathcal {M} _ {k m s} (\Pi , Z)\right). \\ \end{array}
$$
C.4 Proof of Theorem 4.2
We start by providing more details to supplement Sec. 4.2.1.
Matrix perturbation by adding labels. Recall that we define in Eq. 3 that the adjacency matrix is the unlabeled one  $A^{(u)}$  plus the perturbation of the label information  $A^{(l)}$ :
$$
A = \eta_ {u} A ^ {(u)} + \eta_ {l} A ^ {(l)}.
$$
We study the perturbation from two aspects: (1) The direction of the perturbation which is given by  $A^{(l)}$ , (2) The perturbation magnitude  $\eta_{l}$ . We first consider the perturbation direction  $A^{(l)}$  and recall that we defined the concrete form in Eq. 2:
$$
A _ {x x ^ {\prime}} ^ {(l)} = w _ {x x ^ {\prime}} ^ {(l)} \triangleq \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) \mathcal {T} (x ^ {\prime} | \bar {x} _ {l} ^ {\prime}).
$$
For simplicity, we consider  $|\mathcal{Y}_l| = 1$  in this theoretical analysis. Then we observe that  $A_{xx'}^{(l)}$  is a rank-1 matrix can be written as
$$
A _ {x x ^ {\prime}} ^ {(l)} = \mathbb {U} ^ {\top},
$$
where  $\mathfrak{l} \in \mathbb{R}^{N \times 1}$  with  $(\mathfrak{l})_x = \mathbb{E}_{\bar{x}_l \sim \mathcal{P}_{l_1}} \mathcal{T}(x|\bar{x}_l)$ . And we define  $D_l \triangleq \text{diag}(\mathfrak{l})$ .
The perturbation function of representation. We then consider a more generalized form for the adjacency matrix:
$$
A (\delta) \triangleq \eta_ {u} A ^ {(u)} + \delta \mathfrak {U} ^ {\top}.
$$
where we treat the adjacency matrix as a function of the "labeling perturbation" degree  $\delta$ . It is clear that  $A(0) = \eta_u A^{(u)}$  which is the scaled adjacency matrix for the unlabeled case and that  $A(\eta_l) = A$ . When we let the adjacency matrix be a function of  $\delta$ , the normalized form and the derived feature representation should also be the function of  $\delta$ . We proceed by defining these terms.
Without losing the generality, we let  $\text{diag}(\mathbf{1}_N^\top A(0)) = I_N$  which means the node in the unlabeled graph has equal degree. We then have:
$$
D (\delta) \triangleq d i a g \left(\mathbf {1} _ {N} ^ {\top} A (\delta)\right) = I _ {N} + \delta D _ {l}.
$$
The normalized adjacency matrix is given by:
$$
\tilde {A} (\delta) \triangleq D (\delta) ^ {- \frac {1}{2}} A (\delta) D (\delta) ^ {- \frac {1}{2}}.
$$
For feature representation  $Z(\delta)$ , it is derived from the top- $k$  SVD components of  $\tilde{A}(\delta)$ . Specifically, we have:
$$
Z (\delta) Z (\delta) ^ {\top} = D (\delta) ^ {- \frac {1}{2}} \tilde {A} _ {k} (\delta) D (\delta) ^ {- \frac {1}{2}} = D (\delta) ^ {- \frac {1}{2}} \sum_ {j = 1} ^ {k} \lambda_ {j} (\delta) \Phi_ {j} (\delta) D (\delta) ^ {- \frac {1}{2}},
$$
where we define  $\tilde{A}_k(\delta)$  as the top- $k$  SVD components of  $\tilde{A}(\delta)$  and can be further written as  $\tilde{A}_k(\delta) = \sum_{j=1}^k \lambda_j(\delta) \Phi_j(\delta)$ . Here the  $\lambda_j(\delta)$  is the  $j$ -th singular value and  $\Phi_j(\delta)$  is the  $j$ -th singular projector  $(\Phi_j(\delta) = v_j(\delta) v_j(\delta)^\top)$  defined by the  $j$ -th singular vector  $v_j(\delta)$ . For brevity, when  $\delta = 0$ , we remove the suffix (0) since it is equivalent to the unperturbed version of notations. For example, we let
$$
\tilde {A} (0) = \tilde {A} ^ {(u)}, Z (0) = Z ^ {(u)}, \lambda_ {i} (0) = \lambda_ {i} ^ {(u)}, v _ {i} (0) = v _ {i} ^ {(u)}, \Phi_ {i} (0) = \Phi_ {i} ^ {(u)}.
$$
Theorem C.2. (Recap of Th. 4.2) Denote  $V_{\emptyset}^{(u)} \in \mathbb{R}^{N \times (N - k)}$  as the null space of  $V_{k}^{(u)}$  and  $\tilde{A}_{k}^{(u)} = V_{k}^{(u)}\Sigma_{k}^{(u)}V_{k}^{(u)\top}$  as the rank- $k$  approximation for  $\tilde{A}^{(u)}$ . Given  $\delta, \eta_1 > 0$  and let  $\mathcal{G}_k$  as the spectral gap between  $k$ -th and  $k + 1$ -th singular values of  $\tilde{A}^{(u)}$ , we have:
where  $\mathrm{diag}(\cdot)$  converts the vector to the corresponding diagonal matrix and  $\Upsilon \in \mathbb{R}^{N \times N}$  is a matrix encoding the ground-truth clustering structure in the way that  $\Upsilon_{xx'} > 0$  if  $x$  and  $x'$  has the same label and  $\Upsilon_{xx'} < 0$  otherwise.
Proof. As we shown in Sec C.2, we can now also write the K-means measure as the function of perturbation:
$$
\mathcal {M} _ {k m s} (\delta) = \frac {\mathrm {T r} ((I - H _ {\Pi}) Z (\delta) Z (\delta) ^ {\top})}{\mathrm {T r} ((H _ {\Pi} - \frac {1}{N} \mathbf {1} _ {N \times N}) Z (\delta) Z (\delta) ^ {\top})}.
$$
The proof is directly given by the following Lemma C.3.
Lemma C.3. Let  $\eta_1, \eta_2$  be two real values and  $\Upsilon = (1 + \eta_2)H_{\Pi} - I - \frac{\eta_2}{N}\mathbf{1}_N\mathbf{1}_N^\top$ . Let the spectrum gap  $\mathcal{G}_k = \frac{\lambda_k^{(u)}}{\lambda_{k+1}^{(u)}}$ , we have the derivative of the  $K$ -means measure evaluated at  $\delta = 0$ :
$$
\left. \left[ \mathcal {M} _ {k m s} (\delta) \right] ^ {\prime} \right| _ {\delta = 0} = - \eta_ {1} \operatorname {T r} \left(\Upsilon \left(V _ {k} ^ {(u)} V _ {k} ^ {(u) ^ {\top}} \mathfrak {u} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l} + V _ {k} ^ {(u)} V _ {k} ^ {(u) ^ {\top}} \mathfrak {u} ^ {\top} V _ {\emptyset} ^ {(u)} V _ {\emptyset} ^ {(u) ^ {\top}}\right)\right) + O \left(\frac {1}{\mathcal {G} _ {k}}\right).
$$
The proof for Lemma C.3 is lengthy. We postpone it to Sec. C.6.
C.5 Proof of Theorem 4.3
We start by showing the justification of the assumptions made in Theorem 4.3.
Assumption C.4. We assume the spectral gap  $\mathcal{G}_k$  is large. Such an assumption is commonly used in theory works using spectral analysis [32, 57].
Assumption C.5. We assume  $\mathfrak{t}$  lies in the linear span of  $V_{k}^{(u)}$ . i.e.,  $V_{k}^{(u)}V_{k}^{(u)^{\top}}\mathfrak{t} = \mathfrak{t}, V_{\emptyset}^{(u)^{\top}}\mathfrak{t} = 0$ . The goal of this assumption is to simplify  $(V_{k}^{(u)}V_{k}^{(u)^{\top}}\mathfrak{u}^{\top} + V_{k}^{(u)}V_{k}^{(u)^{\top}}\mathfrak{u}^{\top}V_{\emptyset}^{(u)}V_{\emptyset}^{(u)^{\top}})$  to  $\mathfrak{u}^{\top}$ .
Assumption C.6. For any  $\pi_c \in \Pi$ ,  $\forall i, j \in \pi_c, \mathfrak{l}_{(i)} = \mathfrak{l}_{(j)} =: \mathfrak{l}_{\pi_c}$ . Recall that the  $\mathfrak{l}_{(i)}$  means the connection between the  $i$ -th sample to the labeled data. Here we can view  $\mathfrak{l}_{\pi_c}$  as the connection between class  $c$  to the labeled data.
Theorem C.7. (Recap of Theorem 4.3.) With Assumption C.4, C.5 and C.6. Given  $\delta, \eta_1, \eta_2 > 0$ , we have:
$$
\Delta_ {k m s} (\delta) \geq \delta \eta_ {1} \eta_ {2} \sum_ {\pi_ {c} \in \Pi} | \pi_ {c} | \mathbb {I} _ {\pi_ {c}} \Delta_ {\pi_ {c}} (\delta),
$$
$$
\Delta_ {\pi_ {c}} (\delta) = \left(\mathfrak {l} _ {\pi_ {c}} - \frac {1}{N}\right) - 2 \left(1 - \frac {| \pi_ {c} |}{N}\right) \left(\mathbb {E} _ {i \in \pi_ {c}} \mathbb {E} _ {j \in \pi_ {c}} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {j} - \mathbb {E} _ {i \in \pi_ {c}} \mathbb {E} _ {j \notin \pi_ {c}} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {j}\right).
$$
Proof. The proof is directly given by Lemma C.8 and plugging the definition of  $\Delta_{kms}(\delta)$ .
Lemma C.8. With Assumption C.4 C.5 and C.6, we have the derivative of  $K$ -means measure with the upper bound:
$$
\left. [ \mathcal {M} _ {k m s} (\delta) ] ^ {\prime} \right| _ {\delta = 0} \leq - \eta_ {1} \eta_ {2} \sum_ {\pi \in \Pi} | \pi | \mathfrak {l} _ {\pi} \left((\mathfrak {l} _ {\pi} - \frac {1}{N}) - 2 (\pmb {\mu} _ {\pi} ^ {\top} \pmb {\mu} _ {\pi} - \pmb {\mu} _ {\pi} ^ {\top} \pmb {\mu} _ {\Pi})\right).
$$
Proof. By Assumption C.4 C.5 and C.6 and Theorem 4.2, we have
$$
\begin{array}{l} \frac {1}{\eta_ {1}} [ \mathcal {M} _ {k m s} (\delta) ] ^ {\prime} \Big | _ {\delta = 0} = - \mathrm {T r} \left(\Upsilon \left(V _ {k} ^ {(u)} V _ {k} ^ {(u) \top} \mathfrak {U} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = - \operatorname {T r} \left(\Upsilon \left(\mathfrak {u} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = - \operatorname {T r} \left(\left((1 + \eta_ {2}) H _ {\Pi} - I - \frac {\eta_ {2}}{N} \mathbf {1} _ {N} \mathbf {1} _ {N} ^ {\top}\right) \left(\mathfrak {U} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = (1 + \eta_ {2}) \mathcal {M} _ {H} ^ {\prime} + \mathcal {M} _ {I} ^ {\prime} + \eta_ {2} \mathcal {M} _ {1} ^ {\prime}, \\ \end{array}
$$
$$
\begin{array}{l} \mathcal {M} _ {H} ^ {\prime} = - \operatorname {T r} \left(H _ {\Pi} \left(\mathfrak {U} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = - \sum_ {\pi \in \Pi} \left(| \pi | \left(\mathbb {E} _ {i \in \pi} \mathfrak {l} _ {(i)}\right) ^ {2} - \frac {2}{| \pi |} \sum_ {i \in \pi} \sum_ {j \in \pi} \mathfrak {l} _ {(i)} \tilde {A} _ {k, (i, j)} ^ {(u)}\right) \\ = - \sum_ {\pi \in \Pi} \left(| \pi | \mathbf {l} _ {\pi} ^ {2} - 2 | \pi | \mathbf {l} _ {\pi} \mathbb {E} _ {(i, j) \in \pi \times \pi} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {j}\right) \\ = - \sum_ {\pi \in \Pi} | \pi | l _ {\pi} \left(l _ {\pi} - 2 \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\pi}\right), \\ \end{array}
$$
$$
\begin{array}{l} \mathcal {M} _ {I} ^ {\prime} = \operatorname {T r} \left(\left(\mathfrak {U} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = \sum_ {\pi \in \Pi} | \pi | \mathfrak {l} _ {\pi} \left(\mathfrak {l} _ {\pi} - 2 \mathbb {E} _ {i \in \pi} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {i}\right), \\ \end{array}
$$
$$
\begin{array}{l} \mathcal {M} _ {\mathbf {1}} ^ {\prime} = \operatorname {T r} \left(\frac {1}{N} \mathbf {1} _ {N} \mathbf {1} _ {N} ^ {\top} \left(\mathfrak {U} ^ {\top} - 2 \tilde {A} _ {k} ^ {(u)} D _ {l}\right)\right) \\ = \frac {1}{N} - 2 \sum_ {\pi \in \Pi} \sum_ {i \in \pi} \mathfrak {l} _ {(i)} \mathbb {E} _ {j \in [ N ]} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {j} \\ = \frac {1}{N} - 2 \sum_ {\pi \in \Pi} | \pi | l _ {\pi} \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\Pi}. \\ \end{array}
$$
We observe that
$$
\begin{array}{l} \mathcal {M} _ {I} ^ {\prime} + \mathcal {M} _ {H} ^ {\prime} = - \sum_ {\pi \in \Pi} | \pi | \mathrm {l} _ {\pi} \left(\mathrm {l} _ {\pi} - 2 \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\pi}\right) + \sum_ {\pi \in \Pi} | \pi | \mathrm {l} _ {\pi} \left(\mathrm {l} _ {\pi} - 2 \mathbb {E} _ {i \in \pi} \mathbf {z} _ {i} ^ {\top} \mathbf {z} _ {i}\right) \\ = 2 \sum_ {\pi \in \Pi} | \pi | \mathfrak {l} _ {\pi} \left(\left\| \mathbb {E} _ {i \in \pi} \mathbf {z} _ {i} \right\| _ {2} ^ {2} - \mathbb {E} _ {i \in \pi} \| \mathbf {z} _ {i} \| _ {2} ^ {2}\right) \\ \leq 0, \\ \end{array}
$$
where the last inequality is by Jensen's Inequality. We then have
$$
\begin{array}{l} \frac {1}{\eta_ {1} \eta_ {2}} [ \mathcal {M} _ {k m s} (\delta) ] ^ {\prime} \bigg | _ {\delta = 0} \leq \mathcal {M} _ {H} ^ {\prime} + \mathcal {M} _ {1} ^ {\prime} \\ = - \sum_ {\pi \in \Pi} | \pi | \mathrm {l} _ {\pi} \left(\mathrm {l} _ {\pi} - 2 \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\pi}\right) + \frac {1}{N} - 2 \sum_ {\pi \in \Pi} | \pi | \mathrm {l} _ {\pi} \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\Pi} \\ = \frac {1}{N} - \sum_ {\pi \in \Pi} | \pi | \mathfrak {l} _ {\pi} \left(\mathfrak {l} _ {\pi} - 2 \left(\boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\Pi}\right)\right) \\ = - \sum_ {\pi \in \Pi} | \pi | \mathfrak {l} _ {\pi} ((\mathfrak {l} _ {\pi} - \frac {1}{N}) - 2 (\boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\pi} - \boldsymbol {\mu} _ {\pi} ^ {\top} \boldsymbol {\mu} _ {\Pi})). \\ \end{array}
$$
C.6 Proof of Lemma C.3
Notation Recap: We define  $\tilde{A}_k(\delta)$  as the top- $k$  SVD components of  $\tilde{A}(\delta)$  and can be further written as  $\tilde{A}_k(\delta) = \sum_{j=1}^k \lambda_j(\delta) \Phi_j(\delta)$ . Here the  $\lambda_j(\delta)$  is the  $j$ -th singular value and  $\Phi_j(\delta)$  is the  $j$ -th singular projector  $(\Phi_j(\delta) = v_j(\delta) v_j(\delta)^\top)$  defined by the  $j$ -th singular vector  $v_j(\delta)$ . For brevity, when  $\delta = 0$ , we remove the suffix (0) since it is equivalent to the unperturbed version of notations. For example, we let
Proof. By the derivative rule, we have,