SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Rationale for Example C.1. Recall that  $\lambda_1 \leq \dots \leq \lambda_{|\mathcal{X}|}$  are the eigenvalues of the graph Laplacian  $\mathbf{L}(\mathcal{X}) = \mathbf{I} - \overline{\mathbf{W}}(\mathcal{X})$ , and let  $\{\mathbf{v}_1, \dots, \mathbf{v}_{|\mathcal{X}|}\}$  be the associated normalized eigenvectors. For a binary classification with  $\{\mathcal{X}_1, \mathcal{X}_2\}$  perfectly separated in  $G_{\mathcal{X}}$ , we can write  $\lambda_1 = \lambda_2 = 0$ , while  $\mathbf{v}_1, \mathbf{v}_2$  are the scaled identity vectors of  $\mathcal{X}_1, \mathcal{X}_2$ , respectively, such that  $\mathbf{v}_i(\mathbf{x}) = \frac{1}{\sqrt{|\mathcal{X}_i|}} \mathbb{1}\left\{\mathbf{x} \in \mathcal{X}_i\right\}$  for  $i = 1, 2$ .
By denoting  $\mathbf{V}_2 = [\mathbf{v}_1, \mathbf{v}_2] \in \mathbb{R}^{|\mathcal{X}| \times 2}$ , the Eckart-Young-Mirsky theorem [Eckart and Young, 1936] implies that, for any orthogonal matrix  $\mathbf{Q} = [\mathbf{q}_1; \mathbf{q}_2] \in \mathbb{R}^{2 \times 2}$ ,
$$
f (\mathcal {X}) = \mathbf {D} (\mathcal {X}) ^ {- \frac {1}{2}} \mathbf {V} _ {2} \operatorname {d i a g} (1 - \lambda_ {1}, 1 - \lambda_ {2}) ^ {\frac {1}{2}} \mathbf {Q} = \mathbf {D} (\mathcal {X}) ^ {- \frac {1}{2}} \mathbf {V} _ {2} \mathbf {Q}
$$
is a minimizer of  $R(f)$  in Equation (1). That is,  $f(\mathbf{x}) = \frac{1}{\sqrt{w_{\mathbf{x}}|\mathcal{X}_i|}}\mathbf{q}_i$  for  $\mathbf{x} \in \mathcal{X}_i$ ,  $i = 1,2$ .
Spectral clustering alone. However, by taking  $\mathbf{q}_1 = \frac{1}{\sqrt{2}} [1;1]$  and  $\mathbf{q}_2 = \frac{1}{\sqrt{2}} [-1;1]$ , with a uniformly random break of ties, the corresponding  $f \in \mathcal{F}_{\mathbf{L}(\mathcal{X})}$  will misclassify half of  $\mathbf{x} \in \mathcal{X}_1$  as  $\mathcal{X}_2$  in expectation. Then, since  $P(\mathcal{X}_1) = P(\mathcal{X}_2) = \frac{1}{2}$ , by Definition 3.1, we have  $M(f) = \{\mathbf{x} \in \mathcal{X}_1 \mid y_f(\mathbf{x}) = 2\}$  such that  $\mu\left(\mathcal{F}_{\mathbf{L}(\mathcal{X})}\right) \geq P(M(f)) = \frac{1}{4}$  in expectation.
Spectral clustering in Example C.1 fails to yield low clustering error due to the potential violation of Assumption 4.2. For example, consider any prediction function  $f \in \mathcal{F}_{\mathbf{L}(\mathcal{X})}$  described in Example C.1.
(i) When  $\operatorname{argmin}_{\mathbf{x} \in \mathcal{X} \setminus M(f)} w_{\mathbf{x}} \subseteq \mathcal{X}_1$ , recalling  $\mathbf{s}_k = \operatorname{argmax}_{\mathbf{x} \in \mathcal{X} \setminus M(f)} f(\mathbf{x})_k$  and  $f(\mathbf{x}) = \frac{1}{\sqrt{w_{\mathbf{x}}|\mathcal{X}_i|}} \mathbf{q}_i$  for  $\mathbf{x} \in \mathcal{X}_i$  such that, since  $|\mathcal{X}_1| = |\mathcal{X}_2|$ ,
$$
\mathbf{s}_{1}\in \mathcal{X}_{1},\qquad \mathbf{s}_{2}\in \operatorname *{argmax}_{\mathbf{x}\in \mathcal{X}\setminus M(f)}f(\mathbf{x})_{2} = \operatorname *{argmax}_{\mathbf{x}\in \mathcal{X}\setminus M(f)}\frac{1}{\sqrt{w_{\mathbf{x}}|\mathcal{X}_{i}|}} = \operatorname *{argmin}_{\mathbf{x}\in \mathcal{X}\setminus M(f)}w_{\mathbf{x}}\subseteq \mathcal{X}_{1},
$$
there does not exist a skeleton subset  $\mathbf{S} = [\mathbf{s}_1; \mathbf{s}_2]$  as described in Assumption 4.2.
(ii) Otherwise, suppose one can find a skeleton subset  $\mathbf{S} = [\mathbf{s}_1; \mathbf{s}_2]$  as described in Assumption 4.2. Without loss of generality, by taking  $(\operatorname{argmin}_{\mathbf{x} \in \mathcal{X}} w_{\mathbf{x}}) \cap M(f) \neq \emptyset$  in the problem setup, there exists  $\mathbf{x} \in M(f) \subset \mathcal{X}_1$  (with  $y_f(\mathbf{x}) = 2$ ) such that  $w_{\mathbf{x}} \leq w_{\mathbf{s}_1}$ . Therefore,
$$
\gamma_ {1} \triangleq f (\mathbf {s} _ {1}) _ {1} - \max _ {\mathbf {x} \in M (f): y _ {f} (\mathbf {x}) \neq 1} f (\mathbf {x}) _ {1} \leq \frac {1}{\sqrt {2 w _ {\mathbf {s} _ {1}} | \mathcal {X} _ {1} |}} - \frac {1}{\sqrt {2 w _ {\mathbf {x}} | \mathcal {X} _ {1} |}} \leq 0
$$
implies a trivial margin  $\gamma \leq 0$ , which violates  $\gamma > 0$  in Assumption 4.2.
With additional weak supervision. We first observe that for both  $i \in [2]$ ,
$$
- \log \left(\operatorname {s o f t m a x} \left(f \left(\mathbf {x} _ {i}\right)\right) _ {i}\right) \leq e ^ {\widehat {\mathcal {L}} _ {\mathrm {C E}} (f)},
$$
$$
f \left(\mathbf {x} _ {1}\right) _ {1} - f \left(\mathbf {x} _ {1}\right) _ {2} \geq - \log \left(e ^ {\widehat {\mathcal {L}} _ {\mathrm {C E}} (f)} - 1\right), \quad f \left(\mathbf {x} _ {2}\right) _ {2} - f \left(\mathbf {x} _ {2}\right) _ {1} \geq - \log \left(e ^ {\widehat {\mathcal {L}} _ {\mathrm {C E}} (f)} - 1\right)
$$
where  $-\log \left(e^{\widehat{\mathcal{L}}_{\mathrm{CE}}(f)} - 1\right) > 0$  since  $\widehat{\mathcal{L}}_{\mathrm{CE}}(f) < \log \left(1 + e^{-\beta}\right) < \log (2)$ .
Since we assume  $\| f(\mathbf{x}) \|_2 \leq \beta$  for all  $\mathbf{x} \in \mathcal{X}$ ,  $f(\mathbf{x}_i)_1^2 + f(\mathbf{x}_i)_2^2 \leq \beta^2$  for both  $i = 1, 2$ , we therefore have
$$
f \left(\mathbf {x} _ {1}\right) _ {1} - f \left(\mathbf {x} _ {2}\right) _ {1} \geq - \log \left(e ^ {\widehat {\mathcal {L}} _ {\mathrm {C E}} (f)} - 1\right) - \sqrt {2 \beta^ {2} - \log \left(e ^ {\widehat {\mathcal {L}} _ {\mathrm {C E}} (f)} - 1\right) ^ {2}},
$$
This similarly holds for  $f(\mathbf{x}_2)_2 - f(\mathbf{x}_1)_2$ .
Recall now that  $f(\mathbf{x}) = \frac{1}{\sqrt{w_{\mathbf{x}}|\mathcal{X}_i|}}\mathbf{q}_i$  for  $\mathbf{x} \in \mathcal{X}_i$ ,  $i = 1,2$  and  $\mathbf{x}_i \in \mathcal{X}_i$  for  $i = 1,2$ . Assuming  $w_{\mathbf{x}} = 1 / |\mathcal{X}|$  for all  $\mathbf{x} \in \mathcal{X}$ , we have
$$
f (\mathbf {x}) = \left\{ \begin{array}{l l} \sqrt {\frac {w _ {\mathbf {x} _ {1}}}{w _ {\mathbf {x}}}} f \left(\mathbf {x} _ {1}\right) = f \left(\mathbf {x} _ {1}\right), & \mathbf {x} \in \mathcal {X} _ {1} \\ \sqrt {\frac {w _ {\mathbf {x} _ {2}}}{w _ {\mathbf {x}}}} f \left(\mathbf {x} _ {2}\right) = f \left(\mathbf {x} _ {2}\right), & \mathbf {x} \in \mathcal {X} _ {2} \end{array} \right..
$$
Therefore,  $\gamma_1 \geq f(\mathbf{x}_1)_1 - f(\mathbf{x}_2)_1$  and  $\gamma_2 \geq f(\mathbf{x}_2)_2 - f(\mathbf{x}_1)_2$ , which together imply that
Now examine the two extremes of  $\log \left(1 + e^{-\sqrt{2}\beta}\right) \leq \widehat{\mathcal{L}}_{\mathrm{CE}}(f) < \log \left(1 + e^{-\beta}\right)$ . When the cross-entropy loss achieves its minimum feasible value  $\widehat{\mathcal{L}}_{\mathrm{CE}}(f) = \log \left(1 + e^{-\sqrt{2}\beta}\right)$ , we have that  $\gamma \geq \sqrt{2}\beta$ , whereas the margin can be nearly trivial  $\gamma \geq \epsilon_{\mathrm{CE}}$  with  $\epsilon_{\mathrm{CE}} \to 0$  when the cross-entropy loss is on the larger side  $\widehat{\mathcal{L}}_{\mathrm{CE}}(f) \to \log \left(1 + e^{-\beta}\right)$ .
C.3 Connection with Sparsest  $k$ -partition
We now identify the connection between the eigenvalue  $\lambda_{K + 1}$  and spectral graph theoretic notions of clusteredness of the underlying graph. We recall the notion of sparsest  $k$ -partitions [Louis and Makarychev, 2014] based on the Dirichlet conductance of subgraphs.
Definition C.1 (Sparsest \(k\)-partition, HaoChen et al. [2021] Definition 3.4, Louis and Makarychev [2014] Problem 1.1). Given a weighted undirected graph \(G = (\mathcal{X}, w)\), for any \(k \in [| \mathcal{X}|\]\(i\), let \(\mathcal{S} = \{\{S_i\}_{i \in [k]} \mid \cup_{i \in [k]} S_i = \mathcal{X}, S_i \cap S_j = \emptyset, S_i \neq \emptyset \forall i \neq j, i, j \in [k]\}\) be the set of all \(k\)-partitions of \(G\). The sparsest \(k\)-partition of \(G\) is defined as
$$
\phi_ {k} \triangleq \min  _ {\{S _ {i} \} _ {i \in [ k ]}} \max  _ {S} \phi_ {G} (S _ {i}),
$$
where  $\phi_G(S) \triangleq \left( \sum_{\mathbf{x} \in S} \sum_{\mathbf{x}' \notin S} w_{\mathbf{x}\mathbf{x}'} \right) / \left( \sum_{\mathbf{x} \in S} w_{\mathbf{x}} \right)$  is the Dirichlet conductance of  $S \subseteq \mathcal{X}$ .
It is worth mentioning that  $\phi_{k}$  is a non-decreasing function in  $k$  [HaoChen et al., 2021]. In particular,  $\phi_{k} = 0$  when the graph has at least  $k$  disconnected components; whereas  $\lambda_{K + 1} > 0$  (Assumption 4.1) implies that  $\phi_{K + 1} > 0$ .
Meanwhile, Louis and Makarychev [2014] unveils the following connection between the sparsest  $k$ -partition of the graph and the  $k'$ th smallest eigenvalue of its graph Laplacian:
Lemma C.4 (Louis and Makarychev [2014] Proposition 1.2). Given any weighted undirected graph  $G = (\mathcal{X}, w)$ , let  $0 = \lambda_1 \leq \dots \leq \lambda_{|\mathcal{X}|}$  be the eigenvalues of the normalized graph Laplacian in the ascending order. For any  $k, k' \in [| \mathcal{X} |]$ ,  $k < k'$ , there exists a partition  $\{S_i\}_{i \in [k]} \in S$  of  $\mathcal{X}$  such that, for all  $i \in [k]$ ,
$$
\phi_ {G} \left(S _ {i}\right) \lesssim \operatorname {p o l y} \left(\frac {k}{k ^ {\prime} - k}\right) \sqrt {\log (k) \lambda_ {k ^ {\prime}}}.
$$
Leveraging the existing result Lemma C.4 [Louis and Makarychev, 2014] from spectral graph theory, we have the following corollary.
Corollary C.5 (Clustering in terms of sparsest  $k$ -partitions). Under Assumption 4.1 and Assumption 4.2 for every  $f \in \mathcal{F}_{\mathbf{L}(\mathcal{X})}$ , error of clustering with the population (Equation (1)) satisfies that, if  $\alpha > 0$  and  $\phi_k > 0$  for all  $k \in [K]$ , then
$$
\mu \left(\mathcal {F} _ {\mathbf {L} (\mathcal {X})}\right) \lesssim \max \left(\frac {\beta^ {2}}{\gamma^ {2}}, 1\right) \cdot \mathrm {p o l y} \left(\frac {k}{K + 1 - k}\right) \log (k) \frac {\alpha}{\phi_ {k} ^ {2}}.
$$
We also notice that when  $\alpha = 0$ , Theorem 4.1 automatically implies that  $\mu\left(\mathcal{F}_{\mathbf{L}(\mathcal{X})}\right) = 0$ . Intuitively, Corollary C.5 implies that except for the partition of the  $K$  ground truth classes  $\{\mathcal{X}\}_{k \in [K]}$  (i.e., when  $\alpha > 0$ ),  $G_{\mathcal{X}}$  cannot be partitioned into other  $K$  components by removing a sparse set of edges from  $G_{\mathcal{X}}$ .
Proof of Corollary C.5. Lemma C.4 implies that for any  $k \in [K]$ , there exists a partition  $\{S_i\}_{i \in [k]} \in \mathcal{S}$  of  $\mathcal{X}$  such that
$$
\phi_ {k} \leq \max  _ {i \in [ k ]} \phi_ {G} \left(S _ {i}\right) \lesssim \operatorname {p o l y} \left(\frac {k}{K + 1 - k}\right) \sqrt {\log (k) \lambda_ {K + 1}},
$$
$$
\frac {1}{\lambda_ {K + 1}} \lesssim \mathrm {p o l y} \left(\frac {k}{K + 1 - k}\right) \log (k) \frac {1}{\phi_ {k} ^ {2}},
$$
which completes the proof when recalling Theorem 4.1.
D Proofs and Discussions for Section 4.2
Here, we show that  $\widehat{R}_{\mathbf{X}^u}(f)$  shares the same minimizer as the population Laplacian regularizer  $R(f)$  in expectation (Proposition D.1) and provide generalization bounds for  $\widehat{R}_{\mathbf{X}^u}(f)$  (Theorem 4.2) accordingly.
D.1 Unbiased Estimations of Population Laplacian
Proposition D.1.  $\widehat{R}_{\mathbf{X}^u}(f)$  (Equation (3)) serves as an unbiased estimate for  $R(f)$  (Equation (1)):
$$
R (f) = \mathbb {E} _ {\mathbf {X} ^ {u} \sim P (\mathbf {x}) ^ {N}} \left[ \widehat {R} _ {\mathbf {X} ^ {u}} (f) \right].
$$
Proof of Proposition D.1. We first observe that  $R(f)$  can be expanded as following:
$$
\begin{array}{l} R (f) \triangleq \left\| \overline {{\mathbf {W}}} (\mathcal {X}) - \mathbf {D} (\mathcal {X}) ^ {\frac {1}{2}} f (\mathcal {X}) f (\mathcal {X}) ^ {\top} \mathbf {D} (\mathcal {X}) ^ {\frac {1}{2}} \right\| _ {F} ^ {2} \\ = \left\| \mathbf {D} (\mathcal {X}) ^ {- \frac {1}{2}} \mathbf {W} (\mathcal {X}) \mathbf {D} (\mathcal {X}) ^ {- \frac {1}{2}} \right\| _ {F} ^ {2} + \left\| \mathbf {D} (\mathcal {X}) ^ {\frac {1}{2}} f (\mathcal {X}) f (\mathcal {X}) ^ {\top} \mathbf {D} (\mathcal {X}) ^ {\frac {1}{2}} \right\| _ {F} ^ {2} \\ - 2 \operatorname {t r} \left(\mathbf {W} (\mathcal {X}) f (\mathcal {X}) f (\mathcal {X}) ^ {\top}\right) \\ = \sum_ {\mathbf {x} \in \mathcal {X}} \sum_ {\mathbf {x} ^ {\prime} \in \mathcal {X}} \frac {w _ {\mathbf {x x} ^ {\prime}} ^ {2}}{w _ {\mathbf {x}} w _ {\mathbf {x} ^ {\prime}}} - 2 w _ {\mathbf {x x} ^ {\prime}} f (\mathbf {x}) ^ {\top} f (\mathbf {x} ^ {\prime}) + w _ {\mathbf {x}} w _ {\mathbf {x} ^ {\prime}} \left(f (\mathbf {x}) ^ {\top} f (\mathbf {x} ^ {\prime})\right) ^ {2} \\ = \sum_ {\mathbf {x} \in \mathcal {X}} \sum_ {\mathbf {x} ^ {\prime} \in \mathcal {X}} w _ {\mathbf {x}} w _ {\mathbf {x} ^ {\prime}} \cdot \left(\left(\frac {w _ {\mathbf {x x} ^ {\prime}}}{w _ {\mathbf {x}} w _ {\mathbf {x} ^ {\prime}}}\right) ^ {2} - \frac {2 w _ {\mathbf {x x} ^ {\prime}}}{w _ {\mathbf {x}} w _ {\mathbf {x} ^ {\prime}}} f (\mathbf {x}) ^ {\top} f (\mathbf {x} ^ {\prime}) + \left(f (\mathbf {x}) ^ {\top} f (\mathbf {x} ^ {\prime})\right) ^ {2}\right). \\ \end{array}
$$
Then, with  $k_{\psi}(\mathbf{x},\mathbf{x}^{\prime}) = \frac{w_{\mathbf{x}\mathbf{x}^{\prime}}}{w_{\mathbf{x}}w_{\mathbf{x}^{\prime}}}$ ,  $\widehat{R}_{\mathbf{X}^u}(f)$  in Equation (3),  $w_{\mathbf{x}} = P(\mathbf{x})$ , and  $w_{\mathbf{x}^{\prime}} = P(\mathbf{x}^{\prime})$ , we have
$$
\begin{array}{l} R (f) = \mathbb {E} _ {\mathbf {x}, \mathbf {x} ^ {\prime} \sim P (\mathbf {x}) ^ {2}} \left[ k _ {\psi} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right) ^ {2} - 2 k _ {\psi} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right) f (\mathbf {x}) ^ {\top} f \left(\mathbf {x} ^ {\prime}\right) + \left(f (\mathbf {x}) ^ {\top} f \left(\mathbf {x} ^ {\prime}\right)\right) ^ {2} \right] \\ = \mathbb {E} _ {\mathbf {x}, \mathbf {x} ^ {\prime} \sim P (\mathbf {x}) ^ {2}} \left[ \left(f (\mathbf {x}) ^ {\top} f \left(\mathbf {x} ^ {\prime}\right) - k _ {\psi} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right)\right) ^ {2} \right] \\ = \mathbb {E} _ {\mathbf {X} ^ {u} \sim P (\mathbf {x}) ^ {N}} \left[ \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \left(f \left(\mathbf {x} _ {2 i - 1} ^ {u}\right) ^ {\top} f \left(\mathbf {x} _ {2 i} ^ {u}\right) - k _ {\psi} \left(\mathbf {x} _ {2 i - 1} ^ {u}, \mathbf {x} _ {2 i} ^ {u}\right)\right) ^ {2} \right] \\ = \mathbb {E} _ {\mathbf {X} ^ {u} \sim P (\mathbf {x}) ^ {N}} \left[ \widehat {R} _ {\mathbf {X} ^ {u}} (f) \right]. \\ \end{array}
$$
D.2 Proof of Theorem 4.2
Proof of Theorem 4.2. By defining
$$
\widehat {r} _ {\mathbf {x}, \mathbf {x} ^ {\prime}} (f) \triangleq \left(f (\mathbf {x}) ^ {\top} f (\mathbf {x} ^ {\prime}) - k _ {\psi} \left(\mathbf {x}, \mathbf {x} ^ {\prime}\right)\right) ^ {2},
$$
we have  $R(f) = \mathbb{E}_{\mathbf{x},\mathbf{x}^{\prime}\sim P(\mathbf{x})^{2}}\left[\widehat{r}_{\mathbf{x},\mathbf{x}^{\prime}}(f)\right]$  and  $\widehat{R}_{\mathbf{X}^u}(f) = \frac{2}{N}\sum_{i = 1}^{N / 2}\widehat{r}_{\mathbf{x}_{2i - 1}^u,\mathbf{x}_{2i}^u}(f)$  as
$$
\widehat {R} _ {\mathbf {X} ^ {u}} (f) = \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \left(f (\mathbf {x} _ {2 i - 1} ^ {u}) ^ {\top} f (\mathbf {x} _ {2 i} ^ {u}) - k _ {\psi} (\mathbf {x} _ {2 i - 1} ^ {u}, \mathbf {x} _ {2 i} ^ {u})\right) ^ {2},
$$
where  $\left\{\left(\mathbf{x}_{2i-1}^u, \mathbf{x}_{2i}^u\right)\right\}_{i=1}^{N/2}$  are i.i.d. given  $\mathbf{X}^u \sim P(\mathbf{x})^N$ . Consider the function class
$$
\widehat {r} _ {\cdot , \cdot} \circ \mathcal {F} = \left\{\widehat {r} _ {\cdot , \cdot} (f): \mathcal {X} \times \mathcal {X} \rightarrow \mathbb {R} | f \in \mathcal {F} \right\}.
$$
Since  $0 \leq k_{\psi}(\mathbf{x}, \mathbf{x}') \leq B_{k_{\psi}}$ , and by Cauchy-Schwarz inequality,  $f(\mathbf{x})^\top f(\mathbf{x}') \leq \| f(\mathbf{x}) \|_2 \| f(\mathbf{x}') \|_2 \leq B_f$ , we have  $0 \leq \widehat{r}_{\mathbf{x}, \mathbf{x}'}(f) \leq (B_{k_{\psi}} + B_f)^2$  for all  $\mathbf{x}, \mathbf{x}' \in \mathcal{X}$ , which means that  $\widehat{r}_{\cdot, \cdot} \circ \mathcal{F}$  is  $(B_{k_{\psi}} + B_f)^2$ -bounded. Leveraging Lemma F.1 gives that, with probability at least  $1 - \delta / 2$  over  $\mathbf{X}^u$ ,
$$
R \left(f _ {| \mathbf {X} ^ {u}}\right) - R \left(f _ {| \mathcal {X}}\right) \leq 4 \Re_ {N / 2} (\widehat {r} _ {\cdot , \cdot} \circ \mathcal {F}) + 2 \left(B _ {k _ {\psi}} + B _ {f}\right) ^ {2} \sqrt {\frac {\log (4 / \delta)}{N}}.
$$
Now, it remains to show that  $\mathfrak{R}_{N / 2}\left(\widehat{r}_{\cdot ,\cdot} \circ \mathcal{F}\right) \leq 4\sqrt{2B_f}\left(B_f + B_{k_\psi}\right)\mathfrak{R}_{N / 2}\left(\mathcal{F}\right)$ . For this, we recall Equation (4) and observe that
$$
\mathfrak {R} _ {N / 2} \left(\widehat {r} _ {\cdot , \cdot} \circ \mathcal {F}\right) = \underset {\boldsymbol {\rho} \sim \mathrm {R a d} ^ {\frac {N}{2}}} {\mathbb {E}} \underset {N} {\sup} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \left(f \left(\mathbf {x} _ {2 i - 1} ^ {u}\right) ^ {\top} f \left(\mathbf {x} _ {2 i} ^ {u}\right) - k _ {\psi} \left(\mathbf {x} _ {2 i - 1} ^ {u}, \mathbf {x} _ {2 i} ^ {u}\right)\right) ^ {2} \Biggr ],
$$
where with Talagrand's lemma [Ledoux and Talagrand, 2013, Theorem 4.12] for compositions with scalar Lipschitz functions,
$$
\mathfrak {R} _ {N / 2} \left(\widehat {r} _ {\cdot , \cdot} \circ \mathcal {F}\right) \leq 2 \left(B _ {f} + B _ {k _ {\psi}}\right) \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup  _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot f \left(\mathbf {x} _ {2 i - 1} ^ {u}\right) ^ {\top} f \left(\mathbf {x} _ {2 i} ^ {u}\right) \right].
$$
Since  $f\left(\mathbf{x}_{2i - 1}^{u}\right)^{\top}f\left(\mathbf{x}_{2i}^{u}\right)\leq \frac{1}{2}\left(\left\| f\left(\mathbf{x}_{2i - 1}^{u}\right)\right\|_{2}^{2} + \left\| f\left(\mathbf{x}_{2i}^{u}\right)\right\|_{2}^{2}\right),$
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup  _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot f \left(\mathbf {x} _ {2 i - 1} ^ {u}\right) ^ {\top} f \left(\mathbf {x} _ {2 i} ^ {u}\right) \right] \\ \leq \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \frac {1}{2} \left(\left\| f \left(\mathbf {x} _ {2 i - 1} ^ {u}\right) \right\| _ {2} ^ {2} + \left\| f \left(\mathbf {x} _ {2 i} ^ {u}\right) \right\| _ {2} ^ {2}\right) \right] \\ \leq \frac {1}{2} \left(\mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \left\| f (\mathbf {x} _ {2 i - 1} ^ {u}) \right\| _ {2} ^ {2} \right] + \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \| f (\mathbf {x} _ {2 i} ^ {u}) \| _ {2} ^ {2} \right]\right) \\ = \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \| f (\mathbf {x} _ {2 i} ^ {u}) \| _ {2} ^ {2} \right]. \\ \end{array}
$$
By the vector-contraction inequality for Rademacher complexities [Maurer, 2016, Corollary 4], since  $\| f(\mathbf{x})\| _2^2$  is  $(2\sqrt{B_f})$ -Lipschitz in  $f(\mathbf{x})$ , with  $f(\mathbf{x})_k$  denoting the  $k$ -th entry of  $f(\mathbf{x}) \in \mathbb{R}^K$ ,
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {X} ^ {u}, \boldsymbol {\rho}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \rho_ {i} \cdot \| f (\mathbf {x} _ {2 i} ^ {u}) \| _ {2} ^ {2} \right] \leq 2 \sqrt {2 B _ {f}} \cdot \mathbb {E} _ {\underset {\boldsymbol {\rho} \sim \mathrm {R a d} ^ {\frac {N}{2} \times K}} {\mathbf {X} ^ {u} \sim P (\mathbf {x}) ^ {\frac {N}{2}}}} \left[ \sup _ {f \in \mathcal {F}} \frac {2}{N} \sum_ {i = 1} ^ {N / 2} \sum_ {k = 1} ^ {K} \rho_ {i k} \cdot f (\mathbf {x} _ {i} ^ {u}) _ {k} \right] \\ = 2 \sqrt {2 B _ {f}} \mathfrak {N} _ {N / 2} (\mathcal {F}). \\ \end{array}
$$
Overall, we have  $\Re_{N / 2}\left(\widehat{r}_{\cdot ,\cdot}\circ \mathcal{F}\right)\leq 4\sqrt{2B_f}\left(B_f + B_{k_\psi}\right)\Re_{N / 2}(\mathcal{F})$  which completes the proof.
D.3 Unlabeled Sample Complexity with Deep Neural Networks
Here, we ground Theorem 4.2 by exemplifying  $\Re_{N / 2}(\mathcal{F})$  leveraging the existing generalization bound [Golowich et al., 2018] for deep neural networks.
Claim D.2 (Rademacher complexity of deep neural networks [Golowich et al., 2018]). Recall the notion of Rademacher complexity for a vector-valued function class  $\mathcal{F} \ni f: \mathcal{X} \to \mathbb{R}^K$  from Equation (4). Adopting the existing results in Golowich et al. [2018] and following the notations in Section 6, we consider  $\mathcal{F}$  as a class of deep neural networks:
$$
\mathcal {F} = \left\{f (\mathbf {x}) = \mathbf {A} _ {p} \phi \left(\dots \mathbf {A} _ {2} \phi \left(\mathbf {A} _ {1} \mathbf {x}\right) \dots\right) \mid \| \mathbf {A} _ {\iota} \| _ {F} \leq B _ {\mathcal {F}, \iota} \forall \iota \in [ p ] \right\}
$$
with weight matrices  $\{\mathbf{A}_{\iota} \mid \iota \in [p]\}$  and a 1-Lipschitz, positive-homogeneous activation function  $\phi(\cdot)$  (e.g., ReLU) applied entry-wisely. Let  $B_{\mathcal{F}} = \prod_{\iota=1}^{p} B_{\mathcal{F},\iota}$  and  $\|\mathbf{x}\|_2 \leq B_{\mathcal{X}}$  for all  $\mathbf{x} \in \mathcal{X}$ . Then, by Golowich et al. [2018] Theorem 1, we have
$$
\mathfrak {R} _ {N / 2} (\mathcal {F}) \leq \frac {\left(2 \sqrt {p \log (2)} + \sqrt {2}\right) K B _ {\mathcal {X}} B _ {\mathcal {F}}}{\sqrt {N}}.
$$
D.4 Proof of Theorem 4.3
Proof of Theorem 4.3. Let  $\mathbf{U}_{\mathcal{X}} \triangleq \mathbf{D}(\mathcal{X})^{1/2} f_{|\mathbf{X}^u}(\mathcal{X}), \mathbf{F}_{\mathcal{X}} \triangleq \mathbf{D}(\mathcal{X})^{1/2} f_{|\mathcal{X}}(\mathcal{X})$ , and  $\mathbf{y}_k \in [0,1]^{|\mathcal{X}|}$  be the  $k$ -th column of  $\mathbf{Y}_{\mathcal{X}} \triangleq \mathbf{D}(\mathcal{X})^{1/2} \overrightarrow{y_*}(\mathcal{X})$ . We denote  $\mathbf{P}_{\mathbf{U}} \triangleq \mathbf{U}_{\mathcal{X}} \mathbf{U}_{\mathcal{X}}^\dagger$  as the orthogonal projection onto Range  $(\mathbf{U}_{\mathcal{X}})$  and  $\mathbf{P}_{\mathbf{U}}^\perp \triangleq \mathbf{I} - \mathbf{P}_{\mathbf{U}}$  as the orthogonal complement of  $\mathbf{P}_{\mathbf{U}}$ .
Recalling from the proof of Theorem 4.1, Lemma C.1 implies that for all  $f_{|\mathbf{X}^u}\in \mathcal{F}_{\mathbf{L}(\mathbf{X}^u)}$
$$
P \left(M (f _ {| \mathbf {X} ^ {u}})\right) \leq 2 \max  \left(\frac {\beta^ {2}}{\gamma^ {2}}, 1\right) \sum_ {k \in [ K ]} \min  _ {\mathbf {z} _ {k} \in \mathbb {R} ^ {K}} \| \mathbf {y} _ {k} - \mathbf {U} _ {\mathcal {X}} \mathbf {z} _ {k} \| _ {2} ^ {2} = 2 \max  \left(\frac {\beta^ {2}}{\gamma^ {2}}, 1\right) \sum_ {k \in [ K ]} \| \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k} \| _ {2} ^ {2},
$$
where the second equality comes from taking  $\mathbf{z}_k = \mathbf{U}_{\mathcal{X}}^\dagger \mathbf{y}_k$  for every  $k \in [K]$ .
Consider the spectral decomposition of  $\mathbf{L}(\mathcal{X})$  in Equation (13). Since the orthonormal eigenvectors  $\{\mathbf{v}_i\in \mathbb{R}^{|\mathcal{X}|}\}_{i\in [|\mathcal{X}|]}$  associated with eigenvalues  $0 = \lambda_{1}\leq \dots \leq \lambda_{|\mathcal{X}|}\leq 1$  form a basis of  $\mathbb{R}^{|\mathcal{X}|}$ , by denoting  $\mathbf{V}_K\triangleq [\mathbf{v}_1,\ldots ,\mathbf{v}_K]\in \mathbb{R}^{|\mathcal{X}|\times K}$  and  $\mathbf{V}_K^\perp \triangleq [\mathbf{v}_{K + 1},\dots ,\mathbf{v}_{|\mathcal{X}|}]\in \mathbb{R}^{|\mathcal{X}|\times K}$ , for every  $k\in [K]$ ,
$$
\left\| \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k} \right\| _ {2} ^ {2} = \sum_ {i = 1} ^ {| \mathcal {X} |} \left(\mathbf {v} _ {i} ^ {\top} \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k}\right) ^ {2} = \left\| \mathbf {V} _ {K} ^ {\top} \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k} \right\| _ {2} ^ {2} + \left\| \left(\mathbf {V} _ {K} ^ {\perp}\right) ^ {\top} \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k} \right\| _ {2} ^ {2}.
$$
It is shown in the proof of Theorem 4.1 that
$$
\sum_ {k \in [ K ]} \left\| \left(\mathbf {V} _ {K} ^ {\perp}\right) ^ {\top} \mathbf {P} _ {\mathbf {U}} ^ {\perp} \mathbf {y} _ {k} \right\| _ {2} ^ {2} \leq \sum_ {k \in [ K ]} \left\| \left(\mathbf {V} _ {K} ^ {\perp}\right) ^ {\top} \mathbf {y} _ {k} \right\| _ {2} ^ {2} \leq \frac {\alpha}{\lambda_ {K + 1}},
$$
whereas by observing that  $\| \mathbf{y}_k\| _2^2 = \sum_{\mathbf{x}\in \mathcal{X}}w_{\mathbf{x}}\cdot \mathbb{1}\left\{y_*(\mathbf{x}) = k\right\} = P(\mathcal{X}_k)$