SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} w _ {x} = \sum_ {x ^ {\prime}} w _ {x x ^ {\prime}} \\ = \eta_ {l} \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) \sum_ {x ^ {\prime}} \mathcal {T} \left(x ^ {\prime} | \bar {x} _ {l} ^ {\prime}\right) + \eta_ {u} \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \mathcal {T} (x | \bar {x} _ {u}) \sum_ {x ^ {\prime}} \mathcal {T} \left(x ^ {\prime} | \bar {x} _ {u}\right) \\ = \eta_ {l} \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) + \eta_ {u} \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \mathcal {T} (x | \bar {x} _ {u}). \\ \end{array}
$$
Plugging in  $w_{xx'}$  we have,
$$
\begin{array}{l} - 2 \sum_ {x, x ^ {\prime} \in \mathcal {X}} w _ {x x ^ {\prime}} f (x) ^ {\top} f \left(x ^ {\prime}\right) \\ = - 2 \sum_ {x, x ^ {+} \in \mathcal {X}} w _ {x x ^ {+}} f (x) ^ {\top} f \left(x ^ {+}\right) \\ = - 2 \eta_ {l} \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathcal {P} _ {l _ {i}}} \sum_ {x, x ^ {\prime} \in \mathcal {X}} \mathcal {T} (x | \bar {x} _ {l}) \mathcal {T} \left(x ^ {\prime} | \bar {x} _ {l} ^ {\prime}\right) f (x) ^ {\top} f \left(x ^ {\prime}\right) \\ - 2 \eta_ {u} \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \sum_ {x, x ^ {\prime}} \mathcal {T} (x | \bar {x} _ {u}) \mathcal {T} \left(x ^ {\prime} | \bar {x} _ {u}\right) f (x) ^ {\top} f \left(x ^ {\prime}\right) \\ = -2\eta \sum_{i\in \mathcal{Y}_{l}}\underset { \begin{array}{c}\bar{x}_{l}\sim \mathcal{P}_{l_{i}},  \bar{x}_{l}^{\prime}\sim \mathcal{P}_{l_{i}},\\ x\sim \mathcal{T}(\cdot |\bar{x}_{l}),x^{+}\sim \mathcal{T}(\cdot |\bar{x}_{l}^{\prime}) \end{array} }{\mathbb{E}}\left[f(x)^{\top}f\left(x^{+}\right)\right] \\ - 2\eta_{u}\underset { \begin{array}{c}\bar{x}_{u}\sim \mathcal{P},\\ x\sim \mathcal{T}(\cdot |\bar{x}_{u}),x^{+}\sim \mathcal{T}(\cdot |\bar{x}_{u}) \end{array} }{\mathbb{E}}\left[f(x)^{\top}f\left(x^{+}\right)\right] \\ = - 2 \eta_ {l} \mathcal {L} _ {1} (f) - 2 \eta_ {u} \mathcal {L} _ {2} (f). \\ \end{array}
$$
Plugging  $w_{x}$  and  $w_{x^{\prime}}$  we have,
$$
\begin{array}{l} \sum_ {x, x ^ {\prime} \in \mathcal {X}} w _ {x} w _ {x ^ {\prime}} \left(f (x) ^ {\top} f \left(x ^ {\prime}\right)\right) ^ {2} \\ = \sum_ {x, x ^ {-} \in \mathcal {X}} w _ {x} w _ {x ^ {-}} \left(f (x) ^ {\top} f \left(x ^ {-}\right)\right) ^ {2} \\ = \sum_ {x, x ^ {\prime} \in \mathcal {X}} \left(\eta_ {l} \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) + \eta_ {u} \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \mathcal {T} (x | \bar {x} _ {u})\right) \\ \cdot \left(\eta_ {l} \sum_ {j \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathcal {P} _ {l _ {j}}} \mathcal {T} \left(x ^ {-} | \bar {x} _ {l} ^ {\prime}\right) + \eta_ {u} \mathbb {E} _ {\bar {x} _ {u} ^ {\prime} \sim \mathcal {P}} \mathcal {T} \left(x ^ {-} | \bar {x} _ {u} ^ {\prime}\right)\right) \left(f (x) ^ {\top} f \left(x ^ {-}\right)\right) ^ {2} \\ = \eta_ {l} ^ {2} \sum_ {x, x ^ {-} \in \mathscr {X}} \sum_ {i \in \mathscr {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathscr {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) \sum_ {j \in \mathscr {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathscr {P} _ {l _ {j}}} \mathcal {T} (x ^ {-} | \bar {x} _ {l} ^ {\prime}) \left(f (x) ^ {\top} f \left(x ^ {-}\right)\right) ^ {2} \\ + 2 \eta_ {u} \eta_ {l} \sum_ {x, x ^ {-} \in \mathcal {X}} \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \mathcal {T} (x ^ {-} | \bar {x} _ {u}) \left(f (x) ^ {\top} f (x ^ {-})\right) ^ {2} \\ + \eta_ {u} ^ {2} \sum_ {x, x ^ {-} \in \mathcal {X}} \mathbb {E} _ {\bar {x} _ {u} \sim \mathcal {P}} \mathcal {T} (x | \bar {x} _ {u}) \mathbb {E} _ {\bar {x} _ {u} ^ {\prime} \sim \mathcal {P}} \mathcal {T} (x ^ {-} | \bar {x} _ {u} ^ {\prime}) \left(f (x) ^ {\top} f (x ^ {-})\right) ^ {2} \\ = \eta_{l}^{2}\sum_{i\in \mathcal{Y}_{l}}\sum_{j\in \mathcal{Y}_{l}}\underset { \begin{array}{c}\bar{x}_{l}\sim \mathcal{P}_{l_{i}},\bar{x}_{l}^{\prime}\sim \mathcal{P}_{l_{j}},\\ x\sim \mathcal{T}(\cdot |\bar{x}_{l}),x^{-}\sim \mathcal{T}(\cdot |\bar{x}_{l}^{\prime}) \end{array} }{\mathbb{E}}\left[  \big(f(x)^{\top}f\left(x^{-}\right)\big)^{2}\right] \\ +2\eta_{u}\eta_{l}\sum_{i\in \mathcal{Y}_{l}}\underset { \begin{array}{c}\bar{x}_{l}\sim \mathcal{P}_{l_{i}},\bar{x}_{u}\sim \mathcal{P},\\ x\sim \mathcal{T}(\cdot |\bar{x}_{l}),x^{-}\sim \mathcal{T}(\cdot |\bar{x}_{u}) \end{array} }{\mathbb{E}}\left[ \left(f(x)^{\top}f\left(x^{-}\right)\right)^{2}\right] \\ + \eta_{u}^{2}\underset { \begin{array}{c}\bar{x}_{u}\sim \mathcal{P},\bar{x}_{u}^{\prime}\sim \mathcal{P},\\ x\sim \mathcal{T}(\cdot |\bar{x}_{u}),x^{-}\sim \mathcal{T}(\cdot |\bar{x}_{u}^{\prime}) \end{array} }{\mathbb{E}}\left[  \left(f(x)^{\top}f\left(x^{-}\right)\right)^{2}\right] \\ = \eta_ {l} ^ {2} \mathcal {L} _ {3} (f) + 2 \eta_ {u} \eta_ {l} \mathcal {L} _ {4} (f) + \eta_ {u} ^ {2} \mathcal {L} _ {5} (f). \\ \end{array}
$$
B Technical Details for Toy Example
B.1 Calculation Details for Figure 2.
We first recap the toy example, which illustrates the core idea of our theoretical findings. Specifically, the example aims to distinguish 3D objects with different shapes, as shown in Figure 2. These images are generated by a 3D rendering software [31] with user-defined properties including colors, shape, size, position, etc.
Data design. Suppose the training samples come from three types,  $\mathcal{X}_{\square}, \mathcal{X}_{\bigcirc}, \mathcal{X}_{\bigodot}$ . Let  $\mathcal{X}_{\square}$  be the sample space with known class, and  $\mathcal{X}_{\bigcirc}, \mathcal{X}_{\bigodot}$  be the sample space with novel classes. Further, the two novel classes are constructed to have different relationships with the known class. Specifically, we construct the toy dataset with 6 elements as shown in Figure 4(a).
$$
\mathcal {T} (x \mid \bar {x}) = \left\{ \begin{array}{l l} \tau_ {1} & \text {i f c o l o r} (x) = \operatorname {c o l o r} (\bar {x}), \operatorname {s h a p e} (x) = \operatorname {s h a p e} (\bar {x}); \\ \tau_ {c} & \text {i f c o l o r} (x) = \operatorname {c o l o r} (\bar {x}), \operatorname {s h a p e} (x) \neq \operatorname {s h a p e} (\bar {x}); \\ \tau_ {s} & \text {i f c o l o r} (x) \neq \operatorname {c o l o r} (\bar {x}), \operatorname {s h a p e} (x) = \operatorname {s h a p e} (\bar {x}); \\ \tau_ {0} & \text {i f c o l o r} (x) \neq \operatorname {c o l o r} (\bar {x}), \operatorname {s h a p e} (x) \neq \operatorname {s h a p e} (\bar {x}). \end{array} \right. \tag {11}
$$
According to the definition above, the corresponding augmentation matrix  $T$  with each element formed by  $\mathcal{T}(\cdot \mid \cdot)$  is given in Figure 4(b). We proceed by showing the details to derive  $A^{(u)}$  and  $A$  using  $T$ .
(a) Definition of Augmentation Probability
(b) Augmentation Graph
Figure 4: An illustrative example for theoretical analysis. We consider a 6-node graph with one known class (cube) and two novel classes (sphere, cylinder). (a) The augmentation probabilities between nodes are defined by their color and shape in Eq. (11). (b) The augmentation matrices  $T$  derived by Eq. (11) where we let  $\tau_0 = 0$ .
Derivation details for  $A^{(u)}$  and  $A$ . Recall that each element of  $A^{(u)}$  is formed by  $w_{xx'}^{(u)} = \mathbb{E}_{\bar{x} \sim \mathcal{P}} \mathcal{T}(x|\bar{x}) \mathcal{T}(x'|\bar{x})$ . In this toy example, one can then see that  $A^{(u)} = \frac{1}{6} TT^\top$  since augmentation matrix  $T$  is defined that each element  $T_{x\bar{x}} = \mathcal{T}(x|\bar{x})$ . Note that  $T$  is explicitly given in Figure 4(b) and then if we let  $\eta_u = 6$ , we have the close-from:
$$
\eta_ {u} A ^ {(u)} = T ^ {2} = \left[ \begin{array}{c c c c c c} \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {1} \tau_ {s} & 2 \tau_ {1} \tau_ {c} & 2 \tau_ {c} \tau_ {s} & 0 & 0 \\ 2 \tau_ {1} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {c} \tau_ {s} & 2 \tau_ {1} \tau_ {c} & 0 & 0 \\ 2 \tau_ {1} \tau_ {c} & 2 \tau_ {c} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {1} \tau_ {s} & 0 & 0 \\ 2 \tau_ {c} \tau_ {s} & 2 \tau_ {1} \tau_ {c} & 2 \tau_ {1} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 2 \tau_ {1} ^ {2} & 2 \tau_ {1} ^ {2} \\ 0 & 0 & 0 & 0 & 2 \tau_ {1} ^ {2} & 2 \tau_ {1} ^ {2} \end{array} \right].
$$
We then derive the second part  $A^{(l)}$  whose element is given by:
$$
w _ {x x ^ {\prime}} ^ {(l)} \triangleq \sum_ {i \in \mathcal {Y} _ {l}} \mathbb {E} _ {\bar {x} _ {l} \sim \mathcal {P} _ {l _ {i}}} \mathbb {E} _ {\bar {x} _ {l} ^ {\prime} \sim \mathcal {P} _ {l _ {i}}} \mathcal {T} (x | \bar {x} _ {l}) \mathcal {T} \left(x ^ {\prime} | \bar {x} _ {l} ^ {\prime}\right).
$$
Such a form can be simplified in Section 4 by defining  $\mathfrak{l} \in \mathbb{R}^N$ ,  $(\mathfrak{l})_x = \mathbb{E}_{\bar{x}_l \sim \mathcal{P}_{l_1}} \mathcal{T}(x|\bar{x}_l)$  and by letting  $|\mathcal{Y}_l| = 1$ . In this toy example, the known class only has two elements, so  $\mathfrak{l} = \frac{1}{2}(T_{:,1} + T_{:,2})$  (average of  $T$ 's 1st & 2nd column), we then have:
$$
A ^ {(l)} = \mathfrak {U} ^ {\top} = \frac {1}{4} \left[ \begin{array}{c c c c c c} (\tau_ {1} + \tau_ {s}) ^ {2} & (\tau_ {1} + \tau_ {s}) ^ {2} & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & 0 & 0 \\ (\tau_ {1} + \tau_ {s}) ^ {2} & (\tau_ {1} + \tau_ {s}) ^ {2} & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & 0 & 0 \\ \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} ^ {2} & \tau_ {c} ^ {2} & 0 & 0 \\ \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} ^ {2} & \tau_ {c} ^ {2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right].
$$
Finally, if we let  $\eta_l = 4$  and  $A = \eta_u A^{(u)} + \eta_l A^{(l)}$ , we have the full results in Figure 2.
B.2 Calculation Details for Figure 3.
In this section, we present the analysis of eigenvectors and their orders for toy examples shown in Figure 2. In Theorem B.1 we present the spectral analysis for the adjacency matrix with additional label information while in Theorem B.2, we show the spectral analysis for the unlabeled case.
Theorem B.1. Let
$$
\eta_ {u} A ^ {(u)} = \left[ \begin{array}{c c c c c c} \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {1} \tau_ {s} & 2 \tau_ {1} \tau_ {c c c c c} & 2 \tau_ {c} \tau_ {s} & 0 & 0 \\ 2 \tau_ {1} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {c} \tau_ {s} & 2 \tau_ {1} \tau_ {c c c c c} & 0 & 0 \\ 2 \tau_ {1} \tau_ {c c c c c} & 2 \tau_ {c} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 2 \tau_ {1} \tau_ {s} & 0 & 0 \\ 2 \tau_ {c} \tau_ {s} & 2 \tau_ {1} \tau_ {c c c c c} & 2 \tau_ {1} \tau_ {s} & \tau_ {1} ^ {2} + \tau_ {s} ^ {2} + \tau_ {c} ^ {2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 2 \tau_ {1} ^ {2} & 2 \tau_ {1} ^ {2} \\ 0 & 0 & 0 & 0 & 2 \tau_ {1} ^ {2} & 2 \tau_ {1} ^ {2} \end{array} \right],
$$
$$
A = \eta_ {u} A ^ {(u)} + \left[ \begin{array}{c c c c c c} (\tau_ {1} + \tau_ {s}) ^ {2} & (\tau_ {1} + \tau_ {s}) ^ {2} & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & 0 & 0 \\ (\tau_ {1} + \tau_ {s}) ^ {2} & (\tau_ {1} + \tau_ {s}) ^ {2} & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & 0 & 0 \\ \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} ^ {2} & \tau_ {c} ^ {2} & 0 & 0 \\ \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} (\tau_ {1} + \tau_ {s}) & \tau_ {c} ^ {2} & \tau_ {c} ^ {2} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right],
$$
and we assume that  $1 \gg \frac{\tau_c}{\tau_1} > \frac{\tau_s}{\tau_1} > 0$ ,  $\frac{4}{9}\tau_c \leq \tau_s \leq \tau_c$  and  $\tau_1 + \tau_c + \tau_s = 1$ .
Let  $\lambda_1, \lambda_2, \lambda_3$  and  $v_1, v_2, v_3$  be the largest three eigenvalues and their corresponding eigenvectors of  $D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$ , which is the normalized adjacency matrix of  $A$ . Then the concrete form of  $\lambda_1, \lambda_2, \lambda_3$  and  $v_1, v_2, v_3$  can be approximately given by:
$$
\hat {\lambda} _ {1} = 1, \hat {\lambda} _ {2} = 1, \hat {\lambda} _ {3} = 1 - \frac {1 6}{3} \frac {\tau_ {c}}{\tau_ {1}},
$$
$$
\hat {v} _ {1} = [ 0, 0, 0, 0, 1, 1 ],
$$
$$
\hat {v} _ {2} = [ \sqrt {3}, \sqrt {3}, 1, 1, 0, 0 ],
$$
$$
\hat {v} _ {3} = [ 1, 1, - \sqrt {3}, - \sqrt {3}, 0, 0 ].
$$
Note that the approximation gap can be tightly bounded. Specifically, for  $i \in \{1,2,3\}$ , we have  $|\lambda_i - \hat{\lambda}_i| \leq O((\frac{\tau_c}{\tau_1})^2)$  and  $\| \sin (U,\hat{U})^6\| _F \leq O(\frac{\tau_c}{\tau_1})$ , where  $U = [v_{1},v_{2},v_{3}],\hat{U} = [\hat{v}_{1},\hat{v}_{2},\hat{v}_{3}]$ .
<sup>6</sup>The sin operation measures the distance of two matrices with orthonormal columns, which is usually used in the subspace distance. See more in https://trungvietvu.github.io/notes/2020/DavisKahan.
Proof. By  $\tau_{1} + \tau_{c} + \tau_{s} = 1$  and  $1 \gg \frac{\tau_{c}}{\tau_{1}} > \frac{\tau_{s}}{\tau_{1}} > 0$ , we define the following equation which approximates the corresponding terms up to error  $O\left(\left(\frac{\tau_{c}}{\tau_{1}}\right)^{2}\right)$ :
$$
A \approx \widehat {A} = \tau_ {1} ^ {2} \left[ \begin{array}{c c c c c c} 2 + 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 + 4 \frac {\tau_ {s}}{\tau_ {1}} & 3 \frac {\tau_ {c}}{\tau_ {1}} & \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 1 + 4 \frac {\tau_ {s}}{\tau_ {1}} & 2 + 2 \frac {\tau_ {s}}{\tau_ {1}} & \frac {\tau_ {c}}{\tau_ {1}} & 3 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 3 \frac {\tau_ {c}}{\tau_ {1}} & \frac {\tau_ {c}}{\tau_ {1}} & 1 & 2 \frac {\tau_ {s}}{\tau_ {1}} & 0 & 0 \\ \frac {\tau_ {c}}{\tau_ {1}} & 3 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 2 & 2 \\ 0 & 0 & 0 & 0 & 2 & 2 \end{array} \right].
$$
$$
D \approx \widehat {D} = \tau_ {1} ^ {2} d i a g \left(\left[ 3 \left(1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right), 3 \left(1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right), 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 4 \frac {\tau_ {c}}{\tau_ {1}}, 1 + 2 \frac {\tau_ {s}}{\tau_ {1}} + 4 \frac {\tau_ {c}}{\tau_ {1}}, 4, 4 \right]\right).
$$
$$
D ^ {- \frac {1}{2}} \approx \widehat {D ^ {- \frac {1}{2}}} = \frac {1}{\tau_ {1}} d i a g \left(\left[ \sqrt {3} \left(1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {2}{3} \frac {\tau_ {c}}{\tau_ {1}}\right), \sqrt {3} \left(1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {2}{3} \frac {\tau_ {c}}{\tau_ {1}}\right), 1 - \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}}, 1 - \frac {\tau_ {s}}{\tau_ {1}} - 2 \frac {\tau_ {c}}{\tau_ {1}}, 2, 2 \right]\right).
$$
$$
\begin{array}{l} D ^ {- \frac {1}{2}} A D ^ {- \frac {1}{2}} \approx \widehat {D ^ {- \frac {1}{2}}} \widehat {A D ^ {- \frac {1}{2}}} \\ = \left[ \begin{array}{c c c c c} \frac {2}{3} \left(1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right) & \frac {1}{3} \left(1 + 2 \frac {\tau_ {s}}{\tau_ {1}} - \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right) & \sqrt {3} \frac {\tau_ {c}}{\tau_ {1}} & \frac {1}{\sqrt {3}} \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ \frac {1}{3} \left(1 + 2 \frac {\tau_ {s}}{\tau_ {1}} - \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right) & \frac {2}{3} \left(1 - \frac {\tau_ {s}}{\tau_ {1}} - \frac {4}{3} \frac {\tau_ {c}}{\tau_ {1}}\right) & \frac {1}{\sqrt {3}} \frac {\tau_ {c}}{\tau_ {1}} & \sqrt {3} \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ \sqrt {3} \frac {\tau_ {c}}{\tau_ {1}} & \frac {1}{\sqrt {3}} \frac {\tau_ {c}}{\tau_ {1}} & 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 4 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 0 & 0 \\ \frac {1}{\sqrt {3}} \frac {\tau_ {c}}{\tau_ {1}} & \sqrt {3} \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 - 2 \frac {\tau_ {s}}{\tau_ {1}} - 4 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 0 & 0 & 0 & 0 & \frac {1}{2} & \frac {1}{2} \\ 0 & 0 & 0 & 0 & \frac {1}{2} & \frac {1}{2} \end{array} \right]. \\ \end{array}
$$
And we have
$$
\begin{array}{l} \left\| D ^ {- \frac {1}{2}} A D ^ {- \frac {1}{2}} - \widehat {D ^ {- \frac {1}{2}}} \widehat {A} D ^ {- \frac {1}{2}} \right\| _ {2} \\ \leq \left\| D ^ {- \frac {1}{2}} A D ^ {- \frac {1}{2}} - \widehat {D ^ {- \frac {1}{2}}} \widehat {A} D ^ {- \frac {1}{2}} \right\| _ {F} \\ \leq O ((\frac {\tau_ {c}}{\tau_ {1}}) ^ {2}). \end{array}
$$
Let  $\hat{\lambda}_a, \dots, \hat{\lambda}_f$  be six eigenvalues of  $\widehat{D^{-\frac{1}{2}}\hat{A}D^{-\frac{1}{2}}}$ , and  $\hat{v}_a, \dots, \hat{v}_f$  be corresponding eigenvectors. By direct calculation we have
$$
\hat {\lambda} _ {a} = 1, \hat {\lambda} _ {b} = 1, \hat {\lambda} _ {c} = 1 - \frac {1 6}{3} \frac {\tau_ {c}}{\tau_ {1}}, \hat {\lambda} _ {d} = 0
$$
and corresponding eigenvectors as
$$
\hat {v} _ {a} = [ 0, 0, 0, 0, 1, 1 ],
$$
$$
\hat {v} _ {b} = [ \sqrt {3}, \sqrt {3}, 1, 1, 0, 0 ],
$$
$$
\hat {v} _ {c} = [ 1, 1, - \sqrt {3}, - \sqrt {3}, 0, 0 ],
$$
$$
\hat {v} _ {d} = [ 0, 0, 0, 0, 1, - 1 ].
$$
For the remaining two eigenvectors, by the symmetric property, they have the formula
$$
\hat {v} _ {e} = \left[ \alpha \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), - \alpha \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), \beta \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), - \beta \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), 0, 0 \right],
$$
$$
\hat {v} _ {f} = \left[ \beta \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), - \beta \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), - \alpha \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), \alpha \left(\frac {\tau_ {s}}{\tau_ {1}}, \frac {\tau_ {c}}{\tau_ {1}}\right), 0, 0 \right],
$$
where  $\alpha, \beta$  are some real functions. Then, by solving
$$
\widehat {D ^ {- \frac {1}{2}}} \widehat {A D ^ {- \frac {1}{2}}} \hat {v} _ {e} = \hat {\lambda} _ {e} \hat {v} _ {e}
$$
$$
\widehat {D ^ {- \frac {1}{2}}} \widehat {\bar {A}} \widehat {D ^ {- \frac {1}{2}}} \hat {v} _ {f} = \hat {\lambda} _ {f} \hat {v} _ {f},
$$
$$
\hat {\lambda} _ {e} = \frac {1}{9} \left(\sqrt {(3 - 1 2 \frac {\tau_ {s}}{\tau_ {1}} - 1 6 \frac {\tau_ {c}}{\tau_ {1}}) ^ {2} + 1 0 8 (\frac {\tau_ {c}}{\tau_ {1}}) ^ {2}} - 2 4 \frac {\tau_ {s}}{\tau_ {1}} - 2 0 \frac {\tau_ {c}}{\tau_ {1}} + 6\right)
$$
$$
\hat {\lambda} _ {f} = \frac {1}{9} \left(- \sqrt {(3 - 1 2 \frac {\tau_ {s}}{\tau_ {1}} - 1 6 \frac {\tau_ {c}}{\tau_ {1}}) ^ {2} + 1 0 8 (\frac {\tau_ {c}}{\tau_ {1}}) ^ {2}} - 2 4 \frac {\tau_ {s}}{\tau_ {1}} - 2 0 \frac {\tau_ {c}}{\tau_ {1}} + 6\right).
$$
Now, we show that  $\hat{\lambda}_c > \hat{\lambda}_e$ . By  $\frac{\tau_c}{\tau_1} \ll 1$  and  $\frac{4}{9}\tau_c \leq \tau_s \leq \tau_c$
$$
\begin{array}{l} \hat {\lambda} _ {c} \geq \hat {\lambda} _ {e} \Leftrightarrow 3 + 2 4 \frac {\tau_ {s}}{\tau_ {1}} - 2 8 \frac {\tau_ {c}}{\tau_ {1}} \geq \sqrt {(3 - 1 2 \frac {\tau_ {s}}{\tau_ {1}} - 1 6 \frac {\tau_ {c}}{\tau_ {1}}) ^ {2} + 1 0 8 (\frac {\tau_ {c}}{\tau_ {1}}) ^ {2}} \\ \Leftrightarrow 3 6 (\frac {\tau_ {s}}{\tau_ {1}}) ^ {2} + 3 5 (\frac {\tau_ {c}}{\tau_ {1}}) ^ {2} - 1 4 4 \frac {\tau_ {s}}{\tau_ {1}} \frac {\tau_ {c}}{\tau_ {1}} + 1 8 \frac {\tau_ {s}}{\tau_ {1}} - 6 \frac {\tau_ {c}}{\tau_ {1}} \geq 0. \\ \end{array}
$$
Thus, we have  $1 = \hat{\lambda}_a = \hat{\lambda}_b > \hat{\lambda}_c > \hat{\lambda}_e > \hat{\lambda}_f > \hat{\lambda}_d = 0$ . Moreover, we also have
$$
\begin{array}{l} \hat {\lambda} _ {c} - \hat {\lambda} _ {e} = 1 - \frac {1 6}{3} \frac {\tau_ {c}}{\tau_ {1}} - \frac {1}{9} \left(\sqrt {(3 - 1 2 \frac {\tau_ {s}}{\tau_ {1}} - 1 6 \frac {\tau_ {c}}{\tau_ {1}}) ^ {2} + 1 0 8 (\frac {\tau_ {c}}{\tau_ {1}}) ^ {2}} - 2 4 \frac {\tau_ {s}}{\tau_ {1}} - 2 0 \frac {\tau_ {c}}{\tau_ {1}} + 6\right) \\ \geq \Omega \left(\frac {\tau_ {c}}{\tau_ {1}}\right). \\ \end{array}
$$
Let  $\hat{\lambda}_1 = \hat{\lambda}_a, \hat{\lambda}_2 = \hat{\lambda}_b, \hat{\lambda}_3 = \hat{\lambda}_c$ . Then, by Weyl's Theorem, for  $i \in \{1, 2, 3\}$ , we have
$$
\left| \lambda_ {i} - \hat {\lambda} _ {i} \right| \leq \left\| D ^ {- \frac {1}{2}} A D ^ {- \frac {1}{2}} - \widehat {D ^ {- \frac {1}{2}}} \widehat {A D ^ {- \frac {1}{2}}} \right\| _ {2} \leq O \left(\left(\frac {\tau_ {c}}{\tau_ {1}}\right) ^ {2}\right).
$$
By Davis-Kahan theorem, we have
$$
\| \sin (U, \hat {U}) \| _ {F} \leq \frac {O \left(\left(\frac {\tau_ {c}}{\tau_ {1}}\right) ^ {2}\right)}{\Omega \left(\frac {\tau_ {c}}{\tau_ {1}}\right)} \leq O \left(\frac {\tau_ {c}}{\tau_ {1}}\right).
$$
We finish the proof.
Theorem B.2. Recall  $\eta_uA^{(u)}$  is defined in Theorem B.1. Assume  $1\gg \frac{\tau_c}{\tau_1} >\frac{\tau_s}{\tau_1} >0$  and  $\tau_{1} + \tau_{c} + \tau_{s} = 1$ . Let  $\lambda_1^{(u)},\lambda_2^{(u)},\lambda_3^{(u)}$  and  $v_{1}^{(u)},v_{2}^{(u)},v_{3}^{(u)}$  be the largest three eigenvalues and their corresponding eigenvectors of  $D^{(u) - \frac{1}{2}}(\eta_uA^{(u)})D^{(u) - \frac{1}{2}}$ , which is the normalized adjacency matrix of  $\eta_uA^{(u)}$ . Let
$$
\hat {\lambda} _ {1} ^ {(u)} = 1, \hat {\lambda} _ {2} ^ {(u)} = 1, \hat {\lambda} _ {3} ^ {(u)} = 1 - 4 \frac {\tau_ {s}}{\tau_ {1}},
$$
$$
\hat {v} _ {1} ^ {(u)} = [ 0, 0, 0, 0, 1, 1 ],
$$
$$
\hat {v} _ {2} ^ {(u)} = [ 1, 1, 1, 1, 0, 0 ],
$$
$$
\hat {v} _ {3} ^ {(u)} = [ 1, - 1, 1, - 1, 0, 0 ].
$$
Let  $U^{(u)} = [v_1^{(u)}, v_2^{(u)}, v_3^{(u)}], \hat{U}^{(u)} = [\hat{v}_1^{(u)}, \hat{v}_2^{(u)}, \hat{v}_3^{(u)}]$ . Then, for  $i \in \{1, 2, 3\}$ , we have  $|\lambda_i^{(u)} - \hat{\lambda}_i^{(u)}| \leq O((\frac{\tau_c}{\tau_1})^2)$  and  $\| \sin(U^{(u)}, \hat{U}^{(u)}) \|_F \leq O(\frac{\tau_c^2}{\tau_1 (\tau_c - \tau_s)})$ .
Proof. Similar to the proof of Theorem B.1, up to error  $O\left(\left(\frac{\tau_{e}}{\tau_{1}}\right)^{2}\right)$ , we have the following equation,
$$
\widehat {\eta_ {u} A ^ {(u)}} = \tau_ {1} ^ {2} \left[ \begin{array}{c c c c c c} 1 & 2 \frac {\tau_ {s}}{\tau_ {1}} & 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 & 0 \\ 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 & 0 & 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 0 \\ 2 \frac {\tau_ {c}}{\tau_ {1}} & 0 & 1 & 2 \frac {\tau_ {s}}{\tau_ {1}} & 0 & 0 \\ 0 & 2 \frac {\tau_ {c}}{\tau_ {1}} & 2 \frac {\tau_ {s}}{\tau_ {1}} & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 2 & 2 \\ 0 & 0 & 0 & 0 & 2 & 2 \end{array} \right].
$$