SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Sketch for 'Necessity': The goal of this analysis is to nullify the possibility of having a minimizer that exhibits collapse (i.e.,  $\mathbf{H}$  which satisfies  $\mathbf{P1}$ ) for a graph that does not satisfy condition  $\mathbf{C}$ . We do so by analyzing the optimality conditions of the stationary points satisfying  $\mathbf{P1}$ , and obtaining conditions on the  $\widehat{\mathbf{A}}$  matrix. Specifically, we obtain a system of linear equations that is shared by all  $n$  nodes within a class, which in turn leads to condition  $\mathbf{C}$ . Thus, proving its necessity.
Necessity: Analysing the necessity of condition  $\mathbf{C}$  is relatively more complicated than the sufficiency case. Nonetheless, we prove the necessity by considering  $K = 1$ , and by leveraging the idea of a tight-convex alternative for  $\widehat{\mathcal{R}} (\mathbf{W}_2,\mathbf{H})$  as been used in [55, 74] for the conventional UFMs<sup>10</sup>. Formally, assuming  $d_{L - 1}\geq C$ , we minimize:
$$
\widehat {\mathcal {R}} (\mathbf {Z}) := \frac {1}{2 N} \| \mathbf {Z} \widehat {\mathbf {A}} - \mathbf {Y} \| _ {F} ^ {2} + \lambda_ {Z} \| \mathbf {Z} \| _ {*}, \tag {39}
$$
where  $\lambda_{Z} = \sqrt{\lambda_{H}\lambda_{W_{2}}}$ ,  $\mathbf{Z} \in \mathbb{R}^{C \times N}$ , and  $\| \cdot \|_{*}$  denotes the nuclear norm. Namely, if  $\mathbf{W}_{2}$  and  $\mathbf{H}$  minimize the former, then  $\mathbf{Z} = \mathbf{W}_{2}\mathbf{H}$  minimizes the latter, which follows from:
$$
\sqrt {\lambda_ {H} \lambda_ {W _ {2}}} \| \mathbf {Z} \| _ {*} = \min  _ {\mathbf {W} _ {2}, \mathbf {H} s. t. \mathbf {W} _ {2} \mathbf {H} = \mathbf {Z}} \left(\frac {\lambda_ {W _ {2}}}{2} \| \mathbf {W} _ {2} \| _ {F} ^ {2} + \frac {\lambda_ {H}}{2} \| \mathbf {H} \| _ {F} ^ {2}\right). \tag {40}
$$
We start with providing necessity analysis for the case  $\lambda_Z = 0$ . Later, we generalize this analysis to address the case  $\lambda_Z > 0$ .
Analysis for  $\lambda_{Z} = 0$ : When  $\lambda_{Z} = 0$ , observe that  $\mathbf{Z}\widehat{\mathbf{A}} = \mathbf{Y} = [\mathbf{y}_1,\dots ,\mathbf{y}_C]\otimes \mathbf{1}_n^\top = \mathbf{I}_C\otimes \mathbf{1}_n^\top$  gives us the minimum value for  $\widehat{\mathcal{R}} (\mathbf{Z})$ . Here  $\mathbf{y}_c\in \mathbb{R}^C$  represent the one-hot label vectors corresponding to nodes of class  $c\in [C]$ .
Now, note that NC1 implies  $\mathbf{H} = \overline{\mathbf{H}}\otimes \mathbf{1}_n^\top \iff \mathbf{Z} = \overline{\mathbf{Z}}\otimes \mathbf{1}_n^\top$  (where  $\overline{\mathbf{H}}\in \mathbb{R}^{d_{L - 1}\times C}$  and  $\overline{\mathbf{Z}} = \mathbf{W}_2\overline{\mathbf{H}}\in \mathbb{R}^{C\times C}$ ). Thus, by leveraging this Kronecker structure of  $\mathbf{Z} = \overline{\mathbf{Z}}\otimes \mathbf{1}_n^\top = [\overline{\mathbf{z}}_1,\dots ,\overline{\mathbf{z}}_C]\otimes \mathbf{1}_n^\top$  and  $\mathbf{Y} = [\mathbf{y}_1,\ldots ,\mathbf{y}_C]\otimes \mathbf{1}_n^\top$ , we formulate:
$$
\begin{array}{l} \left[ \begin{array}{l l l} \overline {{\mathbf {z}}} _ {1} \otimes \mathbf {1} _ {n} ^ {\top} & \dots & \overline {{\mathbf {z}}} _ {C} \otimes \mathbf {1} _ {n} ^ {\top} \end{array} \right] \left[ \begin{array}{c c c c c c c} \mathbf {a} _ {1, 1} ^ {1} & \dots & \mathbf {a} _ {1, 1} ^ {n} & \dots & \mathbf {a} _ {C, 1} ^ {1} & \dots & \mathbf {a} _ {C, 1} ^ {n} \\ \hline \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ \hline \mathbf {a} _ {1, C} ^ {1} & \dots & \mathbf {a} _ {1, C} ^ {n} & \dots & \mathbf {a} _ {C, C} ^ {1} & \dots & \mathbf {a} _ {C, C} ^ {n} \end{array} \right] \tag {41} \\ = \left[ \begin{array}{c c c} \mathbf {y} _ {1} \otimes \mathbf {1} _ {n} ^ {\top} & \dots & \mathbf {y} _ {C} \otimes \mathbf {1} _ {n} ^ {\top} \end{array} \right] \\ \end{array}
$$
where  $\mathbf{a}_{c,c'}^i\in \mathbb{R}^n$  represents the slice of columns in  $\widehat{\mathbf{A}}$  corresponding to node  $i\in [n]$  belonging to class  $c\in [C]$  and forming edges with nodes from class  $c'\in [C]$ . Additionally, since  $\widehat{\mathbf{A}}$  is the
Note that the tuple can be different for nodes belonging to different classes, but must be the same for nodes within the same class.
10The  $K = 1$  setting allows us to employ analysis strategies based on a tight convex formulation, which is not applicable for  $K > 1$  settings. Specifically, we cannot generalize the problem stated for  $\mathbf{Z} = \mathbf{W}_2\mathbf{H}$  to multiple  $\{\mathbf{H}_k\}$  as they share the same  $\mathbf{W}_2$ .
normalized adjacency matrix, we have  $s_{cc',i} = \mathbf{1}_n^\top \mathbf{a}_{c,c'}^i \in \mathbb{R}$ , which represents the sum of elements in  $\mathbf{a}_{c,c'}^i$ . This gives us:
$$
\sum_ {c ^ {\prime} = 1} ^ {C} s _ {c c ^ {\prime}, i} = 1, \tag {42}
$$
because  $\widehat{\mathbf{A}} = \mathbf{AD}^{-1}$  is the degree-normalized adjacency matrix. Now, multiplying the matrix  $\mathbf{Z} = \overline{\mathbf{Z}}\otimes \mathbf{1}_n^\top$  with the first column of  $\widehat{\mathbf{A}}$  gives us:
$$
s _ {1 1, 1} \overline {{\mathbf {z}}} _ {1} + \dots + s _ {1 C, 1} \overline {{\mathbf {z}}} _ {C} = \mathbf {y} _ {1} \tag {43}
$$
Similarly, due to the block structure of  $\mathbf{Z}$ ,  $\mathbf{Y}$ , we get for all  $i \in [n]$  and  $c = 1$ :
$$
s _ {1 1, i} \overline {{\mathbf {z}}} _ {1} + \dots + s _ {1 C, i} \overline {{\mathbf {z}}} _ {C} = \mathbf {y} _ {1} \tag {44}
$$
where  $s_{11,i}\overline{\mathbf{z}}_1 + \dots + s_{1C,i}\overline{\mathbf{z}}_C = \mathbf{y}_1$  itself can be written as  $C$  linear equations (one for each of the vector components) as formulated below:
$$
\left[ \begin{array}{c c c} \bar {z} _ {1, 1} & \dots & \bar {z} _ {C, 1} \\ \vdots & \ddots & \vdots \\ \bar {z} _ {1, C} & \dots & \bar {z} _ {C, C} \end{array} \right] \left[ \begin{array}{c} s _ {1 1, i} \\ \vdots \\ s _ {1 C, i} \end{array} \right] = \mathbf {y} _ {1} \tag {45}
$$
Now, by treating  $\{s_{11,i},\dots ,s_{1C,i}\}$  as the  $C$  unknowns which satisfy equation 42, observe that the solution to this linear system remains the same for all nodes belonging to class  $c = 1$ . This implies:
$$
s _ {1 c ^ {\prime}, 1} = \dots = s _ {1 c ^ {\prime}, n}, \quad \forall c ^ {\prime} \in [ C ], \tag {46}
$$
The generalization of this result for all  $C$  classes essentially indicates that:
$$
\left(s _ {c 1, 1}, \dots , s _ {c C, 1}\right) = \dots = \left(s _ {c 1, n}, \dots , s _ {c C, n}\right), \quad \forall c \in [ C ], \tag {47}
$$
which exactly represents condition  $\mathbf{C}$  as stated above.
Analysis for  $\lambda_Z > 0$ : When  $\mathbf{H}$  exhibits neural collapse, we have the optimality condition for the minimizer of equation 39 based on the sub-differential of the nuclear norm as follows:
$$
\begin{array}{l} \frac {1}{N} \left(\mathbf {Z} \hat {\mathbf {A}} - \mathbf {Y}\right) \hat {\mathbf {A}} ^ {\top} + \lambda_ {Z} \mathbf {U} _ {Z} \mathbf {V} _ {Z} ^ {\top} = \mathbf {0} \\ \Rightarrow \frac {1}{N} \left(\left(\overline {{\mathbf {Z}}} \otimes \mathbf {1} _ {n} ^ {\top}\right) \widehat {\mathbf {A}} - \mathbf {I} _ {C} \otimes \mathbf {1} _ {n} ^ {\top}\right) \widehat {\mathbf {A}} ^ {\top} + \lambda_ {Z} \mathbf {U} _ {Z} \mathbf {V} _ {\bar {Z}} ^ {\top} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} ^ {\top} = \mathbf {0}. \tag {48} \\ \end{array}
$$
Where  $\overline{\mathbf{Z}}\in \mathbb{R}^{C\times C}$  is the matrix of "collapsed" columns of  $\mathbf{Z}$ , and  $\mathbf{U}_Z\in \mathbb{R}^{C\times C},\mathbf{V}_Z\in \mathbb{R}^{N\times C}$  represent the left and right singular vectors of  $\mathbf{Z}$ . Additionally, since  $\mathbf{Z}$  holds a "block" structure, we represent  $\mathbf{V}_Z^\top = \mathbf{V}_Z^\top \otimes \frac{1}{\sqrt{n}}\mathbf{1}_n^\top$ , where  $\mathbf{V}_{\overline{Z}}\in \mathbb{R}^{C\times C}$ .
- Matrix Quadratic Form: By considering  $-N\lambda_Z\mathbf{U}_Z\mathbf{V}_{\overline{Z}}^\top \otimes \frac{1}{\sqrt{n}}\mathbf{1}_n^\top = \mathbf{B}$ , we get:
$$
(\mathbf {Z} \widehat {\mathbf {A}} - \mathbf {Y}) \widehat {\mathbf {A}} ^ {\top} = \mathbf {B}, \tag {49}
$$
as our optimality condition. Analyzing this condition along the same lines as  $\lambda_{Z} = 0$  is non-trivial due to the outer product of  $\widehat{\mathbf{A}}\widehat{\mathbf{A}}^{\top}$ . To address this complication, we leverage the fact that SSBM graphs tend to be regular with a high probability as  $N$  increases. Thus, by assuming that  $\widehat{\mathbf{A}}$  is symmetric, we obtain the following matrix quadratic form:
$$
\mathbf {Z} \widehat {\mathbf {A}} ^ {2} - \mathbf {Y} \widehat {\mathbf {A}} - \mathbf {B} = \mathbf {0}. \tag {50}
$$
As  $\mathbf{Z}$  is rectangular, we can take the pseudo-inverse and obtain:
$$
\widehat {\mathbf {A}} ^ {2} - \mathbf {Z} ^ {\dagger} \mathbf {Y} \widehat {\mathbf {A}} - \mathbf {Z} ^ {\dagger} \mathbf {B} = \mathbf {0}. \tag {51}
$$
Since  $\mathbf{Z}^{\dagger}\mathbf{Y},\mathbf{Z}^{\dagger}\mathbf{B}\in \mathbb{R}^{N\times N}$ , we can treat  $\widehat{\mathbf{A}}$  as the variable matrix and leverage the results on quadratic matrix equations by Higham and Kim [27]. Especially, we leverage theorem 3 in Higham and Kim [27] and employ a generalized Schur decomposition technique to obtain a condition on  $\widehat{\mathbf{A}}$  (as shown in the following lemma). This condition on  $\widehat{\mathbf{A}}$  allows us to obtain a system of linear equations that are shared by all  $n$  nodes within a class (similar to the  $\lambda_{Z} = 0$  case). Thus, establishing the necessity of condition  $\mathbf{C}$
Lemma D.1. Let  $\mathbf{F} = \left[ \begin{array}{cc}\mathbf{0} & \mathbf{I}\\ \mathbf{Z}^{\dagger}\mathbf{B} & \mathbf{Z}^{\dagger}\mathbf{Y} \end{array} \right],\mathbf{G} = \left[ \begin{array}{cc}\mathbf{I} & \mathbf{0}\\ \mathbf{0} & \mathbf{I} \end{array} \right] = \mathbf{I}\in \mathbb{R}^{2N\times 2N}$ , and the generalized Schur decomposition of  $\mathbf{F},\mathbf{G}$  be given by:
$$
\mathbf {Q} ^ {\top} \mathbf {F} \mathbf {K} = \mathbf {T}, \quad \mathbf {Q} ^ {\top} \mathbf {G} \mathbf {K} = \mathbf {E},
$$
where  $\mathbf{Q},\mathbf{K}$  are unitary and  $\mathbf{T},\mathbf{E}$  are upper triangular.  $\mathbf{K} = \begin{bmatrix} \mathbf{K}_{11} & \mathbf{K}_{12}\\ \mathbf{K}_{21} & \mathbf{K}_{22} \end{bmatrix} \in \mathbb{R}^{2N\times 2N}$  is a  $2\times 2$  block matrix with blocks of size  $N\times N$ . Then  $\widehat{\mathbf{A}}$  satisfies:  $\mathbf{K}_{11}\widehat{\mathbf{A}} = \mathbf{K}_{21}$ .
Proof. The proof is relatively straightforward once we observe that  $\hat{\mathbf{A}}$  satisfies:
$$
\mathbf {F} \left[ \begin{array}{l} \mathbf {I} \\ \widehat {\mathbf {A}} \end{array} \right] = \mathbf {G} \left[ \begin{array}{l} \mathbf {I} \\ \widehat {\mathbf {A}} \end{array} \right] \widehat {\mathbf {A}}. \tag {52}
$$
Now, the condition  $\mathbf{K}_{11}\widehat{\mathbf{A}} = \mathbf{K}_{21}$  is a direct consequence of theorem 3 in [27], which leverages the QR decomposition of  $\begin{bmatrix} \mathbf{I} \\ \widehat{\mathbf{A}} \end{bmatrix}$  using  $\mathbf{K}$  as the orthogonal matrix.
- Kronecker structure of  $\mathbf{F}$ : To leverage the relationship between  $\widehat{\mathbf{A}}$  and  $\mathbf{K}$  as per Lemma D.1, a closer look at  $\mathbf{F}$  is required. Observe that:
$$
\mathbf {F} = \left[ \begin{array}{l l} \mathbf {0} & \mathbf {I} \\ \mathbf {Z} ^ {\dagger} \mathbf {B} & \mathbf {Z} ^ {\dagger} \mathbf {Y} \end{array} \right] = \left[ \begin{array}{l l} \mathbf {Z} ^ {\dagger} \mathbf {0} & \mathbf {Z} ^ {\dagger} \mathbf {Z} \\ \mathbf {Z} ^ {\dagger} \mathbf {B} & \mathbf {Z} ^ {\dagger} \mathbf {Y} \end{array} \right] = \left[ \begin{array}{l l} \mathbf {Z} ^ {\dagger} & \mathbf {0} \\ \mathbf {0} & \mathbf {Z} ^ {\dagger} \end{array} \right] \left[ \begin{array}{l l} \mathbf {0} & \mathbf {Z} \\ \mathbf {B} & \mathbf {Y} \end{array} \right]. \tag {53}
$$
By expanding the Kronecker structures of  $\mathbf{Z}$ ,  $\mathbf{B}$ ,  $\mathbf{Y}$ , we get:
$$
\begin{array}{l} \mathbf {F} = \left[ \begin{array}{c c} \mathbf {Z} ^ {\dagger} & \mathbf {0} \\ \mathbf {0} & \mathbf {Z} ^ {\dagger} \end{array} \right] \left[ \begin{array}{c c} \mathbf {0} \otimes \mathbf {1} _ {n} ^ {\top} & \overline {{\mathbf {Z}}} \otimes \mathbf {1} _ {n} ^ {\top} \\ \overline {{\mathbf {B}}} \otimes \mathbf {1} _ {n} ^ {\top} & \mathbf {I} _ {C} \otimes \mathbf {1} _ {n} ^ {\top} \end{array} \right] \\ = \left[ \begin{array}{c c c c c c} \mathbf {0} & \dots & 0 & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {z}}} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {z}}} _ {C} \\ \mathbf {Z} ^ {\dagger} \overline {{\mathbf {b}}} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {b}}} _ {C} & \mathbf {Z} ^ {\dagger} \mathbf {e} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \mathbf {e} _ {C} \end{array} \right] \otimes \mathbf {1} _ {n} ^ {\top}. \tag {54} \\ \end{array}
$$
Observe that the pseudo-inverse  $\mathbf{Z}^{\dagger}$  can be represented by:
$$
\mathbf {Z} ^ {\dagger} = \left(\mathbf {V} _ {\bar {Z}} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n}\right) \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \tag {55}
$$
which also holds a "block" structure (but with respect to rows, instead of columns):
$$
\mathbf {Z} ^ {\dagger} = \frac {1}{\sqrt {n}} \left[ \begin{array}{c} \mathbf {v} _ {1} ^ {\top} \\ \vdots \\ \mathbf {v} _ {C} ^ {\top} \end{array} \right] \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} = \frac {1}{\sqrt {n}} \left[ \begin{array}{c} \mathbf {v} _ {1} ^ {\top} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \\ \vdots \\ \mathbf {v} _ {C} ^ {\top} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \end{array} \right] _ {N \times C}. \tag {56}
$$
Where  $\mathbf{v}_j\in \mathbb{R}^C,\forall j\in [C]$  and  $\mathbf{v}_j^\top$  is the  $j^{th}$  row of  $\mathbf{V}_{\overline{Z}}\in \mathbb{R}^{C\times C}$ . With this formulation, a matrix-vector product term in  $\mathbf{F}$ , for instance  $\mathbf{Z}^{\dagger}\overline{\mathbf{b}}_i,i\in [C]$ , can be given as:
$$
\mathbf {Z} ^ {\dagger} \overline {{\mathbf {b}}} _ {i} = \frac {1}{\sqrt {n}} \left[ \begin{array}{c} \mathbf {v} _ {1} ^ {\top} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \overline {{\mathbf {b}}} _ {i} \\ \vdots \\ \mathbf {v} _ {C} ^ {\top} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \overline {{\mathbf {b}}} _ {i} \end{array} \right] _ {N \times 1} = \left(\mathbf {V} _ {\bar {Z}} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \overline {{\mathbf {b}}} _ {i}\right) \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n}. \tag {57}
$$
By considering the following notational simplifications:
$$
\begin{array}{l} \widetilde {\mathbf {b}} _ {i} = \mathbf {V} _ {\overline {{Z}}} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \overline {{\mathbf {b}}} _ {i} \\ \widetilde {\mathbf {z}} _ {i} = \mathbf {V} _ {\bar {Z}} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \bar {\mathbf {z}} _ {i} \tag {58} \\ \widetilde {\mathbf {e}} _ {i} = \mathbf {V} _ {\overline {{Z}}} \mathbf {S} _ {Z} ^ {\dagger} \mathbf {U} _ {Z} ^ {\top} \mathbf {e} _ {i}, \\ \end{array}
$$
we can represent  $\mathbf{F}$  as:
$$
\begin{array}{l} \mathbf {F} = \left[ \begin{array}{c c c c c c} \mathbf {0} & \dots & \mathbf {0} & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {z}}} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {z}}} _ {C} \\ \mathbf {Z} ^ {\dagger} \overline {{\mathbf {b}}} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \overline {{\mathbf {b}}} _ {C} & \mathbf {Z} ^ {\dagger} \mathbf {e} _ {1} & \dots & \mathbf {Z} ^ {\dagger} \mathbf {e} _ {C} \end{array} \right] \otimes \mathbf {1} _ {n} ^ {\top} \\ = \left[ \begin{array}{c c c c c c} \mathbf {0} & \dots & \mathbf {0} & \widetilde {\mathbf {z}} _ {1} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} & \dots & \widetilde {\mathbf {z}} _ {C} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \\ \widetilde {\mathbf {b}} _ {1} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} & \dots & \widetilde {\mathbf {b}} _ {C} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} & \widetilde {\mathbf {e}} _ {1} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} & \dots & \widetilde {\mathbf {e}} _ {1} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \end{array} \right] \otimes \mathbf {1} _ {n} ^ {\top} \tag {59} \\ = \left[ \begin{array}{c c c c c c} \mathbf {0} & \dots & \mathbf {0} & \widetilde {\mathbf {z}} _ {1} & \dots & \widetilde {\mathbf {z}} _ {C} \\ \widetilde {\mathbf {b}} _ {1} & \dots & \widetilde {\mathbf {b}} _ {C} & \widetilde {\mathbf {e}} _ {1} & \dots & \widetilde {\mathbf {e}} _ {1} \end{array} \right] _ {2 C \times 2 C} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \otimes \mathbf {1} _ {n} ^ {\top}. \\ \end{array}
$$
- Schur decomposition of  $\mathbf{F}$ : For notational simplicity, let:
$$
\mathbf {F} = \widetilde {\mathbf {F}} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \otimes \mathbf {1} _ {n} ^ {\top}
$$
$$
\text {W h e r e :} \quad \widetilde {\mathbf {F}} = \left[ \begin{array}{l l l l l l} \mathbf {0} & \dots & \mathbf {0} & \widetilde {\mathbf {z}} _ {1} & \dots & \widetilde {\mathbf {z}} _ {C} \\ \widetilde {\mathbf {b}} _ {1} & \dots & \widetilde {\mathbf {b}} _ {C} & \widetilde {\mathbf {e}} _ {1} & \dots & \widetilde {\mathbf {e}} _ {1} \end{array} \right] _ {2 C \times 2 C}. \tag {60}
$$
Since  $\mathbf{G} = \mathbf{I}$ , the diagonal entries of  $\mathbf{S}$  must equal the eigenvalues of  $\mathbf{I}$ . This condition is satisfied when  $\mathbf{Q} = \mathbf{K}$ . This also simplifies the generalized Schur decomposition for square matrices  $\mathbf{F}, \mathbf{G}$  to the standard Schur decomposition of  $\mathbf{F}$ . Now, let the schur decomposition of  $\mathbf{F} \in \mathbb{R}^{2N \times 2N}$ ,  $\widetilde{\mathbf{F}} \in \mathbb{R}^{2C \times 2C}$  be given as:
$$
\mathbf {F} = \mathbf {K T K} ^ {\top}, \widetilde {\mathbf {F}} = \widetilde {\mathbf {K}} \widetilde {\mathbf {T}} \widetilde {\mathbf {K}} ^ {\top}. \tag {61}
$$
Here  $\mathbf{K},\mathbf{T}\in \mathbb{R}^{2N\times 2N}$  and  $\widetilde{\mathbf{K}},\widetilde{\mathbf{T}}\in \mathbb{R}^{2C\times 2C}$ . To find a relation between  $\mathbf{K},\mathbf{T},\widetilde{\mathbf{K}},\widetilde{\mathbf{T}}$ , we can leverage the Schur decomposition properties of Kronecker products [53, 59] and obtain:
$$
\mathbf {F} = \widetilde {\mathbf {F}} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \otimes \mathbf {1} _ {n} ^ {\top} \tag {62}
$$
$$
= \widetilde {\mathbf {F}} \otimes \frac {1}{\sqrt {n}} \mathbf {1} _ {n} \mathbf {1} _ {n} ^ {\top} \tag {63}
$$
$$
\mathbf {K T K} ^ {\top} = \left(\widetilde {\mathbf {K}} \widetilde {\mathbf {T}} \widetilde {\mathbf {K}} ^ {\top}\right) \otimes \left(\mathbf {J O J} ^ {\top}\right) \tag {64}
$$
$$
= (\widetilde {\mathbf {K}} \otimes \mathbf {J}) (\widetilde {\mathbf {T}} \otimes \mathbf {O}) (\widetilde {\mathbf {K}} ^ {\top} \otimes \mathbf {J} ^ {\top}). \tag {65}
$$
Where  $\mathbf{J}$ ,  $\mathbf{O} \in \mathbb{R}^{n \times n}$  are unitary and upper triangular respectively, and are the Schur decomposition factors of  $\frac{1}{\sqrt{n}} \mathbf{1}_n \mathbf{1}_n^\top \in \mathbb{R}^{n \times n}$ .
- Linear systems: In matrix form,  $\mathbf{K} = \widetilde{\mathbf{K}}\otimes \mathbf{J}$  can be represented as:
$$
\mathbf {K} = \left[ \begin{array}{l l} \widetilde {\mathbf {K}} _ {1 1} \otimes \mathbf {J} & \widetilde {\mathbf {K}} _ {1 2} \otimes \mathbf {J} \\ \widetilde {\mathbf {K}} _ {2 1} \otimes \mathbf {J} & \widetilde {\mathbf {K}} _ {2 2} \otimes \mathbf {J} \end{array} \right] _ {2 N \times 2 N}. \tag {66}
$$
Where  $\widetilde{\mathbf{K}}_{11},\widetilde{\mathbf{K}}_{12},\widetilde{\mathbf{K}}_{21},\widetilde{\mathbf{K}}_{22}\in \mathbb{R}^{C\times C}$ . Now, observe that  $\mathbf{K}_{11}\widehat{\mathbf{A}} = \mathbf{K}_{21}$  (based on Lemma D.1) can be reformulated as:
$$
\left(\widetilde {\mathbf {K}} _ {1 1} \otimes \mathbf {J}\right) \widehat {\mathbf {A}} = \left(\widetilde {\mathbf {K}} _ {2 1} \otimes \mathbf {J}\right). \tag {67}
$$
Now, as per equation 41, we leverage the same line of analysis that we followed for the  $\lambda_Z = 0$  case. For notational simplicity, we represent the unitary matrix  $\mathbf{J} \in \mathbb{R}^{n \times n}$  in column format as follows:
$$
\mathbf {J} = \left[ \begin{array}{l l l} \mathbf {j} _ {1} & \dots & \mathbf {j} _ {n} \end{array} \right], \tag {68}
$$
where  $\mathbf{j}_i\in \mathbb{R}^n,i\in [n]$  are linearly independent vectors. Now, by multiplying the first row of  $\widetilde{\mathbf{K}}_{11}\otimes \mathbf{J}$  and the first column of  $\widehat{\mathbf{A}}$  , we get:
$$
\left[ \begin{array}{c c c c c} (\widetilde {\mathbf {K}} _ {1 1}) _ {1, 1}   [ \mathbf {j} _ {1} & \dots & \mathbf {j} _ {n} ] & \dots & (\widetilde {\mathbf {K}} _ {1 1}) _ {1, C}   [ \mathbf {j} _ {1} & \dots & \mathbf {j} _ {n} ] \end{array} \right] \left[ \begin{array}{c} a _ {1, 1} \\ \vdots \\ a _ {n, 1} \\ \hline \vdots \\ \hline a _ {(C - 1) n + 1, 1} \\ \vdots \\ a _ {C n, 1} \end{array} \right] = (\widetilde {\mathbf {K}} _ {2 1}) _ {1, 1} \mathbf {j} _ {1}.
$$
This translates to the following linear equation:
$$
\begin{array}{l} a _ {1, 1} (\tilde {\mathbf {K}} _ {1 1}) _ {1, 1} \mathbf {j} _ {1} + \dots + a _ {n, 1} (\tilde {\mathbf {K}} _ {1 1}) _ {1, 1} \mathbf {j} _ {n} + \dots \\ + a _ {(C - 1) n + 1, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, C} \mathbf {j} _ {1} + \dots + a _ {C n, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, C} \mathbf {j} _ {n} = \left(\widetilde {\mathbf {K}} _ {2 1}\right) _ {1, 1} \mathbf {j} _ {1}. \tag {69} \\ \end{array}
$$
Due to linear independence of vectors  $\mathbf{j}_i, i \in [n]$ , we obtain the following  $n$  equations pertaining to the coefficients of  $\mathbf{j}_i$ :
$$
a _ {1, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, 1} + \dots + a _ {(C - 1) n + 1, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, C} = \left(\widetilde {\mathbf {K}} _ {2 1}\right) _ {1, 1}
$$
$$
a _ {2, 1} (\widetilde {\mathbf {K}} _ {1 1}) _ {1, 1} + \dots + a _ {(C - 1) n + 2, 1} (\widetilde {\mathbf {K}} _ {1 1}) _ {1, C} = 0
$$
$$
a _ {n, 1} (\tilde {\mathbf {K}} _ {1 1}) _ {1, 1} + \dots + a _ {C n, 1} (\tilde {\mathbf {K}} _ {1 1}) _ {1, C} = 0.
$$
By adding all these equations, we get:
$$
s _ {1 1, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, 1} + \dots + s _ {1 C, 1} \left(\widetilde {\mathbf {K}} _ {1 1}\right) _ {1, C} = \left(\widetilde {\mathbf {K}} _ {2 1}\right) _ {1, 1}. \tag {70}
$$
By following the same approach for the other rows of  $\widetilde{\mathbf{K}}_{11} \otimes \mathbf{K}_{1n}$  and the first column of  $\widehat{\mathbf{A}}$ , we get the following system of equations for node  $v_{1,1}$ :
$$
\left[ \begin{array}{c c c} (\widetilde {\mathbf {K}} _ {1 1}) _ {1, 1} & \dots & (\widetilde {\mathbf {K}} _ {1 1}) _ {1, C} \\ \vdots & \ddots & \vdots \\ (\widetilde {\mathbf {K}} _ {1 1}) _ {C, 1} & \dots & (\widetilde {\mathbf {K}} _ {1 1}) _ {C, C} \end{array} \right] \left[ \begin{array}{c} s _ {1 1, 1} \\ \vdots \\ s _ {1 C, 1} \end{array} \right] = \left[ \begin{array}{c} (\widetilde {\mathbf {K}} _ {2 1}) _ {1, 1} \\ \vdots \\ (\widetilde {\mathbf {K}} _ {2 1}) _ {C, 1} \end{array} \right]. \tag {71}
$$
The same line of analysis can be applied for all the rows of  $\widetilde{\mathbf{K}}_{11} \otimes \mathbf{K}_{1_n}$  and the second column of  $\widehat{\mathbf{A}}$ , to get the following system of equations for node  $v_{1,2}$ :
$$
\left[ \begin{array}{c c c} (\widetilde {\mathbf {K}} _ {1 1}) _ {1, 1} & \dots & (\widetilde {\mathbf {K}} _ {1 1}) _ {1, C} \\ \vdots & \ddots & \vdots \\ (\widetilde {\mathbf {K}} _ {1 1}) _ {C, 1} & \dots & (\widetilde {\mathbf {K}} _ {1 1}) _ {C, C} \end{array} \right] \left[ \begin{array}{c} s _ {1 1, 2} \\ \vdots \\ s _ {1 C, 2} \end{array} \right] = \left[ \begin{array}{c} (\widetilde {\mathbf {K}} _ {2 1}) _ {1, 1} \\ \vdots \\ (\widetilde {\mathbf {K}} _ {2 1}) _ {C, 1} \end{array} \right] \tag {72}
$$