SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Emmanuel Abbe, Jianqing Fan, Kaizheng Wang, and Yiqiao Zhong. Entrywise eigenvector analysis of random matrices with low expected rank. Annals of Statistics, 48(3):1452-1474, 2020.
[2] Boris Alexeev, Afonso S Bandeira, Matthew Fickus, and Dustin G Mixon. Phase retrieval with polarization. SIAM Journal on Imaging Sciences, 7(1):35-66, 2014.
[3] Mica Arie-Nachimson, Shahar Z Kovalsky, Ira Kemelmacher-Shlizerman, Amit Singer, and Ronen Basri. Global motion estimation from point matches. In 2012 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, pages 81-88. IEEE, 2012.
[4] Afonso S Bandeira. Random Laplacian matrices and convex relaxations. Foundations of Computational Mathematics, 18(2):345-379, 2018.
[5] Afonso S Bandeira, Nicolas Boumal, and Amit Singer. Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. Mathematical Programming, 163(1-2):145-167, 2017.
[6] Afonso S Bandeira, Amit Singer, and Daniel A Spielman. A Cheeger inequality for the graph connection Laplacian. SIAM Journal on Matrix Analysis and Applications, 34(4):1611-1630, 2013.
[7] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. Manopt, a Matlab toolbox for optimization on manifolds. Journal of Machine Learning Research, 15(1):1455-1459, 2014.
[8] James V Burke and Michael C Ferris. Weak sharp minima in mathematical programming. SIAM Journal on Control and Optimization, 31(5):1340-1359, 1993.
[9] Avishek Chatterjee and Venu Madhav Govindu. Robust relative rotation averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):958-972, 2017.
[10] Yuxin Chen, Jianqing Fan, Cong Ma, and Kaizheng Wang. Spectral method and regularized MLE are both optimal for top- $K$  ranking. Annals of Statistics, 47(4):2204-2235, 2019.
[11] Yuxin Chen and Andrea J Goldsmith. Information recovery from pairwise measurements. In 2014 IEEE International Symposium on Information Theory, pages 2012-2016. IEEE, 2014.
[12] Mihai Cucuringu, Yaron Lipman, and Amit Singer. Sensor network localization by eigenvector synchronization over the Euclidean group. ACM Transactions on Sensor Networks, 8(3):1-42, 2012.
[13] Chandler Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III. SIAM Journal on Numerical Analysis, 7(1):1-46, 1970.
[14] Damek Davis, Dmitriy Drusvyatskiy, Kellie J MacPhee, and Courtney Paquete. Subgradient methods for sharp weakly convex functions. Journal of Optimization Theory and Applications, 179(3):962-982, 2018.
[33] Yoel Shkolnisky and Amit Singer. Viewing direction estimation in cryo-EM using synchronization. SIAM Journal on Imaging Sciences, 5(3):1088–1110, 2012.
[34] Amit Singer. Angular synchronization by eigenvectors and semidefinite programming. Applied and Computational Harmonic Analysis, 30(1):20-36, 2011.
[35] Amit Singer, Zhizhen Zhao, Yoel Shkolnisky, and Ronny Hadani. Viewing angle classification of cryo-electron microscopy images using eigenvectors. SIAM Journal on Imaging Sciences, 4(2):723-759, 2011.
[36] Anthony Man-Cho So. Probabilistic analysis of the semidefinite relaxation detector in digital communications. In Proceedings of the 21st annual ACM-SIAM Symposium on Discrete Algorithms, pages 698-711. SIAM, 2010.
[37] Gilbert W Stewart and Ji-guang Sun. Matrix Perturbation Theory. Academic Press, 1990.
[38] Joel A Tropp. An Introduction to Matrix Concentration Inequalities. Foundations and Trends® in Machine Learning, 8(1-2):1-230, 2015.
[39] Lanhui Wang and Amit Singer. Exact and stable recovery of rotations for robust synchronization. Information and Inference: A Journal of the IMA, 2(2):145-193, 2013.
[40] Wei Hong Yang, Lei-Hong Zhang, and Ruyi Song. Optimality conditions for the nonlinear programming problems on Riemannian manifolds. Pacific Journal of Optimization, 10(2):415-434, 2014.
[41] Stella Yu. Angular embedding: A robust quadratic criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1):158-173, 2011.
[42] Yiqiao Zhong and Nicolas Boumal. Near-optimal bounds for phase synchronization. SIAM Journal on Optimization, 28(2):989-1016, 2018.
1 Introduction 1
2 Algorithm and Setup 3
2.1 ReSync: Algorithm Development 3
2.2 RCM Setup for Theoretical Analysis 4
3.1 Analysis of SpectrIn with Leave-One-Out Technique 5
3.2 Weak Sharpness and Exact Recovery 6
3.3 Convergence Analysis and Proof of Theorem 1 8
4.1 Synthetic Data 8
4.2 Real Data 10
5 Conclusion and Discussions on Limitations 10
A Full Proof of Theorem 2 15
A.1 Initialization Error in Euclidean Norm 16
A.2 Initialization Error in Infinity Norm 17
B Full Proof of Theorem 3 18
B.1 Proof of Lemma 1 18
B.2 Proof of Proposition 1 18
B.3 Proof of Proposition 2 19
C Full Proof of Theorem 4 21
C.1 Proof of Corollary 1 21
C.2 Proof of Contraction 22
A Full Proof of Theorem 2
In this section, we present the full proof of Theorem 2. We will use the notation defined in the proof outline of Theorem 2 in Section 3.1.
Firstly, we know that
$$
\boldsymbol {W} _ {i j} = \left\{ \begin{array}{l l} (1 - p q) \boldsymbol {X} _ {i} ^ {\star} \boldsymbol {X} _ {j} ^ {\star \top}, & \text {w i t h p r o b a b l i t y} p q, \\ \boldsymbol {O} _ {i j} - p q \boldsymbol {X} _ {i} ^ {\star} \boldsymbol {X} _ {j} ^ {\star \top}, & \text {w i t h p r o b a b l i t y} (1 - p) q, \\ - p q \boldsymbol {X} _ {i} ^ {\star} \boldsymbol {X} _ {j} ^ {\star \top}, & \text {o t h e r w i s e .} \end{array} \right. \tag {17}
$$
Since  $\mathbf{O}_{ij}$  is assumed to be uniformly distributed on  $\mathrm{SO}(d)$  in the RCM, given any matrix  $\mathbf{Q} \in \mathrm{SO}(d)$ , it is easy to see that  $\mathbf{O}_{ij}\mathbf{Q}$  is also uniformly distributed on  $\mathrm{SO}(d)$ , so we have
$$
\mathbb {E} (\boldsymbol {O} _ {i j}) = \mathbb {E} (\boldsymbol {O} _ {i j} \boldsymbol {Q}) = \mathbb {E} (\boldsymbol {O} _ {i j}) \cdot \boldsymbol {Q}, \quad \forall \boldsymbol {Q} \in \mathrm {S O} (d).
$$
Let  $\pmb{E}_{kl} \in \mathrm{SO}(d)$ ,  $k \neq l$  denote the diagonal matrix whose  $k$ -th and  $l$ -th diagonal entries are -1 and others are 1, then we have  $\mathbb{E}(\pmb{O}_{ij}) = \mathbb{E}(\pmb{O}_{ij}) \cdot \pmb{E}_{kl}$ , which implies
$$
d \mathbb {E} (\boldsymbol {O} _ {i j}) = \mathbb {E} (\boldsymbol {O} _ {i j}) \cdot \left(\boldsymbol {E} _ {1 2} + \boldsymbol {E} _ {2 3} + \dots + \boldsymbol {E} _ {d 1}\right) = (d - 4) \mathbb {E} (\boldsymbol {O} _ {i j}).
$$
Thus, we have  $\mathbb{E}(\pmb{O}_{ij}) = \mathbf{0}$ . Then, it is easy to see that  $\mathbb{E}(\pmb{W}_{ij}) = 0$  and
$$
\operatorname {V a r} \left(\boldsymbol {W} _ {i j}\right) = (1 - p q) ^ {2} \boldsymbol {I} + (1 - q) p ^ {2} q ^ {2} \boldsymbol {I} + (1 - p) q \left(1 + p ^ {2} q ^ {2}\right) \boldsymbol {I} = q \left(1 - p ^ {2} q\right) \boldsymbol {I}. \tag {18}
$$
Based on the above calculations, we can derive the following Lemma, which is a direct result of the matrix Bernstein inequality [38]. Similar results can also be found in [42, Lemma 9], [27, Proposition 3], and [26, Eq. (5.12)].
Lemma 2. With probability at least  $1 - \mathcal{O}(1 / n)$ , the following holds for any  $m\in [n]$ :
$$
\| \boldsymbol {W} \| _ {2} = \mathcal {O} \left(\sqrt {n q \log n}\right), \quad \| \boldsymbol {W} ^ {(m)} \| _ {2} = \mathcal {O} \left(\sqrt {n q \log n}\right), \quad \| \boldsymbol {W} _ {m} \| _ {2} = \mathcal {O} \left(\sqrt {n q \log n}\right).
$$
Proof. Note that  $\mathbf{W} = \sum_{i < j} \mathbf{W}^{(ij)}$ , where  $\mathbf{W}^{(ij)} \in \mathbb{R}^{nd \times nd}$  denotes the matrix with the  $(i,j)$  and  $(j,i)$ -th block equal to  $\mathbf{W}_{ij}$  and  $\mathbf{W}_{ji}$  and others equal to 0. So  $\{\mathbf{W}^{(ij)}\}$  are i.i.d. centered and bounded random matrices. Besides, we have
$$
\left\| \mathbb {E} (\boldsymbol {W} \boldsymbol {W} ^ {\top}) \right\| _ {2} = \left\| \sum_ {i <   j} \mathbb {E} (\boldsymbol {W} ^ {(i j)} (\boldsymbol {W} ^ {(i j)}) ^ {\top}) \right\| _ {2} = 2 (n - 1) \| \operatorname {V a r} (\boldsymbol {W} _ {i j}) \| _ {2} = \mathcal {O} (n q).
$$
According to the matrix Bernstein inequality [38], we have that  $\| \pmb{W}\| _2 = \mathcal{O}\left(\sqrt{nq\log n}\right)$  holds with probability at least  $1 - \mathcal{O}(1 / n^2)$ . The above argument also holds for each  $\pmb{W}^{(m)}$ , then taking a union bound over the choice of  $m\in [n]$  yields the second result. For  $\pmb{W}_{m}$ , we have
$$
\| \boldsymbol {W}\|_{2} = \max_{\| \boldsymbol {u}\|_{2} = 1}\| \boldsymbol {W}\boldsymbol {u}\|_{2}\geq \max_{\| \boldsymbol {u}\|_{2} = 1}\| \boldsymbol{W}_{m}\boldsymbol {u}\|_{2} = \| \boldsymbol{W}_{m}\|_{2},
$$
which gives the last result.
The following lemma follows from [13]; see also [26, Theorem A.2]. [42, Lemma 11] is a special case where  $d = 1$ . Before that, we need to introduce the distance up to a global orthogonal matrix:
$$
\operatorname {D i s t} \left(\boldsymbol {X}, \boldsymbol {Y}\right) = \| \boldsymbol {X} - \boldsymbol {Y} \boldsymbol {R} ^ {\star} \| _ {F}, \text {w h e r e} \boldsymbol {R} ^ {\star} = \underset {\boldsymbol {R} \in O (d)} {\arg \min } \| \boldsymbol {X} \boldsymbol {R} - \boldsymbol {Y} \| _ {F} ^ {2} = \mathcal {P} _ {O (d)} (\boldsymbol {X} ^ {\top} \boldsymbol {Y}).
$$
Lemma 3 (Davis-Kahan sin  $\Theta$  Theorem). Suppose that  $\mathbf{A},\mathbf{E}\in \mathcal{C}^{n\times n}$  are Hermitian matrices and  $\tilde{\pmb{A}} = \pmb {A} + \pmb{E}$ . Let  $\delta = \lambda_d(\pmb {A}) - \lambda_{d + 1}(\pmb {A}) > 0$  be the gap between the  $d$ -th eigenvalue and  $d + 1$ -th eigenvalue of  $\mathbf{A}$  for some  $1\leq d\leq n - 1$ . Furthermore, let  $\mathbf{U},\tilde{\mathbf{U}}$  be the  $d$ -leading eigenvectors of  $\mathbf{A}$  and  $\tilde{\pmb{A}}$ , respectively, which are normalized such that  $\| \boldsymbol {U}\| _F = \| \tilde{\boldsymbol{U}}\| _F = \sqrt{nd}$ . Then, we have
$$
\operatorname {D i s t} (\boldsymbol {U}, \tilde {\boldsymbol {U}}) \leq \frac {\sqrt {2} \| \boldsymbol {E} \boldsymbol {U} \| _ {F}}{\delta - \| \boldsymbol {E} \| _ {2}}. \tag {19}
$$
A.1 Initialization Error in Euclidean Norm
Based on Lemma 2 and Lemma 3, we have the following bound on Dist. Note that the notations  $\Phi, \Psi, \widetilde{\Phi}, \widetilde{\Psi}$  used in the following analysis are defined in SpectrIn (i.e., Algorithm 2).
Lemma 4. Let  $\Phi$  and  $\Phi^{(m)}$  be the  $d$ -leading eigenvectors of  $\mathbf{Y}$  and  $\mathbf{Y}^{(m)}$ , respectively, which are normalized such that  $\| \Phi \|_F = \| \Phi^{(m)} \|_F = \sqrt{nd}$ . Then, we have
$$
\operatorname {D i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {q}}\right) \quad a n d \quad \operatorname {D i s t} (\boldsymbol {\Phi}, \boldsymbol {\Phi} ^ {(m)}) = \mathcal {O} (1) \tag {20}
$$
hold with probability at least  $1 - \mathcal{O}(1 / n)$
Proof. Let us Choose  $\mathbf{A} = \mathbf{Y}^{(m)}$  and  $\mathbf{E} = \Delta \mathbf{W}^{(m)} = \mathbf{W} - \mathbf{W}^{(m)}$  in Lemma 3, then  $\tilde{\mathbf{A}} = \mathbf{Y}$ ,  $\mathbf{U} = \Phi^{(m)}$  and  $\tilde{\mathbf{U}} = \Phi$ . Since  $\Phi^{(m)}$  is independent of  $\Delta \mathbf{W}^{(m)}$ , similar to Lemma 2, we apply the matrix Bernstein inequality [38] to obtain, with probability at least  $1 - \mathcal{O}(1 / n^2)$ , that
$$
\left\| \boldsymbol {E} \boldsymbol {U} \right\| _ {F} = \left\| \Delta \boldsymbol {W} ^ {(m)} \Phi^ {(m)} \right\| _ {F} = \mathcal {O} (\sqrt {n q \log n}).
$$
In addition,  $\pmb{Y}^{(m)} = pq\pmb{X}^{\star}\pmb{X}^{\star \top} + \pmb{W}^{(m)}$  implies that
$$
\delta = \lambda_ {d} (\boldsymbol {Y} ^ {(m)}) - \lambda_ {d + 1} (\boldsymbol {Y} ^ {(m)}) \geq \lambda_ {d} (p q \boldsymbol {X} ^ {\star} \boldsymbol {X} ^ {\star^ {\top}}) - \| \boldsymbol {W} ^ {(m)} \| _ {2} \geq n p q - \mathcal {O} (\sqrt {n q \log n}).
$$
where the second inequality holds due to  $\lambda_d(X^\star X^{\star \top}) = n$  and the last inequality is from  $\lambda_d(X^\star X^{\star \top}) = n$  and Lemma 2. Based on the condition that  $p^2 q = \Omega \left(\frac{\log n}{n}\right)$ , as long as  $p^2 q \geq \frac{C\log n}{n}$  for some large enough constant  $C$ , we have
$$
\delta - \| \boldsymbol {E} \| _ {2} \geq n p q - \mathcal {O} (\sqrt {n q \log n}) - \| \boldsymbol {E} \| _ {2} \geq n p q - \mathcal {O} (\sqrt {n q \log n}) \geq \frac {1}{2} n p q,
$$
where the second inequality holds because of  $\| \pmb {E}\| _2\leq \| \pmb {W}\| _2 + \| \pmb{W}^{(m)}\| _2 = \mathcal{O}(\sqrt{nq\log n})$  . Hence, by applying Lemma 3, we get
$$
\operatorname {D i s t} \left(\boldsymbol {\Phi}, \boldsymbol {\Phi} ^ {(m)}\right) \leq \frac {\sqrt {2} \| \boldsymbol {E} \boldsymbol {U} \| _ {F}}{\delta - \| \boldsymbol {E} \| _ {2}} \leq \frac {\mathcal {O} (\sqrt {n q \log n})}{n p q} = \mathcal {O} (1) \tag {21}
$$
where the last inequality is because of  $p\sqrt{q} = \Omega(\sqrt{\log n / n})$ . Similarly, by choosing  $A = pqX^{\star}X^{\star \top}$  and  $E = W$ , we can show that
$$
\operatorname {D i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) \leq \frac {\sqrt {2} \| \boldsymbol {W} \boldsymbol {X} ^ {\star} \| _ {F}}{n p q - \mathcal {O} (\sqrt {n q \log n})} \leq \frac {\sqrt {2} \| \boldsymbol {W} \| _ {2} \| \boldsymbol {X} ^ {\star} \| _ {F}}{\frac {1}{2} n p q} = \mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {q}}\right).
$$
Here, the last inequality holds because  $\| \boldsymbol{W}\| _2 = \mathcal{O}(\sqrt{nq\log n})$  (see Lemma 2) and the fact that  $\| X^{\star}\|_{F} = \sqrt{nd}$ .
Following the same analysis as in Lemma 4, we are also able to show that  $\mathrm{Dist}(\Psi, X^{\star}) = \mathcal{O}\left(\frac{\sqrt{\log n}}{p\sqrt{q}}\right)$ , where  $\Psi$  reverses the sign of the last column of  $\Phi$  so that the determinants of  $\Phi$  and  $\Psi$  differ by a sign. However, Lemma 4 only provides the upper bound on "Dist", i.e., the distance up to an orthogonal matrix. The following lemma translates the result to that on "dist".
Lemma 5. Suppose that  $\| \widetilde{\Phi} -\Phi \| _F\leq \| \widetilde{\Psi} -\Psi \| _F$  , where  $\widetilde{\Phi} = \mathcal{P}_{\mathrm{SO}(d)^n}(\Phi)$  and  $\widetilde{\Psi} = \mathcal{P}_{\mathrm{SO}(d)^n}(\Psi)$  then we have
$$
\mathrm {d i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \mathrm {D i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {q}}\right).
$$
Proof. Based on the structure of  $O(d)$  and  $\mathrm{SO}(d)$ , it is easy to see that
$$
\operatorname {D i s t} \left(\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}\right) = \min  \left\{\operatorname {d i s t} \left(\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}\right), \operatorname {d i s t} \left(\boldsymbol {\Psi}, \boldsymbol {X} ^ {\star}\right) \right\}.
$$
Our remaining task is to prove  $\mathrm{Dist}(\Phi, X^{\star}) = \mathrm{dist}(\Phi, X^{\star})$  based on the condition that  $\|\widetilde{\Phi} - \Phi\|_F \leq \|\widetilde{\Psi} - \Psi\|_F$ . If  $\mathrm{Dist}(\Phi, X^{\star}) = \mathrm{dist}(\Psi, X^{\star})$ , then we have
$$
\mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {q}}\right) = \operatorname {D i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \operatorname {d i s t} (\boldsymbol {\Psi}, \boldsymbol {X} ^ {\star}) \geq \| \widetilde {\boldsymbol {\Psi}} - \boldsymbol {\Psi} \| _ {F} \geq \| \widetilde {\boldsymbol {\Phi}} - \boldsymbol {\Phi} \| _ {F}
$$
where the first inequality holds because  $\widetilde{\Psi}$  is the projection of  $\Psi$  on  $\mathrm{SO}(d)^n$ . It is easy to see that
$$
\| \widetilde {\Phi} - \Phi \| _ {F} + \| \widetilde {\Psi} - \Psi \| _ {F} = \| \Phi - \mathcal {P} _ {\mathrm {S O} (d) ^ {n}} (\Phi) \| _ {F} + \| \Phi - \mathcal {P} _ {(O (d) \backslash \mathrm {S O} (d)) ^ {n}} (\Phi) \| _ {F} \geq 2 \sqrt {n}.
$$
The equality holds because the mapping that reverses the sign of the last column of a matrix is a bijection between  $\mathrm{SO}(d)$  and  $O(d) \setminus \mathrm{SO}(d)$ , and the inequality holds since the minimum distance between  $\mathrm{SO}(d)$  and  $O(d) \setminus \mathrm{SO}(d)$  is 2. Under the condition that  $p^2 q = \Omega\left(\frac{\log n}{n}\right)$ , as long as  $p^2 q \geq \frac{C \log n}{n}$  for some large enough constant  $C$ , we could have  $\|\widetilde{\Phi} - \Phi\|_F + \|\widetilde{\Psi} - \Psi\|_F = \mathcal{O}\left(\frac{\sqrt{\log n}}{p \sqrt{q}}\right) < 2\sqrt{n}$ , which contradict to the above inequality. Thus, we have
$$
\operatorname {d i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \operatorname {D i s t} (\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}) = \mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {q}}\right).
$$
A.2 Initialization Error in Infinity Norm
Next, based on the Davis-Kahan theorem, we can bound the distance between  $\widetilde{X}^0$  and  $X^{\star}$  in the infinity norm, which is stated in the following result.
Lemma 6. Let  $\Phi$  be the  $d$ -leading eigenvectors of  $\mathbf{Y}$ . Then, we have
$$
\operatorname {d i s t} _ {\infty} \left(\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}\right) = \mathcal {O} \left(\frac {\sqrt {\log n}}{p \sqrt {n q}}\right) \tag {22}
$$
holds with probability at least  $1 - \mathcal{O}(1 / n)$
Proof. Let  $\Pi^m = \mathrm{argmin}_{\Pi \in \mathrm{SO}(d)}\| \Phi -\Phi^{(m)}\Pi \| _F$ . We can compute
$$
\begin{array}{l} \| (\boldsymbol {W} \boldsymbol {\Phi}) _ {m} \| _ {F} = \| \boldsymbol {W} _ {m} ^ {\top} \boldsymbol {\Phi} \| _ {F} \leq \| \boldsymbol {W} _ {m} ^ {\top} \boldsymbol {\Phi} ^ {(m)} \boldsymbol {\Pi} ^ {m} \| _ {F} + \| \boldsymbol {W} _ {m} ^ {\top} (\boldsymbol {\Phi} - \boldsymbol {\Phi} ^ {(m)} \boldsymbol {\Pi} ^ {m}) \| _ {F} \\ \leq \left\| \boldsymbol {W} _ {m} ^ {\top} \boldsymbol {\Phi} ^ {(m)} \right\| _ {F} + \left\| \boldsymbol {W} _ {m} \right\| _ {2} \left\| \boldsymbol {\Phi} - \boldsymbol {\Phi} ^ {(m)} \boldsymbol {\Pi} ^ {m} \right\| _ {F}. \\ \end{array}
$$
The fact that  $\Phi^{(m)}$  is independent from  $W_{m}$  implies  $\| W_m^\top \Phi^{(m)}\| _F = \mathcal{O}(\sqrt{nq\log n})$ , so we have
$$
\left\| \left(\boldsymbol {W} \boldsymbol {\Phi}\right) _ {m} \right\| _ {F} = \mathcal {O} (\sqrt {n q \log n}) (1 + \mathcal {O} (1)) = \mathcal {O} (\sqrt {n q \log n}). \tag {24}
$$
Next, let  $\Pi^{\star} = \operatorname{argmin}_{\mathbf{II}\in \mathrm{SO}(d)}\| \Phi -X^{\star}\Pi \|_{F}$ , then we have
$$
\left(\boldsymbol {Y} \boldsymbol {\Phi}\right) _ {m} = p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {X} ^ {\star^ {\top}} \boldsymbol {\Phi} + \left(\boldsymbol {W} \boldsymbol {\Phi}\right) _ {m} = n p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {\Pi} ^ {\star} + p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {X} ^ {\star^ {\top}} \left(\boldsymbol {\Phi} - \boldsymbol {X} ^ {\star} \boldsymbol {\Pi} ^ {\star}\right) + \left(\boldsymbol {W} \boldsymbol {\Phi}\right) _ {m}.
$$
Since  $\Phi$  is the  $d$ -leading eigenvector of  $\mathbf{Y}$ , we have  $\mathbf{Y} \Phi = \Phi \boldsymbol{\Sigma}$  with  $\boldsymbol{\Sigma} \in \mathbb{R}^{d \times d}$  consisting of the  $d$ -leading eigenvalues of  $\mathbf{Y}$ . Applying the standard eigenvalue perturbation theory, e.g., [37, Theorem 4.11], we have
$$
\| \boldsymbol {\Sigma} - n p q \boldsymbol {I} \| _ {2} = \mathcal {O} (\| \boldsymbol {W} \| _ {2}) = \mathcal {O} (\sqrt {n q \log n}).
$$
Based on the assumption that  $p^2 q = \Omega \left( \frac{\log n}{n} \right)$ , we can show that  $\pmb{\Sigma} \geq \frac{3}{4} npq \pmb{I}$ . Note that  $\pmb{Y} \pmb{\Phi} = \pmb{\Phi} \pmb{\Sigma}$  implies  $\pmb{\Phi}_m = (\pmb{Y} \pmb{\Phi})_m \pmb{\Sigma}^{-1}$  for each  $m \in [n]$ , then we have
$$
\| \boldsymbol {\Phi} _ {m} \| _ {F} \leq \frac {4}{3 n p q} \| (\boldsymbol {Y} \boldsymbol {\Phi}) _ {m} \| _ {F} \leq \frac {4}{3 n p q} \| p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {X} ^ {\star^ {\top}} \boldsymbol {\Phi} _ {m} \| _ {F} + \frac {4}{3 n p q} \| (\boldsymbol {W} \boldsymbol {\Phi}) _ {m} \| _ {F} \leq 2 \sqrt {d},
$$
where the last inequality holds because  $\frac{4}{3npq}\|pq\pmb{X}_m^\star\pmb{X}^{\star\top}\pmb{\Phi}_m\|_F = \frac{4}{3n}\|\pmb{X}^{\star\top}\pmb{\Phi}_m\|_F \leq \frac{4\sqrt{d}}{3}$  and (24). Therefore, for each  $m \in [n]$ ,
$$
\begin{array}{l} n p q \left(\Phi_ {m} - \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {\Pi} ^ {\star}\right) = \Phi_ {m} \left(n p q \boldsymbol {I} - \boldsymbol {\Sigma}\right) + \Phi_ {m} \boldsymbol {\Sigma} - n p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {\Pi} ^ {\star} \\ = \boldsymbol {\Phi} _ {m} (n p q \boldsymbol {I} - \boldsymbol {\Sigma}) + (\boldsymbol {Y} \boldsymbol {\Phi}) _ {m} - n p q \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {\Pi} ^ {\star}. \\ \end{array}
$$
This further implies
$$
\begin{array}{l} n p q \| \boldsymbol {\Phi} _ {m} - \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {\Pi} ^ {\star} \| _ {F} \leq \| \boldsymbol {\Sigma} - n p q \boldsymbol {I} \| _ {2} \| \boldsymbol {\Phi} _ {m} \| _ {F} + p q \| \boldsymbol {X} _ {m} ^ {\star} \boldsymbol {X} ^ {\star \top} (\boldsymbol {\Phi} - \boldsymbol {X} ^ {\star} \boldsymbol {\Pi} ^ {\star}) \| _ {F} + \| (\boldsymbol {W} \boldsymbol {\Phi}) _ {m} \| _ {F} \\ \leq 2 \sqrt {d} \| \boldsymbol {W} \| _ {2} + \sqrt {n} p q \| \boldsymbol {\Phi} - \boldsymbol {X} ^ {\star} \boldsymbol {\Pi} ^ {\star} \| _ {F} + \mathcal {O} \sqrt {n q (\log n)} \\ = \mathcal {O} (\sqrt {n q \log n}), \\ \end{array}
$$
where the last inequality holds because of Lemma 2 and Lemma 4. This completes the proof.
Finally, since  $X^0 = \mathcal{P}_{\mathrm{SO}(d)^n}(\Phi)$ , according to Lemma 2 in [27], we have
$$
\operatorname {d i s t} \left(\boldsymbol {X} ^ {0}, \boldsymbol {X} ^ {\star}\right) \leq 2 \operatorname {d i s t} \left(\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}\right) \quad \text {a n d} \quad \operatorname {d i s t} _ {\infty} \left(\boldsymbol {X} ^ {0}, \boldsymbol {X} ^ {\star}\right) \leq 2 \operatorname {d i s t} _ {\infty} \left(\boldsymbol {\Phi}, \boldsymbol {X} ^ {\star}\right),
$$
which complete the proof of Theorem 2.
B Full Proof of Theorem 3
In this section, we provide the full proof of Lemma 1, Proposition 1, and Proposition 2, which finishes the proof of Theorem 3. To simplify the notation and the theoretical derivations, we assume without loss of generality that  $\pmb{X}_i^{\star} = \pmb{I}$  for all  $1 \leq i \leq n$  as one can separately rotate the space that each variable  $\pmb{X}_i$  lies in such that the corresponding ground-truth  $\pmb{X}_i^{\star}$  is rotated to identity [39, Lemma 4.1]. Consequently, we have  $\pmb{Y}_{ij} = \pmb{I}$  for  $(i,j) \in \mathcal{A}$ .
For each  $1 \leq i \leq n$ ,  $|\mathcal{E}_i| = \sum_{1 \leq j \leq n} \mathbf{1}_{\mathcal{E}_i}(j)$ , where  $\mathbf{1}_{\mathcal{E}_i}(\cdot)$  denotes the indicator function w.r.t  $\mathcal{E}_i$ . Based on our model,  $\sum_{1 \leq j \leq n} \mathbf{1}_{\mathcal{E}_i}(j)$  follows the binomial distribution  $B(n, q)$ . According to the Bernstein inequality [38], for any constant  $\epsilon \in (0, 1)$ , we have
$$
\begin{array}{l} \Pr \left(\left| \sum_ {1 \leq j \leq n} \mathbf {1} _ {\varepsilon_ {i}} (j) - n q \right| \geq \epsilon n q\right) \leq 2 \exp \left(- \frac {\frac {1}{2} \epsilon^ {2} n ^ {2} q ^ {2}}{\sum_ {1 \leq j \leq n} E \left\{\mathbf {1} _ {\varepsilon_ {i}} ^ {2} (j) \right\} + \epsilon n q / 3}\right) \\ = 2 \exp \left(- \frac {\frac {1}{2} \epsilon^ {2} n ^ {2} q ^ {2}}{n q + \epsilon n q / 3}\right) \leq 2 \exp \left(- \frac {3}{8} \epsilon^ {2} n q\right). \\ \end{array}
$$
The last inequality holds because of  $\epsilon < 1$ . Therefore,
$$
\Pr \left(\bigcup_ {1 \leq i \leq n} \left\{\left| \mathcal {E} _ {i} \right| - n q \right| \geq \epsilon n q \}\right) \leq 2 n \exp \left(- \frac {3}{8} \epsilon^ {2} n q\right) \leq \frac {2}{n ^ {2}}, \tag {25}
$$
where the last inequality holds because we assume that  $\epsilon \geq \frac{\sqrt{8\log n}}{\sqrt{npq}}$ . Similarly, we have
$$
\Pr \left(\bigcup_ {1 \leq i \leq n} \left\{\left| \mathcal {A} _ {i} \right| - n p q \right| \geq \epsilon n p q \}\right) \leq 2 n \exp \left(- \frac {3}{8} \epsilon^ {2} n p q\right) \leq \frac {2}{n ^ {2}}, \tag {26}
$$