SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 4.2. Under  $\lim \inf \lambda / N_1^{1/4} > 1$ , Algorithm 4 recovers  $v_1, \ldots, v_{k-1}$  exactly. Moreover,  $v_k$  is recovered exactly if  $n_k^2 / N_1 \lesssim 1$ .
The analysis of Algorithm 4 entails careful application of spiked matrix model results. By the linearity of the unfolding operator,
$$
\operatorname {M a t} _ {j - 1} \left(X \times_ {1} \hat {v} _ {1}\right) = \lambda \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \left(\otimes_ {\ell \in [ k ] \setminus \{1, j \}} v _ {\ell}\right) v _ {j} ^ {\top} + \operatorname {M a t} _ {j - 1} \left(Z \times_ {1} \hat {v} _ {1}\right). \tag {19}
$$
Observe that  $Z \times_1 \hat{v}_1$  is a reshaping of the vector  $\mathrm{Mat}_1(Z) \hat{v}_1$ . As  $\hat{v}_1$  and  $Z$  are dependent, the second term on the right-hand side is not a matrix of i.i.d. entries. Despite dependencies, we claim that appropriately scaled,  $\mathrm{Mat}_1(Z) \hat{v}_1$  is distributed as Gaussian noise, in which case exact recovery thresholds are a consequence of spiked matrix model results of Section 1.3 applied to (19).
Below, we display simulation results for power iteration and recursive unfolding and compare to tensor unfolding and partial tracing.
Figure 4: Simulations of power iteration with  $k = 3$ , supersymmetric signal  $v_{1} = v_{2} = v_{3}$ , and  $n_1\in \{400,800,1200\}$ . Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The dashed line is the theoretical prediction  $\mathbf{1}(\tau >1)$ . Power iteration typically converges within five iterations.
Figure 5: Simulations of power iteration with  $k = 3$ ,  $n_1 = n_2 = 50$ , and  $n_3 = 10000$  (left) or  $n_3 = 20000$  (right). Solid lines display empirical cosine similarities (each point is the average of 50 realizations). Vector  $v_3$ , corresponding to the longest tensor axis, is estimated less well than  $v_1, v_2$ .
Figure 6: Simulations of a unfolding, partial tracing, and power iteration with  $k = 3$ , supersymmetric signal  $v_{1} = v_{2} = v_{3}$ , and  $n_1 = 1200$ . Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The dashed line is the theoretical limit of unfolding and partial tracing,  $\overline{c}(\tau)$ . Near the phase transition located at  $\tau = 1$ , the cosine similarity of power iteration is over twice that of unfolding.
Figure 7: Simulations of recursive unfolding with  $k = 3$ ,  $n_1 = n_2 = 50$ , and  $n_3 = 10000$  (left) or  $n_3 = 50000$  (right). Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The performance of recursive unfolding is quite similar to that of power iteration.
[1] A. Anandkumar, R. Ge, D. Hsu, and S. M. Kakade. A Tensor Approach to Learning Mixed Membership Community Models. Journal of Machine Learning Research, 1:2239-2312. 2014.
[2] A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor Decompositions for Learning Latent Variable Models. Journal of Machine Learning Research, 15:2773-2832, 2014.
[3] M. Bayati and A. Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory, 57(2):764-785, 2011.
[4] G. Ben Arous, R. Gheissari, and A. Jagannath. Algorithmic thresholds for tensor PCA. Annals of Probability, 48(4):2052-2087, 2020.
[5]G. Ben Arous, D. Z. Huang, and J. Huang. Long Random Matrices and Tensor Unfolding. arXiv preprint arXiv2110.1021, 2021.
[6] G. Ben Arous, S. Mei, A. Montanari, and M. Nica. The landscape of the spiked tensor model. Communications on Pure and Applied Mathematics, 72:2282-2330, 2019.
[7] F. Benaych-Georges and R. R. Nadakuditi. The singular values and vectors of low rank perturbations of large rectangular random matrices. Journal of Multivariate Analysis, 111:120–135, 2012.
[8] A. R. Benson, D. F. Gleich, and J. Leskovec. Tensor spectral clustering for partitioning higher-order network structures. Proceedings of the 2015 SIAM International Conference on Data Mining, 118-126, 2015.
[9] X. Bi, X. Tang, Y. Yuan, Y. Zhang, A. Qu. Tensors in Statistics. Annual Review of Statistics and Its Application, 8:345-368, 2021.
[10] A. Bloemendal, L. Erdos, A. Knowles, H. T. Yau, and J. Yin. Isotropic local laws for sample covariance and generalized Wigner matrices. Electronic Journal of Probability, 19(33):1-53, 2014.
[11] B. B. Chen and G. M. Pan. Convergence of the largest eigenvalue of normalized sample covariance matrices when p and n both tend to infinity with their ratio converging to zero. Bernoulli, 18(4):1405-1420, 2012.
[12] L. De Lathauwer, B. De Moor, and J. Vandewalle. A Multilinear Singular Value Decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253-1278, 2000.
[13] M. J. Feldman. Spiked Singular Values and Vectors under Extreme Aspect Ratios. Journal of Multivariate Analysis, 196:105187, 2023.
[14] C. J. Hillar and L. Lim. Most tensor problems are NP-hard. Journal of the ACM, 60(6):1-39, 2013.
[15] S. B. Hopkins, J. Shi, and D. Steurer. Tensor principal component analysis via sum-of-square proofs. Conference on Learning Theory, 956-1006, 2015.
[16] S. B. Hopkins, T. Schramm, J. Shi, and D. Steurer. Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors. Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, 178-191, 2016.
[17] J. Huang, D. Z. Huang, Q. Yang, and G. Cheng. Power Iteration for Tensor PCA. Journal of Machine Learning Research, 23:1-47, 2022.
[18] J. B. Kruskal, Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics, Linear Algebra and Applications 18(2):95-138, 1977.
[19] H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos. A survey of multilinear subspace learning for tensor data. Pattern Recognition, 44(7):1540-1551, 2011.
[20] A. Montanari and E. Richard. A statistical model for tensor PCA. Proceedings of the 27th International Conference on Neural Information Processing Systems, 2:2897-2905, 2014.
[21] M. E. A. Seddik, M. Guillaume, and R. Couillet. When random tensors meet random matrices. arXiv preprint arXiv:2112.12348, 2021.
[22] N. D. Sidiropoulos, L. De Lathauwer, X. Fu, K. Huang, E. E. Papalexakis, and C. Faloutsos. Tensor Decomposition for Signal Processing and Machine Learning. IEEE Transactions on Signal Processing, 65(13):3551-3582, 2017.
[23] R. Tomioka and T. Suzuki. Spectral norm of random tensors, 2014. arXiv:1407.1870.
[24] M. Wainwright. High-Dimensional Statistics. Cambridge, 2019.
[25] Y. Yu, T. Wang, and R. J. Samworth. A useful variant of the Davis-Kahan theorem for statisticians. Biometrika, 102:315-323, 2015.
Proof of Theorem (2.1). By (11),  $\mathrm{Mat}_S(X)$  is a spiked matrix with aspect ratio  $N(S)^2 / N_1$  and signal strength  $\lambda$  (as  $\otimes_{j \in S} v_j$  is unit norm). There are four cases to consider:
- Let  $N(S)^2 / N_1 \to 0$ ; by the first point of (6), the recovery threshold lies at  $(N_1 / N(S) \cdot N(S))^{1/4} = N_1^{1/4}$ .
- Let  $N(S)^2 / N_1 \to \gamma \in (0, 1]$ ; by the first point of (5), partial recovery occurs for  $\lim \inf \lambda / \sqrt{N_1 / N(S)} > \gamma^{1/4}$ . Equivalently,  $\lim \inf \lambda / N_1^{1/4} > 1$ . Under  $\lambda = \tau(1 + o(1)) N_1^{1/4}$ ,  $\lim \lambda / \sqrt{N_1 / N(S)} = \tau \gamma^{1/4}$ , implying  $|\langle \otimes_{j \in S} v_j, w \rangle|^2 \xrightarrow{a.s.} c^2(\tau \gamma^{1/4}, \gamma)$ .
- Let  $N(S)^2 / N_1 \to \gamma \in (1, \infty]$ ; by the second point of (5), partial recovery occurs for  $\lim \inf \lambda / \sqrt{N(S)} > \gamma^{-1/4}$ . Equivalently,  $\lim \inf \lambda / N_1^{1/4} > 1$ . Under  $\lambda = \tau(1 + o(1)) N_1^{1/4}$ ,  $\lim \lambda / \sqrt{N(S)} = \tau \gamma^{-1/4}$ , implying  $|\langle \otimes_{j \in S} v_j, w \rangle|^2 \xrightarrow{a.s.} c^2 (\tau \gamma^{-1/4} \cdot \gamma^{1/2}, \gamma) = c^2 (\tau \gamma^{1/4}, \gamma)$ .
- Let  $N(S)^2 / N_1 \to \infty$ ; by the second point of (7), the recovery threshold lies at  $\sqrt{N(S)}$ .
Proof of Theorem 3.1. We provide proof for  $k = 3$ ; the proof for higher orders is similar and omitted. For notational simplicity, we suppress the subscripts of  $n_1$  and  $v_{1}$ .
Let  $w_{i} = \operatorname{tr}(Z_{i})$ ,  $i \in [n]$ , and  $w = (w_{1}, \ldots, w_{n})^{\top}$ . Expanding the partial trace matrix,
$$
\begin{array}{l} \operatorname {T r} _ {k} (X) = \sum_ {i = 1} ^ {n} (\lambda v _ {i} + w _ {i}) X _ {i} = \sum_ {i = 1} ^ {n} \left[ (\lambda^ {2} v _ {i} ^ {2} + \lambda v _ {i} w _ {i}) v v ^ {\top} + (\lambda v _ {i} + w _ {i}) Z _ {i} \right] \\ = \sum_ {i = 1} ^ {n} \left[ \left(\lambda^ {2} v _ {i} ^ {2} + \lambda v _ {i} w _ {i}\right) v v ^ {\top} + \left(\lambda v _ {i} + w _ {i}\right) \left(Z _ {i} - \operatorname {d i a g} \left(Z _ {i} - \widetilde {Z} _ {i}\right)\right) + \left(\lambda v _ {i} + w _ {i}\right) \operatorname {d i a g} \left(Z _ {i} - \widetilde {Z} _ {i}\right) \right], \tag {20} \\ \end{array}
$$
where  $\widetilde{Z}$  is an independent copy of  $Z$ .
Let  $M = \| \lambda v + w\| _2^{-1}\sum_{i = 1}^n (\lambda v_i + w_i)(Z_i - \mathrm{diag}(Z_i - \widetilde{Z}_i))$ . As  $w_{i}$  and  $Z_{i} - \mathrm{diag}(Z_{i} - \widetilde{Z}_{i})$  are independent and the Gaussian distribution is rotationally invariant,  $M\stackrel {d}{=}Z_1$ . Hence, we have
$$
\left\| \lambda v + w \right\| _ {2} ^ {- 1} \operatorname {T r} _ {k} (X) = \alpha v v ^ {\top} + M + \left\| \lambda v + w \right\| _ {2} ^ {- 1} \sum_ {i = 1} ^ {n} \left(\lambda v _ {i} + w _ {i}\right) \operatorname {d i a g} \left(Z _ {i} - \widetilde {Z} _ {i}\right), \tag {21}
$$
where  $\alpha = \| \lambda v + w\| _2^{-1}(\lambda^2 +\lambda \langle v,w\rangle)$ . The partial trace matrix is thus proportional to a perturbation of a spiked matrix with aspect ratio  $\gamma = 1$
Let  $\mathcal{E}$  denote the third term on the right-hand side of (21); we shall prove  $n^{-1/2} \| \mathcal{E} \|_2 \xrightarrow{a.s.} 0$ . Denoting  $u = (\lambda v + w) / \| \lambda v + w \|_2$ , the diagonal entries of  $\sum_{i=1}^{n} u_i \mathrm{diag}(\widetilde{Z}_i)$  are i.i.d. Gaussians with variance one, implying  $\| \sum_{i=1}^{n} u_i \mathrm{diag}(\widetilde{Z}_i) \|_2 \lesssim \sqrt{\log n}$ , almost surely. Similarly, since  $w \sim \mathcal{N}(0, nI_n)$  and  $\| \lambda v + w \|_2 = \Theta_{a.s.}(\lambda + n)$ ,
$$
\left. \lambda \| \lambda v + w \| _ {2} ^ {- 1} \right\| \sum_ {i = 1} ^ {n} v _ {i} \operatorname {d i a g} \left(Z _ {i}\right) \Bigg \| _ {2} \lesssim \sqrt {\log n} \left(1 + \frac {\lambda}{n}\right). \tag {22}
$$
To bound the remaining term of  $\mathcal{E}$ , let  $\mathcal{Z} \in \mathbb{R}^{n \times n}$  denote the matrix with entries  $\mathcal{Z}_{ij} = Z_{ijj}$ ,  $i, j \in [n]$ , in which case we may write
$$
\left\| \sum_ {i = 1} ^ {n} w _ {i} \operatorname {d i a g} \left(Z _ {i}\right) \right\| _ {2} = \sup  _ {1 \leq j \leq n} \left| \sum_ {i = 1} ^ {n} w _ {i} Z _ {i j j} \right| = \sup  _ {1 \leq j \leq n} \left| e _ {j} ^ {\top} \mathcal {Z} ^ {\top} \mathcal {Z} \mathbf {1} _ {n} \right|, \tag {23}
$$
where  $\mathbf{1}_n$  is the length-  $n$  vector of ones. As  $e_j^\top \mathcal{Z}^\top \mathcal{Z}e_j\sim \chi_n^2$  and
$$
e _ {j} ^ {\top} \mathcal {Z} ^ {\top} \mathcal {Z} \left(\mathbf {1} _ {n} - e _ {j}\right) \stackrel {{d}} {{=}} \sqrt {n - 1} e _ {j} ^ {\top} \mathcal {Z} ^ {\top} \widetilde {Z} e _ {j} \sim \sqrt {n - 1} \cdot \mathcal {N} (0, 1) \cdot \sqrt {\chi_ {n} ^ {2}},
$$
standard bounds such as (2.19) in [24] yield
$$
\mathbf {P} \left(\sup  _ {1 \leq j \leq n} \left| e _ {j} ^ {\top} \mathcal {Z} ^ {\top} \mathcal {Z} \mathbf {1} _ {n} - n \right| > c \cdot n\right) \lesssim n e ^ {- C n}, \tag {24}
$$
where  $c, C > 0$  are constants. As the right-hand side is summable, the Borel-Cantelli lemma implies
$$
\sup  _ {1 \leq j \leq n} \left| e _ {j} ^ {\top} \mathcal {Z} ^ {\top} \mathcal {Z} \mathbf {1} _ {n} \right| \lesssim n, \tag {25}
$$
almost surely. Collecting the above bounds, we have that  $n^{-1/2} \|\mathcal{E}\|_2 \xrightarrow{a.s.} 0$ .
By Weyl's inequality, the limiting spectral distribution of  $n^{-1/2}(M + \mathcal{E})$  equals that of  $n^{-1/2}M$ , the quarter circle law. The limits (5) from Lemma 1.1 therefore apply to  $\alpha vv^{\top} + M + \mathcal{E}$  as well (see [7]). Basic calculations yield (1)  $\lim \inf \alpha / \sqrt{n} > 1$  if and only if  $\lim \inf \lambda / n^{3/4} > 1$ , and (2) under  $\lambda = \tau(1 + o(1))n^{3/4}$ ,  $\alpha = \tau^2(1 + o_{a.s.}(1))\sqrt{n}$ . Therefore,  $\hat{v}$  partially recovers  $v$  if and only if  $\lim \inf \lambda / n^{3/4} > 1$ , and under  $\lambda = \tau(1 + o(1))n^{3/4}$ ,
$$
| \langle v, \hat {v} \rangle | ^ {2} \xrightarrow {a . s .} c ^ {2} (\tau^ {2}, 1) = \overleftarrow {c} ^ {2} (\tau), \tag {26}
$$
Proof of Theorem 4.1. By the linearity of the operators  $\times_{1},\ldots ,\times_{k}$
$$
w _ {j} = \lambda \prod_ {\ell \in [ k ] \backslash \{j \}} \left\langle v _ {\ell}, \hat {v} _ {\ell} \right\rangle \cdot v _ {j} + Z \times_ {1} \hat {v} _ {1} \dots \times_ {j - 1} \hat {v} _ {j - 1} \times_ {j + 1} \hat {v} _ {j + 1} \dots \times_ {k} \hat {v} _ {k}, \tag {27}
$$
$$
\langle v _ {j}, w _ {j} \rangle = \lambda \prod_ {\ell \in [ k ] \backslash \{j \}} \left\langle v _ {\ell}, \hat {v} _ {\ell} \right\rangle + \left\langle Z, \hat {v} _ {1} \otimes \dots \otimes \hat {v} _ {j - 1} \otimes v _ {j} \otimes \hat {v} _ {j + 1} \dots \otimes \hat {v} _ {k} \right\rangle . \tag {28}
$$
Assume that (17) holds for  $\ell \in [j - 1]$ . For  $\ell \in \{j + 1,\dots ,k\}$ , Corollary 2.1.1 implies  $|\langle v_{\ell},\hat{v}_{\ell}\rangle |$  is bounded away from zero. The first term on the right-hand side of (28) is therefore  $\Theta_{a.s.}(\lambda)$ . The second term is bounded by the spectral norm of  $Z$ : using Theorem 1 of [23],
$$
\left\langle Z, \hat {v} _ {1} \otimes \dots \otimes \hat {v} _ {j - 1} \otimes v _ {j} \otimes \hat {v} _ {j + 1} \otimes \dots \otimes \hat {v} _ {k} \right\rangle \leq \sup  _ {u _ {j} \in \mathbb {S} ^ {j - 1}, j \in [ k ]} \left\langle Z, u _ {1} \otimes \dots \otimes u _ {k} \right\rangle \lesssim \sqrt {n _ {k}}, \tag {29}
$$
almost surely. Thus, as  $(\sqrt{n_k} /\lambda)^4\lesssim n_k^2 /N_1\to 0$
$$
\left| \left\langle v _ {j}, \hat {v} _ {j} \right\rangle \right| = 1 + O _ {a. s.} \left(\frac {\sqrt {n _ {k}}}{\lambda}\right) \xrightarrow {a . s .} 1, \tag {30}
$$
from which (17) follows inductively.
Equation (18) follows from (17) and (29), which imply  $\| v_{1}\otimes \dots \otimes v_{k} - \hat{v}_{1}\otimes \dots \otimes \hat{v}_{k}\|_{F}\xrightarrow{a.s.}0$  and
$$
\lambda^ {- 1} | \hat {\lambda} | = \left| \prod_ {j = 1} ^ {k} \langle v _ {j}, \hat {v} _ {j} \rangle + \lambda^ {- 1} \langle Z, \hat {v} _ {1} \otimes \dots \hat {v} _ {k} \rangle \right| \xrightarrow {a . s .} 1. \tag {31}
$$
Proof of Theorem 4.2. We shall prove  $v_{2}, \ldots, v_{k}$  are recovered exactly; proofs for the first and last axes are similar and omitted. By the linearity of the unfolding operator,
$$
\operatorname {M a t} _ {j - 1} \left(X \times_ {1} \hat {v} _ {1}\right) = \lambda \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \left(\otimes_ {\ell \in [ k ] \backslash \{1, j \}} v _ {\ell}\right) v _ {j} ^ {\top} + \operatorname {M a t} _ {j - 1} \left(Z \times_ {1} \hat {v} _ {1}\right). \tag {32}
$$
Observe that  $Z \times_{1} \hat{v}_{1}$  is a reshaping of the vector  $\mathrm{Mat}_1(Z) \hat{v}_1$ . As  $\hat{v}_{1}$  and  $Z$  are dependent, the second term on the right-hand side is not a matrix of i.i.d. entries. Despite dependencies, we claim that appropriately scaled,  $\mathrm{Mat}_1(Z) \hat{v}_1$  is Gaussian noise, in which case exact recovery thresholds are a consequence of spiked matrix model results of Section 1.3 applied to (32).
By definition,  $\hat{v}_1$  is the first eigenvector of the symmetric matrix
$$
\operatorname {M a t} _ {1} (X) ^ {\top} \operatorname {M a t} _ {1} (X) = \lambda^ {2} v _ {1} v _ {1} ^ {\top} + \operatorname {M a t} _ {1} (Z) ^ {\top} \operatorname {M a t} _ {1} (Z) + \mathcal {E}, \tag {33}
$$
where  $\mathcal{E}$  is a rank-two matrix given by
$$
\mathcal {E} = \lambda v _ {1} \left(\otimes_ {\ell \in [ k ] \backslash \{1 \}} v _ {\ell}\right) ^ {\top} \mathrm {M a t} _ {1} (Z) + \lambda \mathrm {M a t} _ {1} (Z) ^ {\top} \left(\otimes_ {\ell \in [ k ] \backslash \{1 \}} v _ {\ell}\right) v _ {1} ^ {\top}.
$$
Let  $\tilde{v}_1$  denote the first eigenvector of  $\mathrm{Mat}_1(X)^\top \mathrm{Mat}_1(X) - \mathcal{E}$ ; without loss of generality, we assume  $\tilde{v}_1^\top \tilde{v}_1 \geq 0$ . Since  $\mathrm{Mat}_1(Z)^\top (\otimes_{\ell \in [k] \setminus \{1\}} v_\ell) \sim \mathcal{N}(0, I_{n_1})$ , we have  $\| \mathcal{E} \|_2 \lesssim \lambda \sqrt{n_1}$  almost surely. By Theorem 1.1 of [13], the spectral gap of  $\mathrm{Mat}_1(X)^\top \mathrm{Mat}_1(X)$  is  $\Theta(\lambda^2)$ . Thus, using the Davis-Kahan theorem (see Corollary 3 of [25]), we have
$$
\left\| \hat {v} _ {1} - \tilde {v} _ {1} \right\| _ {2} \lesssim \frac {\left\| \mathcal {E} \right\| _ {2}}{\lambda^ {2}} \xrightarrow {a . s .} 0. \tag {34}
$$
Let  $\mathrm{Mat}_1(Z) = U\Lambda V^\top$  be a singular value decomposition. As  $Z$  contains i.i.d. Gaussian entries, (1)  $U$ ,  $\Lambda$ , and  $V$  are independent, (2)  $U$  and  $V$  are Haar-distributed. Moreover, as  $\tilde{v}_1$  is the first eigenvector of  $\lambda^2 v_1 v_1^\top + \mathrm{Mat}_1(Z)^\top \mathrm{Mat}_1(Z) = \lambda^2 v_1 v_1^\top + V\Lambda^2 V^\top$ ,  $\tilde{v}_1$  is independent of  $U$ . Thus,  $U\Lambda V^\top \tilde{v}_1 / \| \Lambda V^\top \tilde{v}_1 \|_2$  is uniform on  $\mathbb{S}^{N_2 - 1}$ . Generating  $\xi \sim \chi_{N_2}^2$  independent of  $Z$ , it follows that
$$
\frac {\xi}{\sqrt {N _ {2}}} \cdot \frac {\operatorname {M a t} _ {1} (Z) \tilde {v} _ {1}}{\| \Lambda V ^ {\top} \tilde {v} _ {1} \| _ {2}} \sim \mathcal {N} \left(0, I _ {N _ {2}}\right). \tag {35}
$$
Defining the constant  $\alpha = \xi / (\sqrt{N_2} \| \Lambda V^\top \tilde{v}_1\|_2)$ , we deduce that  $\alpha Z \times_1 \tilde{v}_1$  (which is a reshaping of the left-hand side of (35)) is distributed as a tensor with i.i.d. Gaussian entries—despite dependencies between  $Z$  and  $\tilde{v}_1$ . Additionally, since  $\Lambda_{11} / \sqrt{N_2} \xrightarrow{a.s.} 1$  and  $\Lambda_{n_1n_1} / \sqrt{N_2} \xrightarrow{a.s.} 1$ , we have  $\| \Lambda V^\top \tilde{v}_1\|_2 / \sqrt{N_2} \xrightarrow{a.s.} 1$  and  $\alpha \xrightarrow{a.s.} 1$ .
$$
\alpha \operatorname {M a t} _ {j - 1} (X \times_ {1} \tilde {v} _ {1}) = \lambda \left(1 + o _ {a. s.} (1)\right) \langle v _ {1}, \tilde {v} _ {1} \rangle \left(\otimes_ {\ell \in [ k ] \backslash \{1, j \}} v _ {\ell}\right) v _ {j} ^ {\top} + \widetilde {Z}, \quad j \in \{2, \dots , k - 1 \}, \tag {36}
$$
where  $\widetilde{Z} \in \mathbb{R}^{n_j \times (N_2 / n_j)}$  contains i.i.d. Gaussian entries, enabling us to apply spiked matrix results. Let  $\tilde{v}_j$  denote the first right singular vector of  $\mathrm{Mat}_{j-1}(X \times_1 \tilde{v}_1)$  (equivalently, that of  $\alpha \mathrm{Mat}_{j-1}(X \times_1 \tilde{v}_1)$ ); without loss of generality, we assume  $\hat{v}_1^\top \tilde{v}_1 \geq 0$ . In particular, since  $|\langle v_1, \tilde{v}_1 \rangle| \asymp |\langle v_1, \hat{v}_1 \rangle| \asymp 1$  by Corollary 2.1.1 and (34), Lemma 1.2 implies  $|\langle v_j, \tilde{v}_j \rangle| \xrightarrow{a.s.} 1$  (we have  $\lambda \gg (n_j \cdot N_2 / n_j)^{1/4}$ ).
It therefore suffices to prove that  $\| \hat{v}_j - \tilde{v}_j\| _2\xrightarrow{a.s.}0$ . By Theorem 2.3 of [5] or Theorem 1.1 of [13] and Cauchy's interlacing inequality, the spectral gap of  $\mathrm{Mat}_{j - 1}(X\times_1\tilde{v}_1)^\top \mathrm{Mat}_{j - 1}(X\times_1\tilde{v}_1)$  is  $\Theta (\lambda^2)$ . Let  $\mathcal{Z}$  denote the reshaping of  $Z$  with dimensions  $n_1\times n_j\times N_2 / n_j$  and slices  $\mathrm{Mat}_{j - 1}(Z_i)$ ,  $i\in [n_{1}]$ . Using the bound
$$
\| \mathbf{Mat}_{j - 1}(Z\times_{1}\left(\hat{v}_{1} - \tilde{v}_{1}\right))\|_{2}\leq \sup_{\substack{u_{1}\in \mathbb{S}^{n_{1} - 1},u_{2}\in \mathbb{S}^{n_{j} - 1},\\ u_{3}\in \mathbb{S}^{N_{2} / n_{j} - 1}}}\left(\mathcal{Z}\times_{1}u_{1}\times_{2}u_{2}\times_{3}u_{3}\right)\cdot \| \hat{v}_{1} - \tilde{v}_{1}\|_{2}
$$
and Theorem 1 of [23],
$$
\begin{array}{l} \| \mathbf {M a t} _ {j - 1} (X \times_ {1} (\hat {v} _ {1} - \tilde {v} _ {1})) \| _ {2} \leq \lambda \| \hat {v} _ {1} - \tilde {v} _ {1} \| _ {2} \| \mathbf {M a t} _ {j - 1} (v _ {2} \otimes \dots \otimes v _ {k}) \| _ {2} + \| \mathbf {M a t} _ {j - 1} (Z \times_ {1} (\hat {v} _ {1} - \tilde {v} _ {1})) \| _ {2} \\ \lesssim \left(\lambda + \left(N _ {2} / n _ {j}\right) ^ {1 / 2}\right) \| \hat {v} _ {1} - \tilde {v} _ {1} \| _ {2}, \tag {37} \\ \end{array}
$$
almost surely. Thus, using the Davis-Kahan theorem (Theorem 4 of [25]), (34), and (37), we have
$$
\left\| \hat {v} _ {j} - \tilde {v} _ {j} \right\| _ {2} \lesssim \frac {\lambda \left(\lambda + \left(N _ {2} / n _ {j}\right) ^ {1 / 2}\right) \| \hat {v} _ {1} - \tilde {v} _ {1} \| _ {2}}{\lambda^ {2}} \xrightarrow {a . s .} 0, \tag {38}
$$