SlowGuess/NeurIPS_2023_OCR huggingface dataset:
David L. Donoho
donoho@stanford.edu
Michael J. Feldman
feldman6@stanford.edu
Many applications seek to recover low-rank approximations of noisy tensor data. We consider several practical and effective matricization strategies which construct specific matrices from such tensors and then apply spectral methods; the strategies include tensor unfolding, partial tracing, power iteration, and recursive unfolding. We settle the behaviors of unfolding and partial tracing, identifying sharp thresholds in signal-to-noise ratio above which the signal is partially recovered. In particular, we extend previous results to a much larger class of tensor shapes where axis lengths may be different. For power iteration and recursive unfolding, we prove that under conditions where previous algorithms partially recovery the signal, these methods achieve (asymptotically) exact recovery. Our analysis deploys random matrix theory to obtain sharp thresholds which elude perturbation and concentration bounds. Specifically, we rely upon recent disproportionate random matrix results, which describe sequences of matrices with diverging aspect ratio.
Tensors—multi-way arrays—and tensor methods are fundamental to modern data analysis. Data given by three or more indices are increasingly the focus in diverse areas including signal and image processing [9, 19, 22], while high-order moments represented by tensors are of interest in problems such as community detection and learning latent variable models [1, 2, 8].
The spiked tensor model, introduced by Montanari and Richard [20], is a simple statistical model of tensor data with latent low-rank structure. Observed data  $X$  of dimensions  $n_1 \times n_2 \times \dots \times n_k$  are the sum of a low-rank tensor and noise:
$$
X _ {i _ {1}, i _ {2}, \dots , i _ {k}} = \lambda v _ {1, i _ {1}} v _ {2, i _ {2}} \dots v _ {k, i _ {k}} + Z _ {i _ {1}, i _ {2}, \dots , i _ {k}}, \quad j \in [ k ], i _ {j} \in [ n _ {j} ], \tag {1}
$$
where  $k \geq 2$  is the tensor order,  $\lambda$  is a signal strength,  $v_{j} \in \mathbb{S}^{n_{j} - 1}$ ,  $j \in [k]$ , are unit vectors, and  $Z$  is a noise tensor. Noise entries are assumed to be independent standard Gaussians:
$$
Z _ {i _ {1}, i _ {2}, \dots , i _ {k}} \stackrel {{i. i. d}} {{\sim}} \mathcal {N} (0, 1), \qquad \qquad j \in [ k ],   i _ {j} \in [ n _ {j} ].
$$
Estimation of  $v_{1}, \ldots, v_{k}$  and the low-rank component of  $X$  generalizes to tensors the problem of low-rank matrix approximation—principal component analysis (PCA)—and is therefore known as tensor PCA.
This model reduces for  $k = 2$  to the spiked matrix model, with maximum likelihood estimators of  $v_{1}$  and  $v_{2}$  given by the first left and right singular vectors  $\hat{v}_{1}$  and  $\hat{v}_{2}$  of  $X$ , respectively. For  $k \geq 3$ , however, computation of the maximum likelihood estimators of the low rank component amounts to solving an NP hard problem (Hillar and Lim [14]),
$$
\sup  _ {j \in [ k ], u _ {j} \in \mathbb {S} ^ {j - 1}} \sum_ {i _ {1}, \dots , i _ {k}} X _ {i _ {1}, i _ {2}, \dots , i _ {k}} u _ {1, i _ {1}} \dots u _ {k, i _ {k}}, \tag {2}
$$
necessitating alternate, efficient algorithms.
This paper studies matricization-based approaches, which convert tensors to matrices (by reshaping or by contracting constructions to be described) and then apply spectral methods. We assume the spiked tensor model in a high-dimensional asymptotic framework in which each array dimension  $n_1, \ldots, n_k$  tends to infinity; we make no assumptions on their relative rates. While previous analyses of tensor PCA algorithms have assumed order  $k = 3$ , supersymmetry  $v_1 = v_2 = \dots = v_k$ , or hypercubical format  $n_1 = n_2 = \dots = n_k$ , we permit tensors of diverse dimensions  $n_j$  and unrelated  $v_j$ .
In this setting, we fully analyze tensor unfolding or reshaping, a widespread technique [12, 19]. Additionally, we analyze the partial tracing approach of Hopkins et al. [16], discovering its asymptotic equivalence to unfolding in performance—an improvement over previous analysis. We identify sharp thresholds in signal-to-noise ratio above which these algorithms partially recover the signal, and provide exact formulas for their limiting performance. Finally, we study generalizations of the power iteration and recursive unfolding algorithms of [20]. Above the same thresholds characterizing partial recovery of unfolding and partial tracing, these algorithms achieve (asymptotically) exact signal recovery. In other words, for signal-to-noise ratios such that unfolding or partial tracing partially recover the signal, power iteration and recursive unfolding exactly recover the signal.
Our approach relies upon fundamental and penetrating random matrix theoretic (RMT) results. Precise analysis of tensor PCA algorithms, including pinpointing of phase transitions and sharp performance quantification as provided here, is not possible purely via more standard tools in theoretical machine learning, such as matrix concentration and vector perturbation bounds. Specifically, we rely upon recent disproportionate random matrix results (introduced in Section 1.3), which describe sequences of matrices with diverging aspect ratio.
This work concisely demonstrates in the context of tensor PCA the advantages of an RMT-backed approach. Indeed, application of spiked matrix results yields simple and elegant theorems and proofs which are easily read. For example, the analysis of the partial tracing in [16], involving challenging matrix concentration and perturbation calculations, is reduced to a few careful lines. Though our results are admittedly purely asymptotic, there are no hidden constants or logarithmic factors. Even for modest-sized tensors, simulations demonstrate close agreement with theory.
1.2 Assumptions and Notation
Without loss of generality, we assume  $\lambda \geq 0$  and  $n_1 \leq n_2 \leq \dots \leq n_k$ , taking  $n_1$  to be the "fundamental" problem index; that is  $n_j = n_j(n_1)$ ,  $j \in [k]$ . We say an estimator  $\hat{v}_j$  partially recovers  $v_j$  if almost surely,
$$
\liminf_{n_{1}\to \infty}|\langle v_{j},\hat{v}_{j}\rangle | > 0,
$$
and  $\hat{v}_j$  exactly recovers  $v_j$  if  $|\langle v_j, \hat{v}_j \rangle| \xrightarrow{a.s.} 1$ . In words, partial recovery demands a positive limit for the cosine similarity, while exact recovery requires asymptotically perfect cosine similarity.
We make the rank-one signal assumption in model (1) purely for expository efficiency; the spectral algorithms in this paper naturally generalize to rank- $r$  spiked tensors of the form
$$
X _ {i _ {1}, i _ {2}, \dots , i _ {k}} = \sum_ {i = 1} ^ {r} \lambda_ {r} v _ {1, i _ {1}} ^ {(i)} v _ {2, i _ {2}} ^ {(i)} \dots v _ {k, i _ {k}} ^ {(i)} + Z _ {i _ {1}, i _ {2}, \dots , i _ {k}}, \quad j \in [ k ], i _ {j} \in [ n _ {j} ] \tag {3}
$$
where  $r \geq 1$  is the rank and  $\{v_j^{(1)}, \ldots, v_j^{(r)}\} \subset \mathbb{S}^{n_j - 1}$  are orthonormal sets,  $j \in [k]$ . Such tensor decompositions are unique by Kruskal's theorem [18]. Where we estimate  $v_j^{(1)}$  by the first right singular vector of an appropriate matrix  $M_j$ ,  $v_j^{(2)}, \ldots, v_j^{(r)}$  may be estimated by the subsequent right singular vectors, with analogous theoretical guarantees holding (that is,  $v_j^{(i)}$  is the right singular vector associated with the  $i$ -th largest singular value of  $M_j$ ).
We denote by  $\otimes$  both the tensor outer product and Kronecker product (the latter is simply a vectorization of the former); it is clear from context which is meant. Let  $\times_{j}$  denote multiplication between a tensor along the  $j$ -th axis and a vector of conformable dimension. The notation  $a(n_1) \lesssim b(n_1)$
means  $a(n_1) \leq C b(n_1)$  for a constant  $C > 0$  and all sufficiently large  $n_1$ , and  $a(n_1) \asymp b(n_1)$  means  $a(n_1) \lesssim b(n_1) \lesssim a(n_1)$ .
Finally, we introduce tensor slices. Let  $T \in \mathbb{R}^{n_1 \times \dots \times n_k}$  and fix  $\ell \in [k]$ . The set  $\{T_{i_1, \ldots, i_l} : j \in [l], i_j \in [n_j]\}$  contains the slices of  $X$  of order  $k - \ell$ . Slices satisfy  $T_{i_1, \ldots, i_\ell} \in \otimes_{j = \ell + 1}^k \mathbb{R}^{n_j}$  and have entries
$$
\left(T _ {i _ {1}, \dots i _ {\ell}}\right) _ {i _ {l + 1}, \dots , i _ {k}} = T _ {i _ {1}, \dots , i _ {k}}, \quad j \in \{\ell + 1, \dots , k \}, i _ {j} \in [ n _ {j} ].
$$
1.3 Spiked Matrix Model
The behavior of matricization-based algorithms fundamentally derives from spectral properties of the spiked matrix model,
$$
X _ {i _ {1}, i _ {2}} = \lambda v _ {1, i _ {1}} v _ {2, i _ {2}} + Z _ {i _ {1}, i _ {2}}, \quad i _ {1} \in [ n _ {1} ], i _ {2} \in [ n _ {2} ]. \tag {4}
$$
This model is extensively studied in random matrix theory, particularly under proportional growth, where  $n_1$  and  $n_2$  are of comparable magnitude:
$$
n _ {1}, n _ {2} \rightarrow \infty , \quad \gamma_ {n _ {1}} = \frac {n _ {1}}{n _ {2}} \rightarrow \gamma > 0.
$$
Under proportional growth, the sample covariance matrix  $XX^{\top} / n_{2}$  does not converge to the identity (its expectation), and the leading left and right singular vectors  $\hat{v}_{1}$  and  $\hat{v}_{2}$  of  $X$  are inconsistent estimators of  $v_{1}$  and  $v_{2}$ , respectively. We highlight the following results of Benaych-Georges and Rao Nadakuditi [7], who establish formulas for the limiting misalignment of  $\hat{v}_{1}$  and  $\hat{v}_{2}$ :
Lemma 1.1. Let  $\hat{v}_1$  and  $\hat{v}_2$  denote the leading left and right singular vectors of  $X$ , respectively, and define
$$
c ^ {2} (\tau , \gamma) = \left\{ \begin{array}{l l} 1 - \frac {\gamma (1 + \tau^ {2})}{\tau^ {2} (\tau^ {2} + \gamma)} & \tau > \gamma^ {1 / 4} \\ 0 & \tau \leq \gamma^ {1 / 4} \end{array} \right..
$$
Under  $\lambda = \tau (1 + o(1))\sqrt{n_2}$ , where  $\tau \geq 0$  is fixed, and  $\gamma_{n_1} \to \gamma \in (0,1]$
$$
\left| \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \right| ^ {2} \xrightarrow {a . s .} c ^ {2} (\tau , \gamma), \quad \left| \left\langle v _ {2}, \hat {v} _ {2} \right\rangle \right| ^ {2} \xrightarrow {a . s .} c ^ {2} \left(\tau \gamma^ {- 1 / 2}, \gamma^ {- 1}\right). \tag {5}
$$
Recently, Ben Arous et al. [5] and Feldman [13] independently studied the spiked matrix model under disproportional growth, where  $\gamma_{n_1} \to 0$  or  $\gamma_{n_1} \to \infty$ . Transposing  $X$  in case  $\gamma_{n_1} \to \infty$ , we assume without loss of generality that  $\gamma_{n_1} \to 0$ . Phase transitions for the left and right singular vectors no longer coincide;  $v_1$  is reliably estimated at signal strengths much weaker than  $\lambda \gtrsim \sqrt{n_2}$ . Drawing upon results in [5], [13], and [7], we have the following disproportionate analogs of Lemma 1.1:
Lemma 1.2. Let  $\hat{v}_1$  and  $\hat{v}_2$  denote the leading left and right singular vectors of  $X$ , respectively, and define
$$
\bar {c} ^ {2} (\tau) = (1 - \tau^ {- 4}) _ {+}, \quad \vec {c} ^ {2} (\tau) = \frac {\tau^ {2}}{1 + \tau^ {2}}.
$$
Under  $\lambda = \tau (1 + o(1))(n_{1}n_{2})^{1 / 4}$ , where  $\tau \geq 0$  is fixed, and  $\gamma_{n_1}\rightarrow 0$ ,
$$
\left| \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \right| ^ {2} \xrightarrow {a . s .} \bar {c} ^ {2} (\tau), \quad \left| \left\langle v _ {2}, \hat {v} _ {2} \right\rangle \right| ^ {2} \xrightarrow {a . s .} 0, \tag {6}
$$
while under  $\lambda = \tau (1 + o(1))\sqrt{n_2}$
$$
\left| \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \right| ^ {2} \xrightarrow {a . s .} 1, \quad \left| \left\langle v _ {2}, \hat {v} _ {2} \right\rangle \right| ^ {2} \xrightarrow {a . s .} \vec {c} ^ {2} (\tau). \tag {7}
$$
Limits (7) are corollaries of Theorems 2.9 and 2.10 of [7]. We note that [5] requires  $n_2$  is polynomially bounded in  $n_1$ . This assumption, however, is necessary only to establish non-asymptotic bounds; the almost-sure results in Lemma 1.2 hold as  $\gamma_{n_1} \to 0$  arbitrarily rapidly. While not explicitly stated in these references, it is easily verified that under proportional growth,  $\lim \inf \lambda / \sqrt{n_2} > \gamma^{1/4}$  implies partial recovery of  $v_1$  and  $v_2$ . Analogous statements hold for disproportional growth.
2 Tensor Unfolding
We study a general unfolding procedure that permits (1) tensors of general, unequal axis lengths and (2) unfolding along arbitrary sets of axes.
Let  $N_{j} = \prod_{\ell = j}^{k}n_{\ell}$ $j\in [k]$  , and for  $S\subset [k],S\neq \emptyset$  , let  $N(S) = \prod_{\ell \in S}n_{\ell}$  . We define a map  $\mathrm{Mat}_S:\mathbb{R}^{n_1\times \dots \times n_k}\to \mathbb{R}^{(N_1 / N(S))\times N(S)}$  as follows: for indices  $i_j\in [n_j],j\in [k]$
$$
a = 1 + \sum_ {j \in [ k ] \backslash S} \left(i _ {j} - 1\right) \prod_ {\ell \in [ k ] \backslash S: j <   \ell} n _ {\ell}, \quad b = 1 + \sum_ {j \in S} \left(i _ {j} - 1\right) \prod_ {\ell \in S: j <   \ell} n _ {\ell} \tag {8}
$$
$$
\left[ \operatorname {M a t} _ {S} (T) \right] _ {a, b} = T _ {i _ {1}, i _ {2}, \dots , i _ {k}}. \tag {9}
$$
When unfolding along a single axis, we write  $\mathrm{Mat}_j = \mathrm{Mat}_{\{j\}}$ . In this case, (8) reduces to
$$
a = 1 + \sum_ {\ell = 1} ^ {j - 1} \left(i _ {\ell} - 1\right) \frac {N _ {\ell + 1}}{n _ {j}} + \sum_ {\ell = j + 1} ^ {k} \left(i _ {\ell} - 1\right) N _ {\ell + 1} \tag {10}
$$
(taking  $N_{k+1} = 1$ ) and  $b = i_j$ . The unfolded matrix  $\operatorname{Mat}_j(X)$  is a spiked matrix with aspect ratio  $n_j^2 / N_1$ :
$$
\operatorname {M a t} _ {j} (X) = \lambda \left(v _ {1} \otimes \dots \otimes v _ {j - 1} \otimes v _ {j + 1} \otimes \dots \otimes v _ {k}\right) v _ {j} ^ {\top} + \operatorname {M a t} _ {j} (Z).
$$
Similarly,  $\mathrm{Mat}_S(X)$  is a spiked matrix with aspect ratio  $N(S)^2 / N_1$ :
$$
\operatorname {M a t} _ {S} (X) = \lambda \left(\otimes_ {j \in [ k ] \backslash S} v _ {j}\right) \left(\otimes_ {j \in S} v _ {j}\right) ^ {\top} + \operatorname {M a t} _ {S} (Z). \tag {11}
$$
We now review prior results on unfolding. For  $v_{1} = \dots = v_{k}$ , Montanari and Richard consider the unfolding  $S = \{1, \ldots, \lfloor k / 2 \rfloor\}$ , estimating  $\otimes^{\lfloor k / 2 \rfloor} v_{1}$  by the first right singular vector of  $\operatorname{Mat}_S(X)$ . They prove that  $\otimes^{\lfloor k / 2 \rfloor} v_{1}$  is (partially) recovered for  $\lambda \gtrsim n_1^{\lceil k / 2 \rceil / 2}$ , and conjecture sufficiency of  $\lambda \gtrsim n_1^{k / 4}$  (note that these bounds differ only for odd  $k$ ). Theorem 5.8 of Hopkins et al. [15] completes this conjecture for  $k = 3$ , establishing sufficiency of  $\lambda \geq (1 + \varepsilon) n_1^{3 / 4}$ .
Complete analysis of unfolding—handling tensors of arbitrary order and asymmetric dimensions—relies on results in Section 1.3, which reveal (1) necessary and sufficient thresholds for recovery and (2) the exact limiting performance of estimates. In addition, spiked matrix results yield the exact limiting cosine similarity of unfolding procedures, not obtained in [20] or [15]. The recent disproportional results summarized in Lemma 1.2 are crucial as unfoldings such as  $\mathrm{Mat}_j(X)$ ,  $j\in [k - 1]$ , have limiting aspect ratio zero (recall that  $n_1\leq \dots \leq n_k$ ). Depending on the relative growth rates of  $n_1,\ldots ,n_k$ , unfoldings such as  $\mathrm{Mat}_k(X)$  may fall under disproportional "tall" growth ( $n_k^2 /N_1\to 0$ ), proportional growth ( $n_k^2 /N_1\succ 1$ ), or disproportional "wide" growth ( $n_k^2 /N_1\to \infty$ ). Our analysis is similar to that of Ben Arous et al. [5], which permits arbitrary order  $k$  yet assumes  $n_1 = \dots = n_k$ .
Theorem 2.1. Let  $S \subset [k]$ . If  $N(S)^2 / N_1 \to 0$ , the first right singular vector  $w$  of the unfolding  $\mathrm{Mat}_S(X)$  partially recovers  $\otimes_{j \in S} v_j$  if and only if  $\lim \inf \lambda / N_1^{1/4} > 1$ . In particular, for  $\lambda = \tau(1 + o(1)) N_1^{1/4}$  with  $\tau \geq 0$  fixed,
$$
| \langle \otimes_ {j \in S} v _ {j}, w \rangle | ^ {2} \xrightarrow {a . s .} \overleftarrow {c} ^ {2} (\tau). \tag {12}
$$
If  $N(S)^2 / N_1 \asymp 1$ ,  $\otimes_{j \in S} v_j$  is partially recovered if and only if  $\lim \inf \lambda / N_1^{1/4} > 1$ . For  $\gamma_{n_1} \to \gamma$  and  $\lambda = \tau(1 + o(1)) N_1^{1/4}$ ,
$$
\left| \langle \otimes_ {j \in S} v _ {j}, w \rangle \right| ^ {2} \xrightarrow {a . s .} c ^ {2} \left(\tau \gamma^ {1 / 4}, \gamma\right). \tag {13}
$$
Finally, if  $N(S)^2 / N_1 \to \infty$ ,  $\otimes_{j \in S} v_j$  is partially recovered if and only if  $\lim \inf \lambda / \sqrt{N(S)} > 0$ . For  $\lambda = \tau(1 + o(1)) \sqrt{N(S)}$ ,
$$
\left| \langle \otimes_ {j \in S} v _ {j}, w \rangle \right| ^ {2} \xrightarrow {a . s .} \vec {c} ^ {2} (\tau). \tag {14}
$$
All proofs are deferred to the supplement. As the recovery threshold is identical across all unfoldings  $S$  such that  $N(S)^2 / N_1 \lesssim 1$ , we propose the following simple algorithm to estimate  $v_j$ , which may be iterated for  $j \in [k]$ :
Algorithm 1 Tensor unfolding
Input:  $X, j$   
return first right singular vector of  $\operatorname{Mat}_j(X)$
Corollary 2.1.1. Algorithm 1 partially recovers  $v_{1}, \ldots, v_{k-1}$  if and only if  $\lim \inf \lambda / N_{1}^{1/4} > 1$ . Vector  $v_{k}$ , corresponding to the largest dimension, is partially recovered as well if  $n_{k}^{2} / N_{1} \lesssim 1$ . On the other hand, if  $n_{k}^{2} / N_{1} \to \infty$ ,  $v_{k}$  is recovered if and only if  $\lim \inf \lambda / \sqrt{n_{k}} > 0$ .
Figure 1: Simulations of tensor unfolding with  $k = 3$ , supersymmetric signal  $v_{1} = v_{2} = v_{3}$ , and  $n_1\in \{400,800,1200\}$ . Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The dashed line is the theoretical limit  $\overline{c} (\tau)$ , which agrees closely. Below the phase transition located at  $\tau = 1$ ,  $\hat{v}_1$  is approximately uniformly distributed on the surface of the unit sphere, so  $|\langle v_1,\hat{v}_1\rangle | = O(n^{-1 / 2})$ .
Figure 2: Simulations of tensor unfolding with  $k = 3$ ,  $n_1 = n_2 = 50$ , and  $n_3 = 10000$  (left) or  $n_3 = 20000$  (right). Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The dashed lines are theoretical limits based on Theorem 2.1, which agree closely. As  $n_3 / (n_1n_2) = O(1)$ , we compare  $|\langle v_1,\hat{v}_1\rangle|$  and  $|\langle v_2,\hat{v}_2\rangle|$  to  $\bar{c}(\tau)$  and  $|\langle v_3,\hat{v}_3\rangle|$  to  $c(\tau(N_1/n_3^2)^{1/4}, N_1/n_3^2) = c(\lambda/\sqrt{n_3}, N_1/n_3^2)$ .
3 Partial Tracing
Hopkins et al. [16] propose the partial tracing method for tensors of order  $k = 3$  as an alternative to unfolding. Their approach generalizes to orders  $k \geq 3$ , although it inherently relies on supersymmetry
of the signal,  $v_{1} = \dots = v_{k}$ . The partial tracing operator  $\mathrm{Tr}_k: \otimes^k\mathbb{R}^n \mapsto \mathbb{R}^{n\times n}$  constructs a matrix by linearly combining tensor slices with partial trace weights:
$$
\operatorname {T r} _ {k} (T) = \sum_ {i _ {1}, \dots , i _ {k - 2} \in [ n ]} \operatorname {t r} \left(T _ {i _ {1}, \dots , i _ {k - 2}}\right) T _ {i _ {1}, \dots , i _ {k - 2}}. \tag {15}
$$
This operation of reducing the order of a tensor by linear combinations of slices is called contraction in tensor analysis.
Algorithm 2 Partial tracing
Input:  $X$
return first right singular vector of  $\operatorname{Tr}_k(X)$
For  $k = 3$ , the runtime of Algorithm 2 is  $O(n^{3})$ , while that of unfolding is  $O(n^{3} \log n)$  (see Table 3 of [16]). Hopkins et al. established a bound on the recovery threshold of partial tracing which is worse than that of unfolding by a logarithmic factor:  $\lambda \gtrsim n^{3/4} (\log n)^{1/2}$ . We eliminate the logarithmic factor here: while delivering improvements in runtime, Algorithm 2 is asymptotically equivalent to unfolding in recovery performance.
Theorem 3.1. The first right singular vector  $\hat{v}_1$  of the partial trace matrix  $\operatorname{Tr}_k(X)$  partially recovers  $v_1$  if and only if  $\lim \inf \lambda / n_1^{k/4} > 1$ . For  $\lambda = \tau(1 + o(1))n_1^{k/4}$ , where  $\tau$  is fixed,
$$
\left| \left\langle v _ {1}, \hat {v} _ {1} \right\rangle \right| ^ {2} \xrightarrow {a . s .} \overleftarrow {c} ^ {2} (\tau). \tag {16}
$$
Under  $n_1 = \dots = n_k$ , the behavior of unfolding is given by (12), matching (16) exactly. Thus, unfolding and partial tracing are asymptotically equivalent in performance.
Figure 3: Simulations of partial tracing with  $k = 3$ , supersymmetric signal  $v_{1} = v_{2} = v_{3}$ , and  $n_1\in \{400,800,1200\}$ . Solid lines display empirical cosine similarities (each point is the average of 50 realizations). The dashed line is the theoretical limit  $\overleftarrow{c} (\tau)$ , which agrees closely. Note the similarity to Figure 1.
4 Exact Recovery
Montanari and Richard [20] consider a two-step recursive unfolding procedure that remarkably exactly recovers  $v_{1}$  above the threshold of unfolding, assuming even order  $k$ , hypercubical shape  $n_1 = \dots = n_k$ , and a supersymmetric signal,  $v_{1} = \dots = v_{k}$ . In this section, we prove that exact recovery is possible for tensors of arbitrary axis lengths. We consider generalizations of the power iteration and recursive unfolding algorithms of [20] (initialized via tensor unfolding). Tensor power iteration is previously studied in [17] (assuming a supersymmetric signal) and [21] (assuming
$n_1 \asymp n_2 \asymp \dots \asymp n_k)$ , though these works assume the initial iterate is independent of  $X$ , precluding initialization via unfolding.
In our more general setting, we find that unfolding-initialized power iteration achieves exact recovery above the partial recovery threshold of unfolding, established in Corollary 2.1.1 (assuming  $n_k^2 / N_1 \to 0$ ). Recursive unfolding exactly recovers  $v_1, \ldots, v_{k-1}$  with no limitation on the growth rate of  $n_k$ . Power iteration, which requires initial "warm" estimates of  $v_1, \ldots, v_k$ , may fail when  $n_k^2$  and  $N_1$  are comparable. Recursive unfolding, on the other hand, does not rely upon an initial estimate of  $v_k$  to recover  $v_1, \ldots, v_{k-1}$  exactly.
Algorithm 3 Power iteration
Input:  $X$  , initial estimates  $\hat{v}_1,\dots ,\hat{v}_k$  from Algorithm 1   
Iterate until convergence: for  $j$  from 1 to  $k$  ..  $w_{j}\gets X\times_{1}\hat{v}_{1}\dots \times_{j - 1}\hat{v}_{j - 1}\times_{j + 1}\hat{v}_{j + 1}\dots \times_{k}\hat{v}_{k}$ $\hat{v}_j\gets w_j / \| w_j\| _2$  end for   
return  $\hat{v}_1,\ldots ,\hat{v}_k$
We prove that exact recovery is achieved in a single iteration of the outer loop. In practice, iterating until estimates converge is beneficial.
Theorem 4.1. If  $\lim \inf \lambda / N_1^{1/4} > 1$  and  $n_k^2 / N_1 \to 0$ , Algorithm 3 recovers  $v_1, \ldots, v_k$  exactly: denoting by  $\hat{v}_1, \ldots, \hat{v}_k$  the estimates after a single iteration of the outer loop,
$$
| \langle v _ {j}, \hat {v} _ {j} \rangle | \xrightarrow {a . s .} 1, \quad j \in [ k ]. \tag {17}
$$
Estimating the signal strength  $\lambda$  by  $\hat{\lambda} = \langle X, \hat{v}_1 \otimes \dots \otimes \hat{v}_k \rangle$ , we have
$$
\lambda^ {- 1} \| \hat {\lambda} \hat {v} _ {1} \otimes \dots \otimes \hat {v} _ {k} - \lambda v _ {1} \otimes \dots \otimes v _ {k} \| _ {F} \xrightarrow {a . s .} 0. \tag {18}
$$
We next consider a recursive unfolding procedure. As stated above, recursive unfolding is guaranteed to recover  $v_{1}, \ldots, v_{k-1}$  exactly if  $\lim \inf \lambda / N_{1}^{1/4} > 1$ ; no bound is needed on the growth rate of  $n_{k}$  as in Theorem 4.1. In practice, it is beneficial to iterate Algorithm 4.
Algorithm 4 Recursive unfolding
Input:  $X$ , initial estimates  $\hat{v}_1, \ldots, \hat{v}_k$  from Algorithm 1  
for  $j$  from 2 to  $k$ :  
 $\hat{v}_j \gets$  first right singular vector of  $\mathrm{Mat}_{j-1}(X \times_1 \hat{v}_1)$   
end for  
 $\hat{v}_1 \gets$  first right singular vector of  $\mathrm{Mat}_1(X \times_2 \hat{v}_2)$   
return  $\hat{v}_1, \ldots, \hat{v}_k$