SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem E.1. Given  $n, r \in \mathbb{Z}$ , if a t-round adaptive algorithm that performs  $k \geq nr$  general linear measurements in each round is such that for every  $n \times n$  matrix  $A$ , the algorithm outputs a rank  $r$  matrix  $B$  such that with probability  $\geq 99/100$ ,  $\| A - B \|_2 \leq 2\sigma_{r+1}(A)$ , then  $t \geq c\frac{\log(n^2 / k)}{\log\log(n)}$ .
We prove the following helper lemma that we use in our proof.
Lemma E.2. If  $A$  is a rank- $r$  matrix and  $B$  is an arbitrary matrix satisfying  $\| A - B \|_2 \leq t$ , then  $\| A - [B]_r \|_2 \leq 2t$ , where  $[B]_r$  denotes the best rank- $r$  approximation of the matrix  $B$  in operator norm.
Proof. As  $A$  has rank at most  $r$ , we have  $t \geq \| A - B \|_2 \geq \| B - [B]_r \|_2$ . Thus,  $\| A - [B]_r \|_2 \leq \| A - B \|_2 + \| B - [B]_r \|_2 \leq 2t$ .
Proof of Theorem E.1. By a standard reduction, it suffices to show that there is a distribution on  $n \times n$  matrices for which any  $t$ -round deterministic algorithm which makes  $k$  general linear measurements in each round and outputs a 2-approximate spectral rank approximation satisfies  $t \geq c\log (n^2 /k) / \log \log (n)$ .
Consider the  $n \times n$  random matrix  $M = G + s\mathbf{u}\mathbf{v}^{\mathrm{T}}$  for  $s = 500 / \sqrt{n}$ . Suppose there is a deterministic  $t$ -round algorithm  $\mathsf{Alg}$  that makes  $k$  linear measurements and outputs rank- $r$  matrix  $K = \mathsf{Alg}(M)$  such that with probability  $\geq 99/100$ ,
$$
\| M - K \| _ {2} \leq 2 \sigma_ {r + 1} (M) \leq 2 \| G \| _ {2}.
$$
This implies that  $\| s\pmb{u}\pmb{v}^{\mathsf{T}} - K\|_{2}\leq 3\| \pmb {G}\|_{2}$  and  $\| \pmb {u}\pmb{v}^{\mathrm{T}} - K / s\|_{2}\leq 3\| \pmb {G}\|_{2} / s$
With probability  $\geq 1 - \exp (-\Theta (n))$ , we have  $2\sqrt{n}\geq \| \pmb {u}\|_{2},\| \pmb {v}\|_{2}\geq \sqrt{n} /2$  and  $\| G\| _2\leq 3\sqrt{n}$  simultaneously. By a union bound, we have that with probability  $\geq 0.98$ ,  $\| \pmb {u}\pmb{v}^{\mathsf{T}} - K / s\|_{2}\leq (9 / 500)n$
By Lemma E.1, we have  $\| \pmb{u}\pmb{v}^{\top} - [K / s]_{1}\|_{2}\leq (9 / 250)n$  which implies that  $\| \pmb {u}\otimes \pmb {v} - \mathrm{vec}([K / s]_1)\| _2^2 =$ $\| \pmb {u}\pmb{v}^{\top} - [K / s]_{1}\|_{\mathbb{F}}^{2}\leq 2\| \pmb {u}\pmb{v}^{\top} - [K / s]_{1}\|_{2}^{2}\leq 2(9 / 250)^{2}n^{2}$  where we used the fact that the matrix  $\pmb {u}\pmb{v}^{\top} - [K / s]_{1}$  has rank at most 2. Let  $q\in \mathbb{R}^{n^2} = \operatorname {vec}([K / s]_1) / \| [K / s]_1\|_{\mathbb{F}}$  be a unit vector. We can show that
$$
\langle q, \boldsymbol {u} \otimes \boldsymbol {v} \rangle^ {2} \geq n ^ {2} / 6 4
$$
by a simple application of the triangle inequality. Thus, using Alg we can construct a deterministic  $t + 1$  round algorithm such that the squared projection of  $\mathbf{u} \otimes \mathbf{v}$  onto the query space is at least  $\geq n^2 / 64$  with probability  $\geq 98 / 100$ . By (4), we have that  $t + 1 \geq c' \log(n^2 / k) / \log \log(n)$  and therefore we have  $t \geq c \log(n^2 / k) / \log \log(n)$  for a small enough constant  $c$ .
The following theorem states our lower bound for algorithms which output a spectral LRA for each matrix with high probability.
Theorem E.3. Given  $n, r \in \mathbb{Z}$ , if a t-round adaptive algorithm that performs  $k \geq nr$  general linear measurements in each round is such that for every  $n \times n$  matrix  $A$ , the algorithm outputs a rank  $r$  matrix  $B$  such that with probability  $\geq 1 - 1 / \mathrm{poly}(n)$ ,  $\| A - B \|_2 \leq 2\sigma_{r + 1}(A)$ , then  $t \geq c\log (n^2 / k)$ .
Proof of Theorem E.3. The proof of this lemma is similar to that of Theorem E.1. The existence of a randomized  $t$  round algorithm that outputs a 2-approximate spectral norm LRA for every instance with probability  $\geq 1 - 1/\mathrm{poly}(n)$  implies the existence of a deterministic algorithm that outputs a 2-approximate spectral norm LRA with probability  $\geq 1 - 1/\mathrm{poly}(n)$  over the distribution of  $\pmb{G} + (\alpha/\sqrt{n})\pmb{u}\pmb{v}^{\top}$ . As in the proof of the Theorem E.1, this algorithm can be used to construct a  $t + 1$  round algorithm that with probability  $\geq 1 - 1/\mathrm{poly}(n)$  over the matrix  $\pmb{G} + (\alpha/\sqrt{n})\pmb{u}\pmb{v}^{\top}$ , computes a unit vector  $q$  satisfying
$$
\langle q, \boldsymbol {u} \otimes \boldsymbol {v} \rangle^ {2} \geq n ^ {2} / 6 4.
$$
Now, (5) implies that  $t + 1 \geq c \log(n^2 / k)$  for a small enough constant  $c$ .
For the random matrix  $M = G + (\alpha / \sqrt{n})\pmb{u}\pmb{v}^{\top}$  for a large enough constant  $\alpha$  considered in the above theorem, we have that  $(\sigma_1(M) / \sigma_2(M)) \geq 2$  with high probability. Now consider the random matrix
$$
M ^ {\prime} = \left[ \begin{array}{c c} M & 0 \\ 0 & 3 \sqrt {n} \alpha I _ {r - 1} \end{array} \right].
$$
With high probability, we have  $\sigma_r(M') = \sigma_1(M) \geq (\alpha/2)\sqrt{n}$  and  $\sigma_{r+1}(M') = \sigma_2(M) \leq 2\sqrt{n}$  implying  $\sigma_{r+1}(M') / \sigma_r(M') \leq 1/2$ . A proof similar to that of the above theorem now shows that algorithms using  $k \geq nr$  general linear measurements in each round and outputting a 2-approximate rank- $r$  LRA with probability  $\geq 1 - 1/\mathrm{poly}(n)$  for matrices  $M$  with  $\sigma_{r+1}(M) / \sigma_r(M) \leq 1/2$  have a lower bound of  $\Omega(\log(n^2/k))$  rounds. Moreover for  $k \geq Cnr$  for a large enough constant  $C$  and  $r = O(1)$ , the randomized subspace iteration algorithm starting with a subspace of  $k/n$ -dimensions, after  $O(\log(n^2/k))$  rounds outputs, with probability  $\geq 1 - 1/\mathrm{poly}(n)$ , a 2-approximate rank- $r$  spectral norm LRA for all matrices satisfying  $\sigma_{r+1}(M) / \sigma_r(M) \leq 1/2$ . See [12, Theorem 5.8] for a proof.
Note that the subspace iteration algorithm starting with a subspace of  $k / n$ -dimensions performs  $(k / n) \cdot n = k$  general linear measurements in each round. Thus for  $r = O(1)$  and  $k \geq Cnr$  for a large enough constant  $C$ , the lower bound of  $\Omega (\log (n^2 /k))$  rounds for high probability spectral norm LRA algorithms for "well-conditioned"  $(\sigma_{r + 1} / \sigma_r \leq 1 / 2)$  instances is tight up to constant factors and shows that general linear measurements offer no improvement over matrix-vector products for well-conditioned problems. Thus we have the following theorem.
Theorem E.4. Any randomized  $t$ -round algorithm that with probability  $\geq 1 - 1 / \mathrm{poly}(n)$  outputs a 2-approximate rank-  $r$  spectral norm LRA for any arbitrary matrix  $A$  satisfying  $\sigma_r(A) / \sigma_{r + 1}(A) \geq 2$ , must have  $t = \Omega (\log (n^2 /k))$ , where  $k \geq nr$  is the number of linear measurements the algorithm makes in each round.
Moreover, for  $r = O(1)$ , the subspace iteration algorithm [12] matches the lower bound up to constant factors and outputs a 2-approximate spectral norm LRA for all such instances with probability  $\geq 1 - 1 / \mathrm{poly}(n)$  in  $O(\log (n^2 /k))$  rounds.
E.2 Symmetric Spectral Norm Low Rank Approximation
An interesting property of the hard distributions from previous works [28, 3] is that those distributions are supported on symmetric matrices and they hence obtain lower bounds for algorithms that work even only on symmetric instances. Although our hard distribution is non-symmetric, we can construct a distribution supported only on symmetric matrices and show lower bounds on algorithms using generalized linear queries for symmetric matrices.
Theorem E.5. Given  $n, r \in \mathbb{Z}$ , if a t-round adaptive algorithm that performs  $k \geq nr$  general linear measurements in each round is such that for every  $n \times n$  matrix  $A$ , the algorithm outputs a rank  $r \geq 2$  matrix  $B$  such that with probability  $\geq 99/100$ ,  $\| A - B \|_2 \leq 2\sigma_{r+1}(A)$ , then  $t \geq c\frac{\log(n^2 / k)}{\log\log n}$ .
Proof. Consider the  $2n \times 2n$  random matrix  $M'$  defined as
$$
M ^ {\prime} = \left[ \begin{array}{c c} 0 _ {n \times n} & M \\ M ^ {\mathsf {T}} & 0 _ {n \times n} \end{array} \right]
$$
where  $M = G + s u v^{\top}$  for  $s = 500 / \sqrt{n}$  and all coordinates of  $G, u, v$  are independent standard Gaussian random variables. We have that for  $i \in [n]$ ,  $\sigma_{2i-1}(M') = \sigma_{2i}(M') = \sigma_i(M)$  and that any arbitrary  $k$  generalized linear measurements of the matrix  $M'$  can be simulated using  $2k$  general linear measurements of  $M$ . Now suppose an algorithm that uses  $k$  general linear measurements in each round outputs a rank  $r \geq 2$  matrix  $K$  satisfying
$$
\| \boldsymbol {M} ^ {\prime} - K \| _ {2} \leq 2 \sigma_ {r + 1} (\boldsymbol {M} ^ {\prime}) \leq 2 \sigma_ {1} (\boldsymbol {M}).
$$
Let the matrix  $K$  be of the form
$$
K = \left[ \begin{array}{c c} K _ {1} & K _ {2} \\ K _ {3} & K _ {4} \end{array} \right]
$$
As any sub-matrix of  $K$  has rank at most that of  $K$ , we obtain that  $K_{2}$  is a rank- $r$  matrix satisfying
$$
\left\| \boldsymbol {M} - K _ {2} \right\| _ {2} \leq 2 \sigma_ {1} (\boldsymbol {M}).
$$
Thus, the existence of a  $t$ -round deterministic algorithm that uses  $k$  generalized queries in each round and outputs a constant factor approximation of rank  $r$ , spectral norm LRA, for the random matrix  $M'$  with probability  $\geq 99/100$  implies the existence of a  $t$ -round algorithm that uses  $2k$  generalized queries in each round and outputs a constant factor approximation of rank- $r$  spectral norm LRA for the random matrix  $M$  with probability  $\geq 99/100$ . Now, as in the proof of Theorem E.1, we obtain that
$$
t \geq c \frac {\log (n ^ {2} / 2 k)}{\log \log n} \geq c ^ {\prime} \frac {\log (n ^ {2} / k)}{\log \log n}.
$$
We obtain the proof by appropriately scaling  $n$  in the statement.
The above theorem proves lower bounds for algorithms that solve constant factor rank- $r$  spectral norm Low Rank Approximation (LRA) for all  $r \geq 2$  even for symmetric instances. This leaves open just a lower bound on algorithms solving rank 1 spectral norm LRA for symmetric instances.
E.3 Schatten Norm Low Rank Approximation
We first note the following lemma which bounds the Schatten- $p$  norm of an  $n \times n$  Gaussian matrix.
Lemma E.6 (Equation 3.3 in [19]). If  $\pmb{G}$  is an  $n\times n$  matrix with independent entries sampled from  $N(0,1)$  and  $p\geq 2$ , then with probability  $\geq 9 / 10$ ,  $\| G\|_{\mathbb{S}_p}\leq 30n^{1 / 2 + 1 / p}$ .
We now state the theorem that shows lower bounds for Schatten norm low rank approximation.
Theorem E.7. Given  $n, r \in \mathbb{Z}$ , if a t-round adaptive algorithm that performs  $k \geq nr$  general linear measurements in each round is such that for every  $n \times n$  matrix  $A$ , the algorithm outputs a rank  $r$  matrix  $B$  such that with probability  $\geq 99/100$  we have
$$
\| A - B \| _ {\mathfrak {s} _ {p}} \leq 2 \min  _ {\text {r a n k - r X}} \| A - X \| _ {\mathfrak {s} _ {p}},
$$
$$
t \geq c \frac {\log (n ^ {2} / k)}{1 + (1 / p) \log (n) + \log \log (n)}.
$$
Proof. The proof is very similar to the proof of Theorem E.1. Let  $M = G + s\mathbf{u}\mathbf{v}^{\top}$  for  $s = \alpha/\sqrt{n}$  for  $\alpha$  to be chosen later. Let Alg be a  $t$ -round deterministic algorithm that outputs a 2-approximate Schatten  $p$ -norm low rank approximation for  $M$  with probability  $\geq 99/100$  over  $M$ . Then we have
$$
\left\| \boldsymbol {u} \boldsymbol {v} ^ {\mathsf {T}} - K / s \right\| \leq \frac {3 \| \boldsymbol {G} \| _ {S _ {p}}}{s}
$$
as  $\min_{\mathrm{rank} - rX}\| G + s\pmb{u}\pmb{v}^{\top} - X\|_{\mathbb{S}_p} \leq \| G\|_{\mathbb{S}_p}$ . Using Lemma E.6, with probability  $\geq 9/10$  over  $M$ , we have that
$$
\| \sum_ {i} \boldsymbol {u} _ {i} \boldsymbol {v} _ {i} ^ {\mathsf {T}} - K / s \| _ {\mathsf {S} _ {p}} \leq 9 0 n ^ {1 + 1 / p} / \alpha .
$$
By picking  $\alpha = Bn^{1 / p}$  for a large enough constant  $B$ , we obtain that
$$
\left\| \boldsymbol {u} \boldsymbol {v} ^ {\mathsf {T}} - K / s \right\| _ {2} \leq \left\| \boldsymbol {u} \boldsymbol {v} ^ {\mathsf {T}} - K / s \right\| _ {\mathsf {S} _ {p}} \leq (9 / 2 5 0) n.
$$
Similar to the proof of Theorem E.1, we can construct a unit vector  $q \in \mathbb{R}^{n^2}$  such that with probability  $\geq 8/10$  over  $M$ , we have
Using (4), we obtain that
$$
t + 1 \geq c \frac {\log (n ^ {2} / k)}{1 + \log (\alpha) + \log \log (n)} \geq c \frac {\log (n ^ {2} / k)}{1 + (1 / p) \log (n) + \log \log (n)}.
$$
In particular, for  $p = O(\log (n) / \log \log (n))$ , this gives a lower bound of  $\Omega (p(2 - \log (k) / \log (n)))$  on the number of adaptive rounds required if an algorithm can query  $k$  general linear measurements in each round. Recently, Bakshi, Clarkson and Woodruff [2, Algorithm 2] gave an algorithm for Schatten-  $p$  low rank approximation for arbitrary matrices that runs in  $O(p^{1 / 2}\log (n))^4$  iterations to output a constant factor approximation. For instances with  $\sigma_r(A) / \sigma_{r + 1}(A) = 1 + \Omega (1)$ , Saibaba [27, Theorem 8] showed that randomized subspace iteration gives a constant factor approximation to rank-  $r$  Schatten-  $p$  norm LRA in  $O(\log (n))$  iterations using  $nr$  linear measurements in each round.
E.4 Ky-Fan Norm Low Rank Approximation
Theorem E.8. Given  $n, r \in \mathbb{Z}$ , if a t-round adaptive algorithm that performs  $k \geq nr$  general linear measurements in each round is such that, for every  $n \times n$  matrix  $A$ , the algorithm outputs a rank- $r$  matrix  $B$  such that with probability  $\geq 99/100$  we have
$$
\| A - B \| _ {\mathsf {F} _ {p}} \leq 2 \min  _ {\text {r a n k - r X}} \| A - X \| _ {\mathsf {F} _ {p}},
$$
$$
t \geq c \frac {\log (n ^ {2} / k)}{\log (p) + \log \log (n)}.
$$
Proof. Again consider the same instance  $M = G + suv^{\top}$  with  $s = \alpha / \sqrt{n}$  for  $\alpha$  chosen later. If  $K$  is a 2-approximate rank- $r$  low rank approximation of  $M$  in Ky-Fan  $p$  norm, then
$$
\left\| \boldsymbol {G} + s u \boldsymbol {v} ^ {\mathsf {T}} - K \right\| _ {\mathsf {F} _ {p}} \leq 2 \left\| \boldsymbol {G} \right\| _ {\mathsf {F} _ {p}}
$$
which by the triangle inequality implies that  $\| s\sum_{i = 1}^{r}\pmb {u}_i\pmb {v}_i^\top -K\|_{\mathsf{F}_p}\leq 3\| \pmb {G}\|_{\mathsf{F}_p}$ . As  $\| G\| _2\leq 2\sqrt{n}$  with high probability, we have that  $\| G\|_{\mathsf{F}_p}\leq 2p\sqrt{n}$  with high probability. Thus,
$$
\| s \sum_ {i = 1} ^ {r} \boldsymbol {u} _ {i} \boldsymbol {v} _ {i} ^ {\top} - K / s \| _ {2} \leq \| s \sum_ {i = 1} ^ {r} \boldsymbol {u} _ {i} \boldsymbol {v} _ {i} ^ {\top} - K / s \| _ {\mathsf {F} _ {p}} \leq \frac {6 p n}{\alpha}.
$$
Although their algorithm is stated to use  $rp^{1/6} \log^2(n)$  matrix-vector products, this does not appear to have been optimized, and looking at the analysis it runs in at most  $p^{1/2} \log(n)$  iterations, where each round queries at most  $r$  matrix-vector products.
Picking  $\alpha = Bp$  for a large enough constant  $B$ , we obtain that using the matrix  $K$ , we can construct a unit vector  $q$  such that  $\langle q, \boldsymbol{u} \otimes \boldsymbol{v} \rangle^2 \geq n^2 / 64$ , thus showing a
$$
t = c \frac {\log (n ^ {2} / k)}{\log (p) + \log \log (n)}
$$
round lower bound on any algorithm that performs  $k$  general linear measurements to output a 2-approximate rank- $r$  approximation in Ky-Fan  $p$  norm.
For  $p = O(1)$ , the Block Krylov iteration algorithm [20] gives an  $O(1)$  approximate solution to the rank-  $r$  Ky-Fan norm low rank approximation problem in  $O(\log (n))$  rounds while querying  $nr$  general linear measurements in each round. Thus our lower bound on constant approximate algorithms for Ky-Fan norm LRA is optimal up to a log  $\log (n)$  factor for  $r, p = O(1)$ .
E.5 Frobenius Norm Low Rank Approximation
Theorem E.9. Given an  $n \times n$  matrix  $A$  with  $\sigma_1(A) / \sigma_2(A) \geq 2$ , if a t-round algorithm outputs a rank-1 matrix  $B$  satisfying  $\| A - B\|_{\mathbb{F}}^2 \leq (1 + 1/n)\| A - [A]_1\|_{\mathbb{F}}^2$  with probability  $\geq 99/100$ , then we have  $t \geq c\log(n^2/k)/\log\log(n)$ .
First, we prove the following helper lemma.
Lemma E.10. Given  $A \in \mathbb{R}^{n \times n}$ , if a rank  $r$  matrix  $B$  is a  $1 + 1 / (n - r)$  approximate rank  $r$  Frobenius norm LRA, then
$$
\left\| A - B \right\| _ {2} ^ {2} \leq 2 \sigma_ {r + 1} (A) ^ {2}.
$$
Proof. We use the fact [12, Theorem 3.4] that if a rank  $r$  matrix satisfies  $\| A - B\|_{\mathbb{F}}^2\leq \| A - [A]_r\|_{\mathbb{F}}^2 +\eta$ , then  $\| A - B\| _2^2\leq \| A - [A]_r\| _2^2 +\eta$ . If  $B$  is a  $1 + 1 / (n - r)$  approximate solution for rank  $r$  Frobenius norm LRA, we have
$$
\begin{array}{l} \| A - B \| _ {\mathsf {F}} ^ {2} \leq \left(1 + \frac {1}{n - r}\right) \| A - [ A ] _ {r} \| _ {\mathsf {F}} ^ {2} \\ \leq \| A - [ A ] _ {r} \| _ {\mathsf {F}} ^ {2} + \frac {\sigma_ {r + 1} (A) ^ {2} + \dots + \sigma_ {n} (A) ^ {2}}{n - r} \\ \leq \left\| A - [ A ] _ {r} \right\| _ {\mathsf {F}} ^ {2} + \sigma_ {r + 1} (A) ^ {2} \\ \end{array}
$$
which implies  $\| A - B\| _2^2\leq \| A - [A]_r\| _2^2 +\sigma_{r + 1}(A)^2 = 2\sigma_{r + 1}(A)^2$
Proof of Theorem E.9. Using the above lemma, we have that whenever  $\| A - B\|_{\mathsf{F}}^2\leq (1 + 1 / n)\| A - [A]_1\|_{\mathsf{F}}^2$  for a rank 1 matrix  $B$ , then  $\| A - B\| _2^2\leq 2\| A - B\| _2^2$ . Consider the random matrix  $M = G + (\alpha /\sqrt{n})u v^{\mathrm{T}}$  considered in Theorem E.1. With probability  $\geq 99 / 100$ , we have that  $\sigma_1(M) / \sigma_2(M)\geq 2$  by picking  $\alpha$  to be a large enough constant.
A  $t$ -round randomized algorithm for  $(1 + 1 / n)$ -approximate rank-1 Frobenius norm LRA thus implies the existence of a  $(t + 1)$  round deterministic algorithm that with probability  $\geq 98 / 100$  over the random matrix  $\pmb{G} + (\alpha / \sqrt{n})\pmb{u}\pmb{v}^{\top}$  makes general queries such that the squared projection of  $\pmb{u} \otimes \pmb{v}$  onto the query space of the algorithm exceeds  $n^2 / 64$  using similar arguments as in proof of Theorem E.1. Now, using (4), we obtain that
$$
t \geq c \log (n ^ {2} / k) / \log \log (n)
$$
for a small enough constant  $c$ .
For matrices  $A$  with  $\sigma_1(A) / \sigma_2(A) \geq 2$ , subspace iteration algorithm gives a  $1 + 1/n$ -approximate solution in  $O(\log(n))$  rounds using  $n$ -linear measurements [10, 20]. The above lower bound shows that if an algorithm is allowed  $o(\log(n) / \log \log(n))$  adaptive rounds, then the algorithm has to make  $n^{2 - o(1)}$  linear measurements in each round, which is almost as many measurements as is required to read the entire matrix.
E.6 Lower Bound for Reduced-Rank Regression in Spectral Norm
Given matrices  $A, B$ , and a rank parameter  $r$ , the reduced-rank regression problem in spectral norm is defined as
$$
\min_{\substack{\text{rank} -r\\ X}}\| AX - B\|_{2}.
$$
Taking  $A = I$ , we have that the spectral norm low rank approximation is a special case of the reduced-rank regression problem. Thus we have the following lower bound on the number of rounds of an adaptive algorithm that outputs a 2-approximate solution for the reduced-rank regression problem.
Theorem E.11. If a  $t$ -round adaptive algorithm which given arbitrary  $n \times n$  matrices  $A, B$  and a parameter  $r$  outputs a 2-approximate solution for the reduced-rank regression problem with probability  $\geq 9/10$ , then  $t = \Omega (\log (n^2 / k) / \log \log (n))$  where  $k$  is the number of linear measurements of the matrices  $A$  and  $B$  that the adaptive algorithm performs in each of the  $t$  rounds.
With  $nr$  adaptive linear measurements in each round, the above theorem gives a lower bound of  $\Omega (\log (n / r) / \log \log (n))$  on the number of rounds required to obtain a factor 2 approximation. Kacham and Woodruff [15] give an algorithm for reduced-rank regression that outputs constant factor approximations to the reduced-rank problem in  $O(\mathrm{poly}(\log n))$  (ignoring polylogarithmic factors from condition numbers) adaptive rounds using  $nr$  linear measurements in each round.
If the matrix  $B = M + P$  has the structure of a planted matrix studied in this paper with  $P$  being a rank  $r$  matrix and  $\| P\| _2 > 10\| B - [B]_r\| _2$  and if the matrix  $A$  is well conditioned, then we can find an  $O(1)$  approximate solution to reduced-rank regression problem in  $O(\log (n) + \log (\| P\| _2 / \mathrm{OPT}))$  rounds, where OPT is the optimal value of the reduced-rank regression problem.
First, we find a rank  $r$  matrix  $P'$  such that  $\| B - P' \|_2 \leq 2 \| B - [B]_r \|_2$  in  $O(\log(n))$  adaptive rounds using the Block Krylov iteration algorithm of [20], which queries  $nr$  linear measurements in each round. Now, for any matrix  $X$ ,
$$
\left\| A X - P ^ {\prime} \right\| _ {2} = \left\| A X - B \right\| _ {2} \pm \left\| B - P ^ {\prime} \right\| _ {2} = \left\| A X - B \right\| _ {2} \pm 2 \cdot \text {O P T}.
$$
Let  $P' = U\Sigma V^{\top}$  be the singular value decomposition with matrix  $U\Sigma$  having  $r$  columns. Then using high precision regression routines (see [33] and references therein), we find a matrix  $\hat{X}$  such that
$$
\| A \tilde {X} - A A ^ {+} U \Sigma \| _ {\mathsf {F}} ^ {2} \leq \varepsilon \| U \Sigma \| _ {\mathsf {F}} ^ {2} \leq 1 0 r \varepsilon \| P \| _ {2} ^ {2}.
$$
in  $O(\log(1/\varepsilon))$  iterations, where each iteration queries  $nr$  linear measurements. Again, using the triangle inequality, we have
$$
\left\| A \tilde {X} V ^ {\mathsf {T}} - P ^ {\prime} \right\| _ {2} = \left\| A \tilde {X} - U \Sigma \right\| _ {2} \leq \left\| A A ^ {+} U \Sigma - A \tilde {X} \right\| _ {2} + \left\| A A ^ {+} U \Sigma - U \Sigma \right\| _ {2}.
$$
If  $X^{*}$  is the optimal solution for the reduced rank regression problem, we have
$$
\begin{array}{l} \left\| A A ^ {+} U \Sigma - U \Sigma \right\| _ {2} = \left\| A A ^ {+} U \Sigma V ^ {\top} - U \Sigma V ^ {\top} \right\| _ {2} \\ \leq \left\| A X ^ {*} - P ^ {\prime} \right\| _ {2} \\ \leq \| A X ^ {*} - B \| _ {2} + 2 \cdot \mathrm {O P T} = 3 \cdot \mathrm {O P T}. \\ \end{array}
$$
We also have  $\| AA^{+}U\Sigma - A\tilde{X}\|_{2} \leq \| AA^{+}U\Sigma - A\tilde{X}\|_{\mathsf{F}} \leq O(\sqrt{r\varepsilon}\| P\|_{2})$ . We set  $\varepsilon = (\mathrm{OPT} / \| P\|_{2})^{2} / Kr$  for a large enough constant  $K$  and obtain that  $\| AA^{+}U\Sigma - A\tilde{X}\|_{2} \leq \mathrm{OPT}$ . Overall, we have  $\| A\hat{X} V^{\top} - B\|_{2} \leq \| A\hat{X} V^{\top} - P'\|_{2} + 2 \cdot \mathrm{OPT} \leq 6 \cdot \mathrm{OPT}$ . Thus we obtain a 6-approximate solution in  $O(\log (n) + \log (\| P\|_{2} / \mathrm{OPT}))$  iterations given that  $A$  is well-conditioned and  $B$  is of the form  $M + P$  with  $\| P\|_{2} \geq 10\| M\|_{2}$  for a rank  $r$  matrix  $P$ . Thus the lower bound is near optimal for algorithms that output  $O(1)$  approximate solution for planted models and well-conditioned coefficient matrices.
E.7 Lower Bound for Approximating the  $i$ -th Singular Vectors
Theorem E.12. If a  $t$ -round algorithm, given an  $n \times n$  matrix  $A$  and  $i \in [n]$ , outputs unit vectors  $u_{i}'$  and  $v_{i}'$  such that with probability  $\geq 9/10$ ,
$$
\left\| u _ {i} ^ {\prime} - u _ {i} \right\| _ {2} \leq 1 / 1 0 a n d \left\| v _ {i} ^ {\prime} - v _ {i} \right\| _ {2} \leq 1 / 1 0
$$
where  $u_{i}$  and  $v_{i}$  are respectively the  $i$ -th left and right singular vectors of the matrix  $A$ , then  $t = \Omega (\log (n^{2} / k) / \log \log (n))$ , where  $k$  is the number of linear measurements the algorithm performs on the matrix  $A$  in each of the  $t$  rounds.
We also have a  $t = \Omega (\log (n^2 /k))$  lower bound on the number of rounds required for an adaptive algorithm to compute approximations to the  $i$ -th left and right singular vectors with probability  $\geq 1 - 1 / \mathrm{poly}(n)$ .