1: Input: arm set  $\mathcal{X},\mathcal{Z}$ , confidence  $\delta$ , rank  $r$  of  $\Theta_{*}$ , spectral bound  $S_{r}$  of  $\Theta_{*}$ ,  $S,S_{\ell}^{\perp}:= \frac{8d_1d_2r}{\tau_\ell^E S_r^2}\log \left(\frac{d_1 + d_2}{\delta_\ell}\right)$ ,  $\lambda, \lambda_{\ell}^{\perp}:= \tau_{\ell -1}^{G} / 8(d_1 + d_2)r\log (1 + \frac{\tau_{\ell - 1}^{G}}{\lambda})$ . Let  $p:= d_1d_2$ ,  $k:= (d_1 + d_2)r$ .
2: Let  $\underline{\mathcal{W}}_1\gets \underline{\mathcal{W}},\ell \gets 1,\tau_0^G\coloneqq \log (4\ell^2 |\mathcal{X}| / \delta)$ . Define  $\Lambda_{\ell}$  as in (6),  $B_{*}^{\ell}\coloneqq (8\sqrt{\lambda} S + \sqrt{\lambda_{\ell}^{\perp}} S_{\ell}^{\perp})$
3: Define a vectorized arm  $\overline{\mathbf{w}} := [\mathbf{x}_{1:d_1};\mathbf{z}_{1:d_2}]$  and  $\overline{\mathbf{w}} \in \overline{\mathcal{W}}$ . Let  $\tau_{\ell}^{E} := \frac{\sqrt{8d_{1}d_{2}r\log(4\ell^{2}|\mathcal{W}| / \delta_{\ell})}}{S_{r}}$ . Let the  $E$ -optimal design be  $\mathbf{b}_{\ell}^{E} := \arg \min_{\mathbf{b} \in \triangle_{\overline{\mathcal{W}}}} \left\| \left( \sum_{\overline{\mathbf{w}} \in \overline{\mathcal{W}}} \mathbf{b}_{\overline{\mathbf{w}}}\overline{\mathbf{w}}\overline{\mathbf{w}}^{\top} \right)^{-1} \right\|$ .
4: while  $|\underline{W}_{\ell}| > 1$  do
5:  $\epsilon_{\ell} = 2^{-\ell}, \delta_{\ell} = \delta / \ell^{2}$ .
6: (Stage 1:) Explore the Low-Rank Subspace
7: Pull arm  $\overline{\mathbf{w}}\in \overline{\mathcal{W}}$  exactly  $\left[\widehat{\mathbf{b}}_{\ell ,\overline{\mathbf{w}}}^{E}\tau_{\ell}^{E}\right]$  times and observe rewards  $r_t$ , for  $t = 1,\dots ,\tau_{\ell}^{E}$ .
8: Compute  $\widehat{\Theta}_{\ell}$  using (2).
9: (Stage 2:) Reduction to low dimensional linear bandits
10: Let the SVD of  $\widehat{\pmb{\Theta}}_{\ell} = \widehat{\mathbf{U}}_{\ell}\widehat{\mathbf{D}}_{\ell}\widehat{\mathbf{V}}_{\ell}^{\top}$ . Rotate arms in active set  $\underline{\mathcal{W}}_{\ell -1}$  to build  $\underline{\mathcal{W}}_{\ell}$  following (4).
11: Let  $\widehat{\mathbf{b}}_{\ell}^{G} \coloneqq \arg \min_{\mathbf{b}_{\underline{\mathbf{w}}}} \max_{\underline{\mathbf{w}}, \underline{\mathbf{w}}' \in \underline{\mathcal{W}}_{\ell}} \| \underline{\mathbf{w}} - \underline{\mathbf{w}}' \|_{(\sum_{\underline{\mathbf{w}} \in \underline{\mathcal{W}}}\underline{\mathbf{b}}_{\underline{\mathbf{w}}} \underline{\mathbf{w}} \underline{\mathbf{w}}^{\top} + \boldsymbol{\Lambda}_{\ell} / n)^{-1}}^{2}$ .
12: Define  $\rho^G (\mathcal{V}(\underline{\mathcal{W}}_\ell))\coloneqq \min_{\mathbf{b}_{\underline{\mathbf{w}}}}\max_{\underline{\mathbf{w}},\underline{\mathbf{w}}'\in \underline{\mathcal{W}}_\ell}\| \underline{\mathbf{w}} -\underline{\mathbf{w}} '\| _(\sum_{\underline{\mathbf{w}}\in \underline{\mathcal{W}}}^2\mathbf{b}_{\underline{\mathbf{w}}}\underline{\mathbf{w}}\underline{\mathbf{w}}^\top +\boldsymbol {\Lambda}_\ell /n)^{-1}$
13: Set  $\tau_{\ell}^{G} \coloneqq \lceil \frac{64B_{*}^{\ell}\rho^{G}(\mathcal{Y}(\mathcal{W}_{\ell}))\log(4\ell^{2}|\mathcal{W}| / \delta_{\ell})}{\epsilon_{\ell}^{2}} \rceil$ . Then pull arm  $\underline{\mathbf{w}} \in \underline{\mathcal{W}}$  exactly  $\left\lceil \widehat{\mathbf{b}}_{\ell,\underline{\mathbf{w}}}^{G}\tau_{\ell}^{G}\right\rceil$  times and construct the least squares estimator  $\widehat{\pmb{\theta}}_{\ell}$  using only the observations of this phase where  $\widehat{\pmb{\theta}}_{\ell}$  is defined in (7). Note that  $\widehat{\pmb{\theta}}_{\ell}$  is also rotated following (4).
14: Eliminate arms such that  $\underline{\mathcal{W}}_{\ell + 1} \gets \underline{\mathcal{W}}_{\ell} \backslash \{\underline{\mathbf{w}} \in \underline{\mathcal{W}}_{\ell} : \max_{\mathbf{w}' \in \mathcal{W}_{\ell}} \langle \underline{\mathbf{w}}' - \underline{\mathbf{w}}, \widehat{\boldsymbol{\theta}}_{\ell} \rangle > 2\epsilon_{\ell}\}$
15:  $\ell \gets \ell + 1$
16: Output the arm in  $\mathcal{W}_{\ell}$  and reshape to get the  $\widehat{\mathbf{x}}_*$  and  $\widehat{\mathbf{z}}_*$