[18] Christoph Dann, Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E Schapire. On Oracle-Efficient PAC RL with Rich Observations. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. (Cited on page 10.)
[19] Mark A. Davenport and Justin K. Romberg. An overview of low-rank matrix recovery from incomplete observations. IEEE J. Sel. Top. Signal Process., 10(4):608-622, 2016. (Cited on page 9.)
[20] Simon Du, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal, Miroslav Dudik, and John Langford. Provably efficient RL with Rich Observations via Latent State Decoding. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1665-1674. PMLR, 09-15 Jun 2019. (Cited on page 10.)
[21] Justin Eldridge, Mikhail Belkin, and Yusu Wang. Unperturbed: spectral analysis beyond davis-kahan. In Algorithmic Learning Theory, pages 321-358. PMLR, 2018. (Cited on pages 2 and 9.)
[22] Jianqing Fan, Weichen Wang, and Yiqiao Zhong. An  $\ell_{\infty}$  eigenvector perturbation bound and its application to robust covariance estimation. Journal of Machine Learning Research, 18(207):1-42, 2018. (Cited on pages 2 and 9.)
[23] Vivek Farias, Andrew A Li, and Tianyi Peng. Near-optimal entrywise anomaly detection for low-rank matrices with sub-exponential noise. In International Conference on Machine Learning, pages 3154-3163. PMLR, 2021. (Cited on page 34.)
[24] Dylan Foster, Alexander Rakhlin, David Simchi-Levi, and Yunzong Xu. Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective. In Proceedings of Thirty Fourth Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Research, pages 2059–2059. PMLR, 15–19 Aug 2021. (Cited on page 10.)
[25] Noah Golowich, Ankur Moitra, and Dhruv Rohatgi. Learning in Observable POMDPs, without Computationally Intractable Oracles. In Advances in Neural Information Processing Systems, volume 35. Curran Associates, Inc., 2022. (Cited on page 10.)
[26] Samuel B Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer. Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 178-191, 2016. (Cited on page 26.)
[27] Prateek Jain and Soumyabrata Pal. Online low rank matrix completion. In Proc. of ICLR, 2023. (Cited on pages 1, 9, and 50.)
[28] Yassir Jedra, Junghyun Lee, Alexandre Proutiere, and Se-Young Yun. Nearly optimal latent state decoding in block mdps. In International Conference on Artificial Intelligence and Statistics, pages 2805–2904. PMLR, 2023. (Cited on pages 2, 8, and 9.)
[29] Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E. Schapire. Contextual decision processes with low Bellman rank are PAC-learnable. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1704–1713. PMLR, 06–11 Aug 2017. (Cited on page 10.)
[30] Chi Jin, Akshay Krishnamurthy, Max Simchowitz, and Tiancheng Yu. Reward-Free Exploration for Reinforcement Learning. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4870-4879. PMLR, 13-18 Jul 2020. (Cited on page 8.)
[31] Kwang-Sung Jun, Rebecca Willett, Stephen Wright, and Robert Nowak. Bilinear bandits with low-rank structure. In International Conference on Machine Learning, pages 3163-3172. PMLR, 2019. (Cited on page 50.)