SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. We briefly describe the algorithm from the supplement for completeness here, as correctness follows straightforwardly from combining Corollary 1 with Lemmas 6 and 7. The algorithm is an instance of the conceptual Algorithm 1. Each step of Algorithm 1 then first implements Algorithm 2 (used in Lemma 6), with the parameters required by Lemma 4, to obtain a gradient step oracle. Similarly, it next implements Algorithm 3 (used in Lemma 7), with the parameters required by Lemma 4, to obtain an extragradient step oracle. Correctness follows from Corollary 1. For the runtime, without loss of generality  $\mathrm{nnz}(\mathbf{A})\geq \max (n,d)$  (else we may drop columns or rows appropriately), and each of  $T$  iterations is dominated by a constant number of matrix-vector multiplications.
We would like thank our long-term collaborators Yujia Jin and Aaron Sidford for many helpful discussions at earlier stages of this project, which improved our understanding of area convexity. We also thank Victor Reis for his collaboration on related ideas to this work in [JRT23]. Finally, we thank anonymous reviewers for their helpful suggestions in improving our presentation.
[vdBLL+21] Jan van den Brand, Yin Tat Lee, Yang P. Liu, Thatchaphol Saranurak, Aaron Sidford, Zhao Song, and Di Wang. Minimum cost flows, mdps, and  $\ell_1$ -regression in nearly linear time for dense instances. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2021, pages 859-869, 2021.
[WK06] Manfred K. Warmuth and Dima Kuzmin. Randomized PCA algorithms with regret bounds that are logarithmic in the dimension. In Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, 2006, pages 1481-1488, 2006.
[ZLO16] Zeyuan Allen Zhu, Yin Tat Lee, and Lorenzo Orecchia. Using optimization to obtain a width-independent, parallel, simpler, and faster positive SDP solver. In Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2016, pages 1824-1831. SIAM, 2016.
[ZO15] Zeyuan Allen Zhu and Lorenzo Orecchia. Nearly-linear time positive LP solver with faster convergence rate. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC 2015, pages 229-236. ACM, 2015.
[ZP96] Uri Zwick and Mike Paterson. The complexity of mean payoff games on graphs. Theor. Comput. Sci., 158(1&2):343-359, 1996.