[19] Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6):26-38, 2017.
[20] Richard Stuart Sutton. Temporal credit assignment in reinforcement learning. University of Massachusetts Amherst, 1984.
[21] Paul Cecil Martin, ED Siggia, and HA Rose. Statistical dynamics of classical systems. Physical Review A, 8(1):423, 1973.
[22] A Crisanti and H Sompolinsky. Path integral approach to random neural networks. Physical Review E, 98(6):062120, 2018.
[23] Moritz Helias and David Dahmen. Statistical field theory for neural networks, volume 970. Springer, 2020.
[24] Blake Bordelon and Cengiz Pehlevan. Self-consistent dynamical field theory of kernel evolution in wide neural networks. arXiv preprint arXiv:2205.09653, 2022.
[25] Blake Bordelon and Cengiz Pehlevan. The influence of learning rule on representation dynamics in wide neural networks. arXiv preprint arXiv:2210.02157, 2022.
[26] Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan. Spectrum dependent learning curves in kernel regression and wide neural networks. In International Conference on Machine Learning, pages 1024-1034. PMLR, 2020.
[27] Bruno Loureiro, Cedric Gerbelot, Hugo Cui, Sebastian Goldt, Florent Krzakala, Marc Mezard, and Lenka Zdeborova. Learning curves of generic features maps for realistic datasets with a teacher-student model. Advances in Neural Information Processing Systems, 34:18137-18151, 2021.
[28] Hong Hu and Yue M Lu. Universality laws for high-dimensional learning with random features. IEEE Transactions on Information Theory, 2022.
[29] Blake Bordelon and Cengiz Pehlevan. Learning curves for SGD on structured features. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=WPI2vbkA13Q.
[30] Abdulkadir Canatar, Blake Bordelon, and Cengiz Pehlevan. Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks. Nature communications, 12(1):2914, 2021.
[31] Bernhard Schölkopf, Alexander J Smola, Francis Bach, et al. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
[32] Manel Tagorti and Bruno Scherrer. On the rate of convergence and error bounds for lstd  $(\lambda)$ . In International Conference on Machine Learning, pages 1521-1529. PMLR, 2015.
[33] Yangchen Pan, Adam White, and Martha White. Accelerated gradient temporal difference learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
[34] Alborz Geramifard, Michael Bowling, Martin Zinkevich, and Richard S Sutton. ilstd: Eligibility traces and convergence analysis. Advances in Neural Information Processing Systems, 19, 2006.
[35] Fernando J Pineda. Mean-field theory for batched td  $(\lambda)$ . Neural computation, 9(7):1403-1419, 1997.
[36] Gandhary Patil, LA Prashanth, Dheeraj Nagaraj, and Doina Precup. Finite time analysis of temporal difference learning with linear function approximation: Tail averaging and regularisation. In International Conference on Artificial Intelligence and Statistics, pages 5438-5448. PMLR, 2023.
[37] Rayadurgam Srikant and Lei Ying. Finite-time error bounds for linear stochastic approximation and td learning. In Conference on Learning Theory, pages 2803-2830. PMLR, 2019.