Z. Ji and M. Telgarsky. Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow relu networks. arXiv preprint arXiv:1909.12292, 2019.
V. Koltchinskii. Local Rademacher complexities and oracle inequalities in risk minimization. The Annals of Statistics, 34:2593-2656, 2006.
M. Ledoux and M. Talagrand. Probability in Banach Spaces. Isoperimetry and Processes. Springer, New York, 1991. MR1102015.
S. Mei, A. Montanari, and P.-M. Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E7665-E7671, 2018.
S. Mendelson. Improving the sample complexity using global data. IEEE Transactions on Information Theory, 48:1977-1991, 2002.
M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning. The MIT Press, 2012.
A. Mousavi-Hosseini, D. Wu, T. Suzuki, and M. A. Erdogdu. Gradient-based feature learning under structured data. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), 2023.
A. Nitanda and T. Suzuki. Stochastic particle gradient descent for infinite ensembles. arXiv preprint arXiv:1712.05438, 2017.
A. Nitanda, D. Wu, and T. Suzuki. Convex analysis of the mean field Langevin dynamics. In G. Camps-Valls, F. J. R. Ruiz, and I. Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 9741-9757. PMLR, 28-30 Mar 2022.
M. Refinetti, S. Goldt, F. Krzakala, and L. Zdeborova. Classifying high-dimensional gaussian mixtures: Where kernel methods fail and neural networks succeed. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8936-8947. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/refinetti21b.html.
G. M. Rotskoff and E. Vanden-Eijnden. Trainability and accuracy of neural networks: An interacting particle system approach. arXiv preprint arXiv:1805.00915, 2018.
J. Sirignano and K. Spiliopoulos. Mean field analysis of neural networks: A central limit theorem. Stochastic Processes and their Applications, 130(3):1820-1852, 2020.
N. Srebro, K. Sridharan, and A. Tewari. Smoothness, low noise and fast rates. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010. URL https://proceedings.neurips.cc/paper_files/paper/2010/file/76cf99d3614e23eabab16fb27e944bf9-Paper.pdf.
T. Suzuki, A. Nitanda, and D. Wu. Uniform-in-time propagation of chaos for the mean field gradient Langevin dynamics. In Submitted to The Eleventh International Conference on Learning Representations, 2023a. URL https://openreview.net/forum?id=_JScUk9TBUn.
T. Suzuki, D. Wu, and A. Nitanda. Convergence of mean-field Langevin dynamics: Time-space discretization, stochastic gradient, and variance reduction. arXiv preprint, 2023b.
M. Telgarsky. Feature selection and low test error in shallow low-rotation ReLU networks. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=swEskiem99.
S. van de Geer. Empirical Processes in M-Estimation. Cambridge University Press, 2000.
C. Wei, J. D. Lee, Q. Liu, and T. Ma. Regularization matters: Generalization and optimization of neural nets vs their induced kernel. In Advances in Neural Information Processing Systems, pages 9712-9724, 2019.