Arnulf Jentzen and Adrian Riekert. Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation. J. Math. Anal. Appl., 517(2):126601, 2023.
Ziwei Ji and Matus Telgarsky. Directional convergence and alignment in deep learning. In NeurIPS, 2020. 1
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon S. Du, and Jason D. Lee. Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing. In ICML, pages 15200-15238, 2023. 2
Sangmin Lee, Byeongsu Sim, and Jong Chul Ye. Magnitude and Angle Dynamics in Training Single ReLU Neurons. CoRR, abs/2209.13394, 2022.
Qianxiao Li, Cheng Tai, and Weinan E. Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations. J. Mach. Learn. Res., 20(40):1-47, 2019. 4
Zhiyuan Li, Yuping Luo, and Kaifeng Lyu. Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning. In ICLR, 2021. 2
Kaifeng Lyu and Jian Li. Gradient Descent Maximizes the Margin of Homogeneous Neural Networks. In ICLR, 2020. 1
Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, and Sanjeev Arora. Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias. In NeurIPS, pages 12978-12991, 2021.
Hartmut Maennel, Olivier Bousquet, and Sylvain Gelly. Gradient Descent Quantizes ReLU Network Features. CoRR, abs/1803.08367, 2018. 2, 5
Odelia Melamed, Gilad Yehudai, and Gal Vardi. Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Data Manifolds. CoRR, abs/2303.00783, 2023. Accepted to NeurIPS 2023.
Hancheng Min, René Vidal, and Enrique Mallada. Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization. CoRR, abs/2307.12851, 2023.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring Generalization in Deep Learning. In NeurIPS, pages 5947-5956, 2017. 1
Greg Ongie, Rebecca Willett, Daniel Soudry, and Nathan Srebro. A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case. In ICLR, 2020.
Scott Pesme and Nicolas Flammarion. Saddle-to-Saddle Dynamics in Diagonal Linear Networks. CoRR, abs/2304.00488, 2023. Accepted to NeurIPS 2023.
Mary Phuong and Christoph H. Lampert. The inductive bias of ReLU networks on orthogonally separable data. In ICLR, 2021. 3
B.T. Polyak. Gradient methods for the minimisation of functionals. USSR Computational Mathematics and Mathematical Physics, 3(4):864-878, 1963. 8
Noam Razin and Nadav Cohen. Implicit Regularization in Deep Learning May Not Be Explainable by Norms. In NeurIPS, 2020. 3
Roei Sarussi, Alon Brutzkus, and Amir Globerson. Towards Understanding Learning in Neural Networks with Linear Teachers. In ICML, pages 9313-9322, 2021.
Pedro Savarese, Itay Evron, Daniel Soudry, and Nathan Srebro. How do infinite width bounded norm networks look in function space? In  $COLT$ , pages 2667-2690, 2019.
Lawrence Stewart, Francis Bach, Quentin Berthet, and Jean-Philippe Vert. Regression as Classification: Influence of Task Formulation on Neural Network Features. In AISTATS, pages 11563-11582, 2023.
Nadav Timor, Gal Vardi, and Ohad Shamir. Implicit Regularization Towards Rank Minimization in ReLU Networks. In ALT, pages 1429-1459, 2023. 2
Gal Vardi. On the Implicit Bias in Deep-Learning Algorithms. Commun. ACM, 66(6):86-93, 2023. 1
Gal Vardi and Ohad Shamir. Implicit Regularization in ReLU Networks with the Square Loss. In  $COLT$ , pages 4224-4258, 2021. 2
Gal Vardi, Gilad Yehudai, and Ohad Shamir. Learning a Single Neuron with Bias Using Gradient Descent. In NeurIPS, pages 28690-28700, 2021.