Kenji Fukumizu, Shoichiro Yamaguchi, Yoh-ichi Mototake, and Mirai Tanaka. Semi-flat minima and saddle points by embedding neural networks to overparameterization. In Advances in Neural Information Processing Systems 32, pages 13868-13876. Curran Associates, 2019. Access via NeurIPS. Cited on pages 2, 5, and 9.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P. Vetrov, and Andrew G. Wilson. Loss surfaces, mode connectivity, and fast ensembling of DNNs. In Advances in Neural Information Processing Systems 31, pages 8789-8798. Curran Associates, 2018. Access via NeurIPS. Cited on page 9.
Egbert Harzheim. Ordered Sets. Springer, 2005. Access via Crossref. Cited on page 13.
Robert Hecht-Nielsen. On the algebraic structure of feedforward network weight spaces. In Advanced Neural Computers, pages 129-135. North-Holland, Amsterdam, 1990. Access via Crossref. Cited on pages 1 and 2.
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. Journal of Machine Learning Research, 22(241):1-124, 2021. Access via JMLR. Cited on page 10.
Věra Kürková and Paul C. Kainen. Functionally equivalent feedforward neural networks. Neural Computation, 6(3):543-558, 1994. Access via Crossref. Cited on pages 1 and 2.
Rohith Kuditipudi, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, and Sanjeev Arora. Explaining landscape connectivity of low-cost solutions for multilayer nets. In Advances in Neural Information Processing Systems 32, pages 14601-14610. Curran Associates, 2019. Access via NeurIPS. Cited on page 9.
Edmund Lau, Daniel Murfet, and Susan Wei. Quantifying degeneracy in singular models via the learning coefficient. 2023. Preprint arXiv:2308.12108 [stat.ML]. Cited on page 10.
Mary Phuong and Christoph H. Lampert. Functional vs. parametric equivalence of ReLU networks. In 8th International Conference on Learning Representations. OpenReview, 2020. Access via OpenReview. Cited on pages 1 and 2.
Stefan M. Rüger and Arnfried Ossen. The metric structure of weight space. Neural Processing Letters, 5(2):1-9, 1997. Access via Crossref. Cited on pages 2 and 4.
Berfin Şimşek, François Ged, Arthur Jacot, Francesco Spadaro, Clément Hongler, Wulfram Gerstner, and Johann Brea. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. In Proceedings of the 38th International Conference on Machine Learning, pages 9722-9732. PMLR, 2021. Access via PMLR. Cited on pages 2, 5, 6, and 9.
Pierre Stock and Rémi Gribonval. An embedding of ReLU networks and an analysis of their identifiability. Constructive Approximation, 2022. Access via Crossref. Cited on page 2.
Héctor J. Sussmann. Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks, 5(4):589-593, 1992. Access via Crossref. Cited on pages 1, 2, 3, 4, 5, 13, 14, 15, and 16.
Verner Vlačić and Helmut Bölskei. Affine symmetries and neural network identifiability. Advances in Mathematics, 376:107485, 2021. Access via Crossref. Cited on pages 1 and 2.
Verner Vlačić and Helmut Bölsceki. Neural network identifiability for a family of sigmoidal nonlinearities. Constructive Approximation, 55(1):173-224, 2022. Access via Crossref. Cited on page 2.
Sumio Watanabe. *Algebraic Geometry and Statistical Learning Theory*. Cambridge University Press, 2009. Cited on pages 1 and 10.
Susan Wei, Daniel Murfet, Mingming Gong, Hui Li, Jesse Gell-Redman, and Thomas Quella. Deep learning is singular, and that's good. IEEE Transactions on Neural Networks and Learning Systems, 2022. Access via Crossref. To appear in an upcoming volume. Cited on pages 1 and 10.