[40] S. Lee, S. Goldt, and A. Saxe, "Continual learning in the teacher-student setup: Impact of task similarity," in Proceedings of the 38th International Conference on Machine Learning, PMLR, 2021.
[41] I. Evron, E. Moroshko, R. Ward, N. Srebro, and D. Soudry, "How catastrophic can catastrophic forgetting be in linear regression?", in Proceedings of Thirty Fifth Conference on Learning Theory, 2022.
[42] W. J. Maddox, S. Tang, P. G. Moreno, A. G. Wilson, and A. Damianou, "Fast adaptation with linearized neural networks," 2021. 9
[43] T. Nguyen, M. Raghu, and S. Kornblith, “On the origins of the block structure phenomenon in neural network representations,” Transactions on Machine Learning Research, 2022. 9
[44] Z. Zhu, T. Ding, J. Zhou, X. Li, C. You, J. Sulam, and Q. Qu, “A geometric analysis of neural collapse with unconstrained features,” Advances in Neural Information Processing Systems, vol. 34, 2021. 9, 10
[45] T. Galanti, A. György, and M. Hutter, “On the role of neural collapse in transfer learning,” in International Conference on Learning Representations, 2022.
[46] L. Hui, M. Belkin, and P. Nakkiran, "Limitations of neural collapse for understanding generalization in deep learning," arXiv preprint arXiv:2202.08384, 2022. 9, 10
[47] S. Bai, J. Z. Kolter, and V. Koltun, "Deep equilibrium models," Advances in Neural Information Processing Systems, vol. 32, 2019. 9
[48] R. Dabre and A. Fujita, "Recurrent stacking of layers for compact neural machine translation models," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6292-6299, 2019. 9
[49] S. Bai, J. Z. Kolter, and V. Koltun, “Trellis networks for sequence modeling,” arXiv preprint arXiv:1810.06682, 2018.
[50] M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, “Universal transformers,” arXiv preprint arXiv:1807.03819, 2018. 9
[51] A. Morcos, H. Yu, M. Paganini, and Y. Tian, "One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers," Advances in neural information processing systems, vol. 32, 2019. 9
[52] A. M. Saxe, Y. Bansal, J. Dapello, M. Advani, A. Kolchinsky, B. D. Tracey, and D. D. Cox, “On the information bottleneck theory of deep learning,” Journal of Statistical Mechanics: Theory and Experiment, vol. 2019, no. 12, p. 124020, 2019. 9
[53] R. Shwartz-Ziv and N. Tishby, “Opening the black box of deep neural networks via information,” arXiv preprint arXiv:1703.00810, 2017. 9
[54] A. Ansuini, A. Laio, J. H. Macke, and D. Zoccolan, "Intrinsic dimension of data representations in deep neural networks," Advances in Neural Information Processing Systems, vol. 32, 2019. 10
[55] X. Li, S. Liu, J. Zhou, X. Lu, C. Fernandez-Granda, Z. Zhu, and Q. Qu, “Principled and efficient transfer learning of deep models via neural collapse,” arXiv preprint arXiv:2212.12206, 2022. 10
[56] A. Rangamani, M. Lindegaard, T. Galanti, and T. A. Poggio, “Feature learning in deep classifiers through intermediate neural collapse,” in International Conference on Machine Learning, pp. 28729–28745, PMLR, 2023. 10
[57] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations, 2015. 15