[33] S. Du, J. Lee, H. Li, L. Wang, and X. Zhai, "Gradient descent finds global minima of deep neural networks," in International conference on machine learning. PMLR, 2019, pp. 1675-1685.
[34] D. Zou and Q. Gu, "An improved analysis of training over-parameterized deep neural networks," Advances in neural information processing systems, vol. 32, 2019.
[35] Q. N. Nguyen and M. Mondelli, “Global convergence of deep networks with one wide layer followed by pyramidal topology,” Advances in Neural Information Processing Systems, vol. 33, pp. 11961–11972, 2020.
[36] N. Lee, T. Ajanthan, S. Gould, and P. H. S. Torr, "A signal propagation perspective for pruning neural networks at initialization," in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HJeTo2VFwH
[37] K. Bhardwaj, G. Li, and R. Marculescu, "How does topology influence gradient propagation and model performance of deep networks with densenet-type skip connections?" in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13498-13507.
[38] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, "Visualizing the loss landscape of neural nets," Advances in neural information processing systems, vol. 31, 2018.
[39] R. A. Horn and C. R. Johnson, Matrix analysis. Cambridge university press, 2012.
[40] Y. Cao and Q. Gu, "Generalization bounds of stochastic gradient descent for wide and deep neural networks," Advances in neural information processing systems, vol. 32, 2019.
[41] X. Dong and Y. Yang, "Nas-bench-201: Extending the scope of reproducible neural architecture search," in International Conference on Learning Representations, 2020.
[42] X. Dong, L. Liu, K. Musial, and B. Gabrys, "Nats-bench: Benchmarking nas algorithms for architecture topology and size," IEEE transactions on pattern analysis and machine intelligence, 2021.
[43] A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images.” Toronto, ON, Canada, 2009.
[44] P. Chrabaszcz, I. Loshchilov, and F. Hutter, "A downsampled variant of imagenet as an alternative to the CIFar datasets," arXiv preprint arXiv:1707.08819, 2017.
[45] A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese, “Taskonomy: Disentangling task transfer learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3712–3722.
[46] A. Zela, J. N. Siems, L. Zimmer, J. Lukasik, M. Keuper, and F. Hutter, "Surrogate nas benchmarks: Going beyond the limited search spaces of tabular nas benchmarks," in International Conference on Learning Representations, 2022.
[47] Y. Duan, X. Chen, H. Xu, Z. Chen, X. Liang, T. Zhang, and Z. Li, "Transnas-bench-101: Improving transferability and generalizability of cross-task neural architecture search," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5251-5260.
[48] C. Ying, A. Klein, E. Christiansen, E. Real, K. Murphy, and F. Hutter, “Nas-bench-101: Towards reproducible neural architecture search,” in International Conference on Machine Learning. PMLR, 2019, pp. 7105–7114.
[49] M. Chen, H. Peng, J. Fu, and H. Ling, "Autoformer: Searching transformers for visual recognition," in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 12270-12280.
[50] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, "Once-for-all: Train one network and specialize it for efficient deployment," in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HylxE1HKwS