[34] JANZAMIN, M., SEDGHI, H., AND ANANDKUMAR, A. Beating the perils of non-convexity: Guaranteed training of neural networks using tensor methods. arXiv (2016).
[35] JASTRZEBSKI, S., ET AL. Residual connections encourage iterative inference. In ICLR (2018).
[36] KARKAR, S., AYED, I., DE BÉZENAC, E., AND GALLINARI, P. A principle of least action for the training of neural networks. In ECML-PKDD (2020).
[37] KRIZHEVSKY, A. Learning multiple layers of features from tiny images. University of Toronto Technical Report (2009).
[38] LECUN, Y., CORTES, C., AND BURGES, C. J. MNIST handwritten digit database. yann.lecun.com/exdb/mnist (2010).
[39] LIU, Z., LIN, Y., CAO, Y., HU, H., WEI, Y., ZHANG, Z., LIN, S., AND GUO, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021).
[40] LIUTKUS, A., IMSEKLI, U., MAJEWSKI, S., DURMUS, A., AND STOTER, F.-R. Slicedwasserstein flows: Nonparametric generative modeling via optimal transport and diffusions. In ICML (2019).
[41] MA, W.-D. K., LEWIS, J., AND KLEIJN, W. B. The hsic bottleneck: Deep learning without back-propagation. In AAAI (2020).
[42] MANDAR KULKARNI, S. K. Layer-wise training of deep networks using kernel similarity. In DLPR workshop, ICPR (2016).
[43] MARQUEZ, E. S., HARE, J. S., AND NIRANJAN, M. Deep cascade learning. IEEE Transactions on Neural Networks and Learning Systems (2018).
[44] NGUYEN, T. T., AND CHOI, J. Layer-wise learning of stochastic neural networks with information bottleneck. Entropy 21 (2019).
[45] NØKLAND, A., AND EIDNES, L. H. Training neural networks with local error signals. In ICML (2019).
[46] PATHAK, P., ZHANG, J., AND SAMARAS, D. Local learning on transformers via feature reconstruction, 2022.
[47] PYEON, M., MOON, J., HAHN, T., AND KIM, G. Sedona: Search for decoupled neural networks toward greedy block-wise learning. In ICLR (2021).
[48] SANDLER, M., HOWARD, A. G., ZHU, M., ZHMOGINOV, A., AND CHEN, L. Mobilenetv2: Inverted residuals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 (2018), Computer Vision Foundation / IEEE Computer Society, pp. 4510-4520.
[49] SANTAMBROGIO, F. Optimal Transport for Applied Mathematicians. Birkhäuser, 2015.
[50] SANTAMBROGIO, F. Euclidean, metric, and wasserstein gradient flows: an overview. arXiv (2016).
[51] SAXE, A. M., McCLELLAND, J. L., AND GANGULI, S. Exact solutions to the nonlinear dynamics of learning in deep linear neural network. In ICLR (2014).
[52] SIMONYAN, K., AND ZISSERMAN, A. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR 2014) (2014).
[53] SINDY LÖWE, PETER O’CONNOR, B. V. Putting an end to end-to-end: Gradient-isolated learning of representations. In NeurIPS (2019).
[54] STICH, S. U. Local sgd converges fast and communicates little. In ICLR (2019).
[55] SUN, Q., DONG, H., CHEN, Z., SUN, J., LI, Z., AND DONG, B. Layer-parallel training of residual networks with auxiliary-variable networks. arXiv (2021).
[56] TAN, M., AND LE, Q. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning (09-15 Jun 2019), K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97 of Proceedings of Machine Learning Research, PMLR, pp. 6105-6114.
[57] TANG, Y., TENG, Q., ZHANG, L., MIN, F., AND HE, J. Layer-wise training convolutional neural networks with smaller filters for human activity recognition using wearable sensors. IEEE Sensors Journal (2021).