[40] He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. arXiv 10.48550, 1512.03385 (2015).
[41] Vaswani, A. et al. Attention is all you need. arXiv 10.48550, 1706.03762 (2017).
[42] Tolstikhin, I. et al. Mlp-mixer: An all-MLP architecture for vision. arXiv 10.48550, 2105.01601 (2021).
[43] Lin, T., Horne, B. G., Tino, P. & Giles, C. L. Learning long-term dependencies is not as difficult with narx recurrent neural networks. In Advances in Neural Information Processing Systems (1998).
[44] Zhang, S. et al. Architectural complexity measures of recurrent neural networks. In Advances in Neural Information Processing Systems, vol. 29 (2016).
[45] Chang, S. et al. Dilated recurrent neural networks. In Advances in Neural Information Processing Systems, vol. 30 (Curran Associates, Inc., 2017).
[46] Yu, A. W., Lee, H. & Le, Q. V. Learning to skim text. arXiv 10.48550, 1704.06877 (2017).
[47] Gui, T. et al. Long short-term memory with dynamic skip connections. Proceedings of the AAAI Conference on Artificial Intelligence 33, 6481-6488 (2019).
[48] Kim, R. & Sejnowski, T. J. Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks. Nature Neuroscience 24, 129-139 (2020).
[49] Echeveste, R., Aitchison, L., Hennequin, G. & Lengyel, M. Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference. Nature Neuroscience 23, 1138-1149 (2020).
[50] Dubreuil, A., Valente, A., Beiran, M., Mastrogiuseppe, F. & Ostojic, S. The role of population structure in computations through neural dynamics. Nature Neuroscience 25, 783-794 (2022).
[51] Schuessler, F., Mastrogiuseppe, F., Dubreuil, A., Ostojic, S. & Barak, O. The interplay between randomness and structure during learning in rnns. In Advances in Neural Information Processing Systems, vol. 33, 13352-13362 (2020).
[52] Stroud, J. P., Watanabe, K., Suzuki, T., Stokes, M. G. & Lengyel, M. Optimal information loading into working memory in prefrontal cortex explains dynamic coding. bioRxiv 10.1101, 2021.11.16.468360 (2021).
[53] Murray, J. M. Local online learning in recurrent networks with random feedback. eLife 8 (2019).
[54] Zhang, X., Long, X., Zhang, S.-J. & Chen, Z. S. Excitatory-inhibitory recurrent dynamics produce robust visual grids and stable attractors. Cell Reports 41, 111777 (2022).
[55] Rajakumar, A., Rinzel, J. & Chen, Z. S. Stimulus-driven and spontaneous dynamics in excitatory-inhibitory recurrent neural networks for sequence representation. Neural Computation 33, 2603-2645 (2021).
[56] Liebe, S. et al. Phase of firing does not reflect temporal order in sequence memory of humans and recurrent neural networks. bioRxiv 10.1101, 2022.09.25.509370 (2022).
[57] Kim, R., Li, Y. & Sejnowski, T. J. Simple framework for constructing functional spiking recurrent neural networks. Proceedings of the National Academy of Sciences 116, 22811-22820 (2019).
[58] Keller, A. J. et al. A disinhibitory circuit for contextual modulation in primary visual cortex. Neuron 108, 1181-1193.e8 (2020).
[59] Song, H. F., Yang, G. R. & Wang, X.-J. Training excitatory-inhibitory recurrent neural networks for cognitive tasks: A simple and flexible framework. PLOS Computational Biology 12, e1004792 (2016).