K. Clark, U. Khandelwal, O. Levy, and C. D. Manning. What does BERT look at? an analysis of BERT's attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276-286, Florence, Italy, Aug. 2019. Association for Computational Linguistics. doi: 10.18653/v1/W19-4828. URL https://aclanthology.org/W19-4828.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
W. Francis and H. Kucera. The brown university standard corpus of present-day american english. In Classified and Annotated Corpus of Present-Day American English, volume 1, pages 3-36. Brown University Press, 1979.
G. Hickok and D. Poeppel. The cortical organization of speech processing. Nature Reviews Neuroscience, 8(5):393-402, 2007. ISSN 1471-0048. doi: 10.1038/nrn2113. URL https://doi.org/10.1038/nrn2113.
S. Hochreiter and J. Schmidhuber. Lstm can solve hard long time lag problems. Advances in neural information processing systems, 9, 1996.
M. Honnibal and I. Montani. spacy: Industrial-strength natural language processing in python. Software, 2020. URL https://spacy.io/.
S. Jain, V. A. Vo, L. Wehbe, and A. G. Huth. Computational language modeling and the promise of in silico experimentation. Neurobiology of Language, 2023. doi: 10.1162/nol_a_00101. URL https://doi.org/10.1162/nol_a_00101. Advance publication.
M. Keshishian, S. Norman-Haignere, and N. Mesgarani. Understanding adaptive, multiscale temporal integration in deep speech recognition systems. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 24455-24467. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ccce2fab7336b8bc8362d115dec2d5a2-Paper.pdf.
Y. Lakretz, G. Kruszewski, T. Desbordes, D. Hupkes, S. Dehaene, and M. Baroni. The emergence of number and syntax units in lstm language models, 2019.
Y. Lerner, C. J. Honey, L. J. Silbert, and U. Hasson. Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. Journal of Neuroscience, 31(8):2906-2915, 2011. ISSN 0270-6474. doi: 10.1523/JNEUROSCI.3684-10.2011. URL https://www.jneurosci.org/content/31/8/2906.
H. Lin and M. Tegmark. Critical behavior from deep dynamics: A hidden dimension in natural language. 06 2016.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3285-3295, 2019.
E. Loper and S. Bird. Nltk: The natural language toolkit, 2002. URL https://arxiv.org/abs/cs/0205028.
W. Luo, Y. Li, R. Urtasun, and R. Zemel. Understanding the effective receptive field in deep convolutional neural networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/c8067ad1937f728f51288b3eb986afaa-Paper.pdf.
S. Mahto, V. A. Vo, J. S. Turek, and A. G. Huth. Multi-timescale representation learning in LSTM language models. arXiv preprint arXiv:2009.12727, 2020.