The architectural variances between transformers and recurrent models, notably the presence of a working memory bottleneck in recurrent models, have a substantial impact on their capacity to capture distinct aspects of human language processing such as local interference effects and lexical priming. Recurrent models, characterized by their limited working memory, are adept at modeling local interference effects. This is attributed to their ability to simulate the constraints of human working memory, enabling them to effectively capture the interference that occurs during real-time language processing. Conversely, transformers, with their fixed context window and unrestricted access to all preceding words, may encounter challenges in accurately modeling local interference effects compared to recurrent models.
Regarding lexical priming, transformers may possess an advantage due to their direct access to prior words, facilitating a more precise representation of priming effects where the comprehension of one word influences the processing of subsequent words. This feature equips transformers to leverage the statistical regularities in the data more efficiently, potentially making them more proficient in modeling lexical priming effects in contrast to recurrent models.
In essence, the architectural distinctions between transformers and recurrent models play a pivotal role in shaping their efficacy in capturing specific facets of human language processing. Recurrent models excel in modeling phenomena like local interference effects by virtue of their working memory limitations, while transformers may demonstrate superiority in capturing lexical priming effects owing to their broader contextual access to words.