Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies (2024)

Abstract: As artificial neural networks grow in complexity, understanding their inner workings becomes increasingly challenging, which is particularly important in healthcare applications. The intrinsic evaluation metrics of autoregressive neural language models (NLMs), perplexity (PPL), can reflect how “surprised” an NLM model is at novel input. PPL has been widely used to understand the behavior of NLMs. Previous findings show that changes in PPL when masking attention layers in pre-trained transformer-based NLMs reflect linguistic anomalies associated with Alzheimer’s disease dementia. Building upon this, we explore a novel bidirectional attention head ablation method that exhibits properties attributed to the concepts of cognitive and brain reserve in human brain studies, which postulate that people with more neurons in the brain and more efficient processing are more resilient to neurodegeneration. Our results show that larger GPT-2 models require a disproportionately larger share of attention heads to be masked/ablated to display degradation of similar magnitude to masking in smaller models. These results suggest that the attention mechanism in transformer models may present an analogue to the notions of cognitive and brain reserve and could potentially be used to model certain aspects of the progression of neurodegenerative disorders and aging.

## introduction alzheimer's disease (ad) dementia is a currently incurable neurodegenerative condition that leads to a progressive and irreversible decline in cognitive function. due to the challenging nature of early diagnosis of this condition, there is a pressing need for efficient and cost-effective screening tools @xcite to mitigate the negative consequences of delayed or absent diagnosis @xcite . previous studies have demonstrated that changes in cognitive status can be reflected in spoken language and spontaneous speech @xcite @xcite . automated analysis of such speech, employing supervised machine learning models, shows its potential as an early screening tool. these models can be trained to identify subtle linguistic anomalies associated with dementia from transcripts of both healthy individuals and those with dementia. recent advances in machine learning, such as deep learning models and the transformer with attention architecture @xcite , have mediated remarkable performance on this downstream task (for a review, see @xcite ). deep learning models, inspired by the human brain, are artificial neural networks (anns) that process vast amounts of data and learn complicated patterns, making them well-suited for analyzing subtle linguistic patterns. the transformer architecture, in particular, has advanced performance on natural language processing (nlp) tasks by enabling models to capture long-range dependencies more effectively via the attention mechanism @xcite . as anns get larger and more complicated, it becomes even harder to interpret their inner workings. the performance of autoregressive neural language models (nlms) (e.g., predicting the next word given the context) is frequently estimated with a single somewhat interpretable feature, perplexity (ppl), which has shown to be a suitable measurement for evaluating cognitive impairment from spontaneous speech @xcite . as the name "perplexity" suggests, it can be considered as an indicator of how "surprised" a model is by novel (i.e., not used in model's training) input. the more different the input is from a particular model's training data, the "harder" it is for the model to predict, resulting in higher ppl. therefore, it is reasonable to hypothesize that ppl may have some degree of diagnostic utility, as an indicator of patterns of language use that fall outside the scope of the typical language used to train a model. in the context of ad, changes in language and cognitive function often manifest as differences in language complexity, with individuals experiencing difficulty in forming coherent sentences and selecting appropriate words. as ad progresses, the language used by patients with dementia becomes more unpredictable and less coherent, leading to higher ppl with models trained on language from individuals presumed to be cognitively healthy. while training data from cognitively healthy individuals is plentiful, language data produced by patients with dementia is much more impractical to obtain in sufficient quantity to train a large nlm. in hyperdimensional computing @xcite , high-dimensional vector representations are manipulated using operators that alter their distance from other learned representations. a prior work inspired by this concept @xcite demonstrates that masking the attention sub-modules of pre-trained transformer-based nlms and thereby artificially increasing ppl on text from cognitively healthy individuals, can provide an effective solution to the challenge of limited data availability. by strategically altering these sub-modules and introducing controlled perturbations in the nlms' attention layers, the degraded nlms induce the linguistic anomalies and unpredictability associated with dementia. recent work in neuroscience using functional magnetic resonance imaging (fmri) and electrocorticography (ecog) has demonstrated that nlm's ppl is associated with predicting neural activation patterns during language comprehension tasks in the human brain @xcite . this suggests a potential connection between the predictive capabilities of these models and understanding human information processing. in particular, one of the less well-understood phenomena in how neurodegeneration affects the human brain is the notion of cognitive and brain reserve. this notion is hypothesized to be responsible for findings that indicate individuals with higher innate abilities and/or aspects of life experience, such as educational and professional attainment, are able to mask the effects of dementia longer than those without these characteristics @xcite @xcite @xcite . in some cases, the notion of cognitive and brain reserve may even allow individuals to revert from initial signs of cognitive impairment to normal function @xcite . building upon these findings, our study seeks to further explore the potential of probing pre-trained gpt-2 family models @xcite to simulate cognitive impairment observed in patients with dementia with a specific focus on the cognitive reserve hypothesis. using a set of transcripts from a widely-used "cookie theft" picture description cognitive task, we propose that the impaired information processing as the disease progresses can be simulated by masking a certain share of attention heads in a pre-trained gpt-2 model. specifically, we follow the previously established pairedperplexity paradigm @xcite using a pair of unmasked ("control") and masked ("dementia") nlms. in this approach, the difference between ppls produced by these two nlms is used to discriminate between picture descriptions by patients with dementia and healthy controls. we hypothesize that larger gpt-2 models with more attention heads will exhibit greater resilience to masking (i.e., a proxy for neural degeneration), necessitating a larger share of attention heads to be masked to achieve comparable classification performance to smaller models. we evaluate this hypothesis by targeting two subsets of attention heads that are a) most important, and b) least important to representation of the content of the "cookie theft" task, in which the degree of importance is ranked by the gradient changes in each attention head during finetuning of a pre-trained gpt-2 model to the content of the "cookie theft" transcripts. the contributions of this work can be summarized as follows: a) we provide preliminary evidence suggesting that the concept of cognitive reserve observed in human cognition appears to have an analog in anns; and b) our attention masking approach achieves comparable classification performance to another approach developed in prior work that directly artificially degrades nlm parameters @xcite , and the state-of-the-art (sota) model trained from scratch (taghibeyglou and rudzicz, 2024) with significantly fewer trainable parameter masking/fitting. foot_0 2 background ## discussion the results of experiments presented in this paper suggest that the notion of cognitive reserve in the brain may have an analogue in transformer-based anns that is localized to the attention mechanism. recent neuroscientific evidence shows that nlms' ppl is predictive of human behavioral responses and and neural responses in functional mri studies @xcite . based on this evidence, we interpret our findings of the differences in log ppl changes as a result of masking attention heads in nlms of variable size as at least suggestive that the resilience to damage is non-linear to the number of attention heads in nlms. in other words, it takes disproportionately more masking to damage larger nlms to elicit the same level of degradation in performance, as compared with smaller nlms. furthermore, the dissociation in performance as a result of damaging attention heads vs. the token embedding weights suggests that the nlm's artificial neural reserve effects are localized to the attention mechanism. our results also suggest that masking attention heads within the paired-perplexity paradigm using the ratio of unmasked ("control") and masked ("dementia") pre-trained gpt-2 models results in good classification performance without requiring a corresponding large dataset produced by dementia patients and extensive parameter tuning. this can be achieved with as little as masking only 9% of attention heads of a pre-trained gpt-2 small model. in contrast to previous studies, which typically involved purging attention heads determined to be the least important, our bidirectional masking method adds supporting evidence of content units @xcite , suggesting the importance of these contextual features in addition to the predominant emphasis on linguistic structure and representation modeling in previous research (e.g., @xcite ). the results of bidirectional masking also offers an interpretable explanation for transfer learning's remarkable performance using pre-trained nlms. it suggests that during fine-tuning, pre-trained nlms use a combination of both task-specific (the most important) and task-agnostic (the least important) heads to achieve remarkable performance on various downstream tasks. those task-agnostic attention heads may play an important role in transfer learning. this also may explain why distilled nlms in which the "nonvolitional cues" that fall outside of common nlp benchmarks are purged during the distillation, generalize less-than-ideally to other types of data produced by individuals with dementia @xcite . with larger models, there are considerably more attention heads that can serve as those "nonvolitional cues," helping a larger nlm perform better @xcite . as the columns of the token embedding matrix in a pre-trained nlms represent the global semantics of tokens in the vocabulary, the observations that the best-performing masking pattern appears in the later stage of the token embedding matrix masking are consistent with previously published findings that semantic impairment often occurs in the later stage (i.e., moderate) of the disease @xcite @xcite . as illustrated in figure 5 , when masking the later 66% columns (675 out of 1024) of the word embedding matrix, the paired unmasked and masked gpt-2 medium achieves an acc of 0.85 on the adress test set. this finding is consistent with a previous work @xcite , suggesting that some syntactic information is embedded implicitly in the word embedding matrix. this also provide an empirical support of our findings that masking word embedding matrix of a pre-trained nlm can provide some degree of discriminating effect on this downstream task. however, masking the word embedding matrix is far less effective than masking attention heads to simulate dementia-related cognitive impairment. our results suggest that similar mechanisms of resilience may exist in both human cognition and computational models. this could lead to more nuanced strategies in response to develop early screening tools for the delayed onset of cognitive impairment in the population with high risk. our study holds promise for deploying such early-screening method in resource-constrained clinical settings to improve early intervention and patient management for ad. ## conclusion we presented experimental findings suggesting the presence of artificial neural reserve in transformerbased nlms, analogous to the concepts of brain/cognitive reserve in studies of human cognition. in addition, we introduced a novel bidirectional attention head ablation method that enables using unmasked and masked gpt-2 models in the paired-perplexity paradigm for detecting linguistic anomalies with significantly less parameter masking or fitting.