Modeling Sentence Comprehension Deficits in Aphasia: A Computational Evaluation of the Direct-access Model of Retrieval (2021)

Abstract: Several researchers have argued that sentence comprehension is mediated via a content-addressable retrieval mechanism that allows fast and direct access to memory items. Initially failed retrievals can result in backtracking, which leads to correct retrieval. We present an augmented version of the direct-access model that allows backtracking to fail. Based on self-paced listening data from individuals with aphasia, we compare the augmented model to the base model without backtracking failures. The augmented model shows quantitatively similar performance to the base model, but only the augmented model can account for slow incorrect responses. We argue that the modified direct-access model is theoretically better suited to fit data from impaired populations.

## introduction comprehending a sentence involves building linguistic dependencies between words. in the sentence processing literature, several researchers have argued that linguistic dependency resolution is carried out via a cue-based retrieval mechanism @xcite . cue-based retrieval theory assumes that word representations are retrieved from working memory via their syntactic and semantic features. consider the following sentences: in (1a), the noun boy would be encoded in memory with features such as @xcite . when the reader reaches the verb tickled, a retrieval is triggered with retrieval cues that match the features of boy. at this point in time, boy is the only element that matches the retrieval cues of the verb. by contrast, in (1b), another noun intervenes between tickled and boy that partially matches the cues set at the retrieval: girl [+animate, -subj]. the partial feature overlap causes similarity-based interference between the two items, making the dependency more difficult to resolve in (1b) compared to (1a). interference effects have been attested in multiple studies, see for example @xcite ; @xcite ; @xcite ; @xcite . one model of cue-based retrieval that predicts these interference effects is the directaccess model developed by @xcite @xcite . the direct-access model (da) assumes that retrieval cues allow parallel access to candidate items in memory, as opposed to a serial search mechanism. due to the parallelism assumption, the speed of retrieval is predicted to be constant across items (aside from individual differences and stochastic noise in the retrieval process). factors such as increased distance between the target and the retrieval point and the presence of distractor items can lower the probability of retrieving the correct dependent (also known as availability). low availability of the target dependent can lead to failures in parsing or to misretrievals of competitor items. when such errors occur, a backtracking process can be initiated, which by assumption leads to the correct retrieval of the target @xcite . the backtracking process requires additional time that is independent of the retrieval time. according to the direct-access model, (1a) should have shorter processing times than (1b) on average, because in (1b) some trials require costly backtracking due to lower availability of the target item boy. the direct-access model can be adapted to explain impaired sentence comprehension in individuals with aphasia (iwa; @xcite . however, there is one crucial aspect of the directaccess model that is at odds with the aphasia literature, specifically with the finding that iwa have longer processing times for incorrect than for correct responses (e.g., @xcite . the direct-access model assumes that some percentage of correct interpretations are only obtained after costly backtracking, and thus predicts that the average processing time for incorrect responses should be faster than for correct responses. to address this issue, we implement a modified version of the direct-access model that is specifically relevant for sentence processing in iwa. in this model, backtracking can lead to correct retrieval of the target, as in the base model, but can also result in misretrieval and parsing failure. ## data the data that we model come from a self-paced listening task in german @xcite . 50 control participants and 21 iwa completed the experiment. sentences were presented auditorily, word by word. participants paced the presentation themselves, choosing to hear the next word by pressing a computer key. the time between key presses (here called listening time) was recorded. at the end of the sentence, two images (target and foil) were presented, and participants had to select which image matched the meaning of the sentence they had just heard. accuracies for the picture-selection task were also recorded. to assess test-retest reliability, each subject completed the task twice, with a break of two months in between. our modeling is based on the pooled data of both sessions. ## direct-access model our implementation of the direct-access model follows @xcite . the model assumes that listening times for correct responses come from a mixture distribution, given that there are trials with backtracking, where an additional processing cost δ is added, and trials without backtracking, where no such cost is added. by contrast, incorrect responses never involve backtracking, and the average listening time should be the same as for correct responses without backtracking. a graphical representation of the model is displayed in figure (2). the three possible cases are as follows: the model includes both fixed and random effects in order to account for sentence complexity, group differences, and individual variability. the hierarchical structure is shown in equation (1). all parameters have an adjustment by group (iwa versus control), because we expect iwa to have different parameter estimates from control participants. since da assumes that retrieval times are not affected by sentence complexity, the average listening times (µ) do not have an adjustment for condition. by contrast, the probability of retrieval of the target, θ, includes a condition adjustment. this parameter can be thought of as indexing memory availability. the probability of backtracking p b , the cost of backtracking δ, and σ do not depend on sentence complexity, but may vary between iwa and controls. the hierarchical structure is embedded within the parameters when possible (we report the maximal hierarchical structure that could be fit). in equation (1), the terms u and w are the by-participant and by-item adjustments to the fixed effects, respectively. these are assumed to come from two multivariate normal distributions. all parameters had regularizing priors, listed in appendix b. the model was implemented in the probabilistic programming language stan (stan development team, 2020), and fit via the rstan package @xcite in r (r core team, 2020). the model was fit with 3 chains and 8,000 iterations, half of which were used as warm-up. ## modified direct-access model based on the original da model's suboptimal fit, we propose a modified version (mda). in this version, the distribution of listening times for both correct and incorrect responses is a mixture of directly accessed and backtracked retrievals. the mda model assumes that backtracking can fail. in terms of implementation, the main difference between the models is a newly-introduced parameter θ b , which is the probability of correct retrieval after backtracking. figure 5 : graphical representation of the modified direct-access model. the model was run with 10,000 iterations, half of which were used as warm-up. ## model comparison in order to quantitatively compare the performance of the models, we computed bayes factors. we chose bayes factors over other alternatives (e.g. cross-validation), because the two models seem to predict similar distributions, and bayes factors are especially suited for nested models, or models that make very similar predictions. the hypothesis being tested is whether there is a non-zero parameter θ b that indexes the probability of successful backtracking, assumed by the mda model, or whether backtracking is always successful, as assumed by the base da model. in order to perform the comparison, the models were run for 40,000 iterations, of which 3,000 were used for warm-up. bayes factors were computed using the bridgesampling package @xcite in r. the bayes factor of da over mda was estimated to be 2. this result is inconclusive, and indicates that the models provide similar quantitative fit to the data. ## discussion and conclusion in the present paper, we implemented and tested two versions of the direct-access model of cuebased retrieval and evaluated their predictive performance on data from individuals with aphasia and control participants. specifically, we modeled interference in an under-studied linguistic construction, namely control structures. both the base model and the modified model are in line with a combination of processing deficits in iwa: slow syntax, resource reduction, and intermittent deficiencies. neither of the two models showed support for delayed lexical access as a source of retrieval difficulty specifically for iwa. although a delay in ldt was connected to a decrease in the probability of correct retrieval, the effect of ldt was similar for iwa and control participants. in general, our results are consistent with other studies showing that a combination of processing deficits may be the source of impairments in sentence comprehension in iwa @xcite @xcite . unlike the base direct-access model, our modified da model (mda) assumes that backtracking can fail, resulting in slow, incorrect retrievals. however, this added assumption does not result in a decisive advantage in fit for the mda model, as shown by the posterior predictive checks and the bayes factor analysis. this result is unexpected, and leads us to think that the mda model may be overparametrized. in mda, all of the main parameters include a group adjustment. as a consequence, for instance, the mean listening times, µ, are estimated to be higher for iwa than for controls. the cost of backtracking, which is only added to µ if backtracking is performed, accounts for slower re-sponses. however, because iwa's µ is estimated to be higher than controls' µ, the model may not need to rely on backtracking in order to account for slow responses in iwa. this could be the reason why the probability of backtracking for iwa is very low (7%) relative to controls (80%). in addition, iwa's θ b has to be estimated from the 7% of trials that include backtracking. given the size of the iwa group (21 participants), and the small amount of trials that include backtracking, perhaps the model cannot correctly estimate the θ b parameter. this could be investigated in several ways. one possibility would be to remove the group adjustments from µ, p b , δ, and θ b one at the time, and see which of these models shows a better quantitative fit for the data (see @xcite . another possibility would be to evaluate how these parameters interact with and without group adjustments (e.g., do p b and/or δ for iwa increase if there is no group adjustment in µ?). we will address these questions in future work. the present paper contributes to the aphasia literature by proposing a modification of the directaccess model that can account for incorrect slow responses. despite our inconclusive results, we believe that the modified direct-access model offers a more appropriate set of assumptions for individuals with aphasia than the direct-access model. the modified-direct access model can account for slow incorrect responses, which are frequently found in studies on sentence processing in iwa (e.g., @xcite @xcite . it remains to be seen, by testing the new modified direct-access model against more data from individuals with aphasia, whether there is a difference in predictive performance between the two models.