A Multinomial Processing Tree Model of RC Attachment (2021)

Abstract: In the field of sentence processing, speakers’ preferred interpretation of ambiguous sentences are often determined using a variant of a discrete choice task, in which participants are asked to indicate their preferred meaning of an ambiguous sentence. We discuss participants’ degree of attentiveness as a potential source of bias and variability in such tasks. We show that it may distort the estimates of the preference of a particular interpretation obtained in such experiments and may thus complicate the interpretation of the results as well as the comparison of the results of several experiments. We propose an analysis method based on multinomial processing tree models (Batchelder and Riefer, 1999) which can correct for this bias and allows for a separation of parameters of theoretical importance from nuisance parameters. We test two variants of the MPT-based model on experimental data from English and Turkish and demonstrate that our method can provide deeper insight into the processes underlying participants’ answering behavior and their interpretation preferences than an analysis based on raw percentages.

## introduction one of the key questions in the field of sentence processing has been: what does the human sentence processing mechanism do when confronted with an ambiguity? a variety of different proposals regarding online disambiguation strategies have been made over the years, such as the garden-path theory @xcite , the tuning hypothesis @xcite , the competition-integration model @xcite and many others. their diverging predictions have led to a significant body of empirical research documenting, among other things, substantial cross-linguistic variation in the interpretation of ambiguous sentences: for instance, @xcite compared the rc attachment preferences of english and spanish speakers in ambiguous sentences like (1) and (2), in which the relative clause 'who had an accident' can attach either to the np headed by the first noun (n1, 'daughter') or to the np headed by the second noun (n2, 'colonel'). 1 cuetos and mitchell presented spanish-speaking and english-speaking participants with ambiguous sentences like (1) and (2) and asked them to answer comprehension questions like 'who had an accident?'. participants' responses indicated that english sentences like (1) were assigned an n2 interpretation in 61% of the cases, while their spanish counterparts like (2) were assigned an n1 interpretation in 72% of the cases. the authors interpret this finding as an argument against a crosslinguistically universal parsing strategy in the resolution of rc attachment ambiguities. the periodista journalist entrevisto interviewed a to la the hija n 1 daughter del of the coronel n 2 colonel [que [who tuvo had el an accidente]. accident]. although disambiguation strategies seem to be at least partially determined by the linguistic properties of a given language, various other factors appear to influence the resolution of rc attachment ambiguities. for example, in a questionnaire study, @xcite demonstrated a substantial influence of construction type. they asked participants to indicate which of the two available noun phrases was modified by the rc in several constructions. they found that the percentage of n2 attachment responses ranged between approximately 20% to 70% for their english sentences and 1 to any ambiguity in the context of typologically diverse languages, we will refer to the two interpretation options as n1 attachment and n2 attachment, with n1 and n2 referring to the order of occurrence of the noun phrases head nouns instead of the more common terms high attachment and low attachment. rc attachment preferences have also been studied in turkish, where the order of the rc and the complex noun phrase is reversed, compared to english and spanish. in a questionnaire study with sentences like (3), @xcite found that animacy may affect attachment preferences such that when both nps were @xcite , there was no significant difference between the proportions of the n1 and n2 attachment, while an n1 attachment manifested when both nps were [-human]. contrary to this finding, dinçtopal-deniz (2010) found an across-the-board preference for n1 attachment in turkish. in her questionnaire study, monolingual turkish speakers read turkish sentences with ambiguous rc-attachment and answered questions about them by indicating one of two options on each trial. the results of this study showed that participants preferred n1 attachment over n2 attachment: 66% percent of the responses indicated an n1 interpretation of the sentence. ## the role of guessing what most of the above studies of rc attachment preferences have in common is that they use some variant of a discrete choice task, in which participants select one of two response options to indicate their interpretation of the ambiguity. the relative proportion of responses indicating n1 and n2 attachment, respectively are interpreted as estimates of the magnitude of n1 or n2 attachment. a potential complication in interpreting the percentage of responses favoring an alternative in this way is that participants' responses may not always reflect their interpretation. at least on some trials, participants may process the sentence only partially or fail to pay attention to it altogether. in such cases, participants' question responses must be based on an incomplete or nonexistent representation, and are more likely to resemble guesses than informed responses. evidence for such incomplete processing comes from the widely known fact that participants' accuracy in experimental tasks is often far from perfect, even for relatively simple tasks such as acceptability judgments: for example, @xcite found in an offline acceptability judgment study that ungrammatical sentences like (4) are judged acceptable on 18% of the trials. since it appears unlikely that sentences like (4) are considered grammatical and interpretable when fully processed, the explanation for such responses must lie in their incomplete processing followed by guessing. one way of conceptualizing a simple generative model of erroneous responses in relatively simple discrete choice tasks is to assume that at least some participants on some occasions fail to pay attention to the stimulus, and as a result, select a random response. if so, the relation between the probability of response x being actually preferred to alternative responses (p x ) and the probability of observing response x (p ′ x ) can be formalized as in equation 1: p ′ x is the weighted average of (i) the probability of x being preferred to the alternative when the stimulus is fully attended to (p x ) and (ii) the probability of selecting x when the stimulus is not attended to (g x ), where a is the probability of attending to the stimulus. equation 1 illustrates that under the above assumptions, the proportion of responses indicating a preference for x conflates multiple factors. as a result, many preference estimates for x (p ′ x ) are compatible with a wide range of underlying preferences (p x ) under different assumptions regarding participants' degree of attentiveness and guessing behavior (a and g x ). given that participants in most if not all psycholinguistic tasks produce a sizeable amount of erroneous responses, it appears a priori quite plausible that such mechanisms are also at play in attachment preference studies. this means that empirical estimates of attachment preferences (p ′ x ) are likely to be (i) biased towards the guessing parameter g x to a degree determined by a, and (ii) are likely to vary between studies as a function of the between-study differences in a and g x . in the following, we propose a method for disentangling the contributions of attachment preferences and guessing using multinomial processing tree models (mpt; @xcite based on response patterns in unambiguous baseline sentences. we will first assess the empirical adequacy of two alternative mpt models on two experiments in english and turkish, in which participants answered polar comprehension questions about sentences with ambiguous and unambiguous rc attachment. we will then compare the two experiments with regard to the parameter estimates obtained from the mpt models. ## experiments to evaluate our method, which will be presented in the next section, we used question-answering data from two experiments in which participants read sentences with ambiguous and unambiguous rc attachments and answered polar comprehension questions about them. ## mpt models of question-answering and attachment in accounting for the influence of extraneous cognitive processes, we considered two mechanisms that may generate erroneous question responses, and implemented both as multinomial processing tree (mpt) models (batchelder and riefer, 1999). in the following sections, we will use the model with the better empirical fit to obtain less biased estimates of the attachment preferences in the ambiguous conditions. mpt models offer a way to formalize hypotheses about how a mixture of several latent processes generates a categorical response (cf. @xcite , for an overview). that is, under the assumption that different sequences of events may occur on different trials, the latent processes hypothesized to be involved in processing are represented as a probability tree, with each path through the processing tree corresponding to unique combinations of cognitive processes which give rise to a particular response, along with the probabilities of each path. importantly, this formalization provides a framework in which the probabilities of relevant latent processes can be estimated. we will use them to estimate the magnitude of the rc attachment preference in turkish and english. ## method we implemented both mpt models 2 in brms and rstan @xcite in r (r core team, 2018) according to equations 3-8. we fitted the models to each experiment separately, using 4 mcmc chains with 1, 000 warm-up and 3, 000 post-warm-up iterations. for the sake of computational convenience, we estimated all model parameters on the logit scale, and in the following, we will use θ ′ to refer to the logittransform of any parameter θ. we used mildly informative gaussian priors for all logit-transformed population parameters in both models: h ′ , g ′ ∼ n (0, 1), and a, r 1 , r 2 ∼ n (0, 1). to account for individual differences in all parameters, we used hierarchical models with bysubject intercepts for all parameters, where each participant k's responses were modeled as a function of population-level parameters θ with subject subject-level adjustments δ θ,k , with θ ′ @xmath0,k , where the by-subject adjustments are distributed as δ θ ′ ,k ∼ n (0, σ θ ′ ). ## model comparison figures 4 and 5 show the average percentages of 'yes' responses by experiment (circles and connecting lines) alongside 95% posterior predictive intervals generated by, model 1 and 2, respectively (error bars). figure 4 shows that although model 1 could approximate the experimental findings it systematically overestimated the proportion of responses compatible with the preferred rc attachment (n2 in english, n1 in turkish) in both unambiguous conditions: for example, in the n1 attachment con-english turkish elpd elpd model 1 -511.3 (18.1) -796.4 (15.0) model 2 -469.4 (14.8) -750.5 (13.5) ∆ elpd ∆ elpd model 2-1 41.9 (11.6) 45.9 (9.3) in order to compare the models more formally, we using psis-loo-cv @xcite to compute each model's expected log pointwise predictive density (elpd). elpd provides an estimate of the model's out-of-sample performance and thus penalizes additional model flexibility, which puts models 1 and 2 on an equal footing although model 2 has more parameters. both ∆ elpd estimates are relatively large relative to their standard errors, and thus point towards model 2 having better out-of-sample performance. this finding suggests that the two attachment processes are affected by the error-generating process to different degrees. ## results having established model 2 as an adequate model of rc attachment in the context of questionanswering, we used its parameter estimates to understand the pattern of responses in the experimental data: figure 6 shows the model 2 population parameter estimates for both experiments as well as 95% credible intervals for all four parameters. in addition to the difference in the guessing bias g between experiments, it also shows a lot of uncertainty in the estimates of the attachment parameter h, which represents the probability with which the parser adopts an n1 attachment interpretation over an n2 attachment structure in ambiguous attachment conditions. both estimates of the n1 attachment probability have rather wide credible intervals, with 42% (@xmath1; 55]) for the english experiment and 48% (@xmath2; 70]) for the turkish experiment. while the estimate for english is consistent with weak evidence for an n2 attachment preference, the estimate for turkish indicates no clear preference. the explanation for the surprising absence of evidence for an n1 attachment preference in the parameter h in turkish lies in the the substantial difference between the successful recall probabilities r 1 (49%, @xmath3; 59]) and r 2 (9%, @xmath4; 20]), which indicate that n1 interpretations were successfully processed and recalled with a higher probability than their n2 counterparts. according to the assumptions of model 2, this leads to a question response pattern which appears to suggest an n1 preference even when there isn't one (@xmath5.5): when participants decide to adopt an n1 interpretation, their question responses indicate n1 attachment on most trials -sometimes due to successful recall of the n1 interpretation, and at other times as a result of guessing. when participants decide to adopt an n2 interpretation, however, they fail to recall the correct interpretation most of the time, and thus engage in guessing. importantly, guesses result in n1 responses 50% of the time, since questions about n1 and n2 interpretations are balanced. as a result, a substantial difference between r 1 and r 2 , such that r 1 < r 2 will lead to more n1 responses than n2 responses to questions about ambiguous sentences because n1 interpretations are more successfully recalled, even if ambiguous sentences are assigned n1 interpretations only 50% of the time. whatever the source of higher error rates in the n2 attachment conditions in the turkish experiment is, our mpt analysis suggests that what appears as a weak n1 attachment preference in our turkish experiment is actually a consequence of a large number of guessing trials associated with n2 attachment. in sum, our analysis shows that (i) the n2 attachment preference in the english experiment appears to hold up even when guessing trials are taken into account, and (ii) that what appears to be an n1 attachment preference in turkish is readily explained by the processing difficulty associated with processing and recalling n2 attachment structures in turkish. ## summary based on the assumption that readers sometimes do not allocate the required amount of attention to the task they are performing, we have discussed a previously neglected source of bias and variability that may affect studies of attachment preferences and of interpretation preferences more generally. we attempted to account for the role of guessing as a strategy used in answering comprehension questions when the answer is not known. we argue that understanding the role of guessing in discrete choice tasks is crucial because data consisting of responses to comprehension questions where participants sometimes fail to arrive at a full interpretation of the structure may be confounded. to this end, we proposed an mpt-based analysis method that allows to de-confound parameters of theoretical importance from nuisance parameters such as the guessing rate. we tested two variants of the mptbased model on experimental data from english and turkish, and demonstrated that this method can provide further insight into the processes underlying participants' answering behavior as well as their attachment preferences.