The source ambiguity problem : distinguishing the effects of grammar and processing on acceptability judgments Running header : The source ambiguity problem

Judgments of linguistic unacceptability may theoretically arise from either grammatical deviance or significant processing difficulty. Acceptability data are thus naturally ambiguous in theories that explicitly distinguish formal and functional constraints. Here, we consider this source ambiguity problem in the context of Superiority effects: the dispreference for ordering a wh-phrase in front of a syntactically ”superior” wh-phrase in multiple wh-questions, e.g. What did who buy? Evidence from acceptability and self-paced reading experiments demonstrates that (i) judgments for Superiority violations vary significantly with processing manipulations, (ii) judgments increase with exposure while processing times decrease, (iii) reading times are highly predictive of acceptability judgments, and (iv) preferences for the complexity and animacy of the wh-phrases produce additive or super-additive effects. This evidence supports the conclusion that Superiority effects owe their character, at least in part, to differences in processing, and that similar evidence may be useful in diagnosing other syntactic phenomena. 1 Acceptability Judgments Intuitions of native speakers reveal whether sentences are “good” or “bad”. But these pretheoretic judgments do not themselves determine whether a sentence is judged “bad” because it is ungrammatical or whether it is deviant in some other way, e.g. it is impossible to parse [Fodor (1978), p. 428]. Linguists rely on the ability to tell apart grammatical sentences from ungrammatical sentences. Such decisions typically depend on judgments of linguistic acceptability made by the linguist or perhaps by a handful of fellow linguists. The problem for the language theorist, however, is that similar judgments may result from different sources: perceptions of unacceptability may result from ungrammaticality, or they may result from significant processing difficulty. For instance, Miller and Chomsky (1963) point out that sentences with multiple center-embeddings, as in (1a), are nearly incomprehensible and sound highly deviant to most speakers of English: (1) a. The boy the girl the host knew brought left. b. The boy the girl brought left. The structure in (1a) can be derived by repeated application of the grammatical rule that licenses the relatively acceptable sounding variant in (1b), e.g. “NP→ NP S”. Hence, they suggest that the unacceptability of such examples can be attributed to the severe cognitive processing difficulties accompanying such structures, rather than a violation of grammar. In other cases of linguistic unacceptability, however, the source of the deviance is less clear. Consequently, this “source ambiguity problem” makes it challenging to understand what constraints grammar imposes, and what constraints should be explained in terms of more general cognitive processes.1 An example of this problem emerges in the discussion of multiple wh-questions, as in (2): (2) Who finished what? Such questions commonly elicit pair-list readings, where an appropriate answer contains information about both the agent and the patient, e.g. “John’-”the stats”, “Hank”-”the formatting”, etc.2 As first noted by Kuno & Robinson (1972), examples like (3) are a considerably less natural way to ask the question posed by (2): (3) What did who finish? Hence, the question arises: is (3) worse than (2) because the grammar for English states restrictions that prevent the structure in (3), or do limitations on processing predict that (2) should be preferable to (3), or possibly some combination of the two? Historically, the contrast between (2) & (3) has been ascribed to universal grammatical constraints. More specifically, the rarity of utterances like (3) purportedly stems from grammatical constraints that prevent moving one wh-phrase over a syntactically superior wh-phrase (Chomsky, 1973). A competing line of thought suggests that differences in processing difficulty separate examples like (2) & (3) (Arnon, Hofmeister, Jaeger, Sag, & Snider, 2005; Hofmeister, Jaeger, Sag, Arnon, & Snider, 2007; Culicover, 2008). In particular, this line of argumentation asserts that multiple wh-questions like (3) pose significant processing difficulties that are absent in cases like (2) and that yield perceptions of unacceptability. Thus, we are left with the question of how to discriminate between these two potential explanations for the observed contrast in acceptability. Answering this question is complicated by the fact that few diagnostics exist for distinguishing grammarrelated sources of unacceptability from processing-related sources of unacceptability. Granted that ungrammaticality and processing difficulty can both lead perceptions of unacceptability, what kinds of evidence could reveal the source of the unacceptability? If greater processing difficulty results in lower acceptability judgments, and reduced grammaticality leads to slower or more inefficient processing, then the linguist is left in a serious quandary. There is no accepted experimental methodology that distinguishes one from the other. 3 In this paper, we offer some guidelines for discriminating between these sources, particularly with respect to the case of wh-ordering in multiple wh-questions. Note that we do not intend to suggest that these guidelines resolve the issue, or make it trivial to identify the source of unacceptability. Nevertheless, we argue that applying these guidelines to other cases can make it easier to decide between the two available kinds of explanations. In addition, the guidelines and types of evidence we present are by no means exhaustive, but we hope that they stimulate further research into what types of evidence can tease apart the effects of grammar and processing. In large part, being able to distinguish the effects of grammar and processing depends on the assumptions made about each source. Not all conceptions of grammar are the same, nor are all models of sentence processing. In the current investigation, we compare purely grammatical accounts that analyze Superiority effects in terms of categorical grammatical constraints with those that argue that Superiority effects owe their character to online processing constraints to some significant extent. From the purely grammatical perspective, Superiority phenomena reflect constraints on constituent ordering that operate with a high (if not absolute) degree of autonomy from other cognitive systems. Purely grammatical constraints, in general, need not necessarily align with processing difficulty. That is, it is theoretically possible for a sentence with a grammatical violations to be easier to process than a sentence without such a violation (see Gibson and Thomas (1999); Vasishth, Suckow, Lewis, and Kern (in press)). In short, purely grammatical constraints characterize shared linguistic knowledge within a community of speakers but are not contingent upon active processing preferences. Alternatively, Superiority effects may result, at least in part, from online processing constraints. Cognitive difficulties may arise during the processing of a sentence that produce the perception of aberrance or linguistic unacceptability. According to some portrayals of this relationship, when the computational demands of a sentence pass some threshold, causing a breakdown in processing, this breakdown generates the perception of ‘ungrammaticality’ (Kluender & Kutas, 1993b; Kluender, 1998). This kind of analysis assumes that processing difficulty accumulates during sentence processing, such that several sources of difficulty may combine to produce additive or even super-additive effects, given a sufficient degree of difficulty. As we discuss in further detail, this perspective does not exclude the possibility that effects of grammaticization also contribute in some way to Superiority effects. In other words, this perspective stresses that online sentence processing constraints predict and explain a considerable amount of the observed variation in acceptability judgments. Such an emphasis allows that some form of partially or completely grammaticalized preferences also may be acting in tandem with the active processing constraints. While we focus on grammatical accounts that make categorical predictions, there are also grammarbased approaches that are gradient or probabilistic in nature. Notably, some of these theories make radically different predictions about how syntactic constraints should align with processing difficulty and how constraints change with experience. Since the division between processing and grammar is blurred in these cases, we postpone discussion of such theories until the final section of this paper. In the next section, we provide a brief review of the facts surrounding Superiority effects, some of the previous syntactic analyses, and what processing difficulties may determine such effects. Subsequently, we present experimental data from acceptability studies and self-paced reading experiments that test some of the predictions made by a processing account, and that provide an empirical basis on which to resolve the source ambiguity problem. 2 Superiority effects Kuno and Robinson (1972) propose a constraint that blocks the movement of a wh-phrase over another wh-phrase to explain the contrast in (4a) & (4b). (4) a. Who read what? b. *What did who read? Chomsky (1973) states that this condition on wh-ordering actually represents an instance of a broader limitation on constituent movement, termed Superiority: (5) No rule can involve X, Y in the structure: . X . [α . Z . −WYV . ] where the rule applies ambiguously to Z and Y and Z is superior to [m-commands] Y. Subsequent renditions of this constraint, e.g. Attract Closest or the Minimal Link Condition (Chomsky, 1995, 2000), retain much the same spirit, although notions like m-command give way to closeness. In all their renditions, grammatical accounts of Superiority remain highly specific, and