SPAWN ing Structural Priming Predictions from a Cognitively Motivated Parser (2024)

Abstract: Structural priming is a widely used psycholinguistic paradigm to study human sentence representations. In this work we introduce SPAWN, a cognitively motivated parser that can generate quantitative priming predictions from contemporary theories in syntax which assume a lexicalized grammar. By generating and testing priming predictions from competing theoretical accounts, we can infer which assumptions from syntactic theory are useful for characterizing the representations humans build when processing sentences. As a case study, we use SPAWN to generate priming predictions from two theories (Whiz-Deletion and Participial-Phase) which make different assumptions about the structure of English relative clauses. By modulating the reanalysis mechanism that the parser uses and strength of the parser’s prior knowledge, we generated nine sets of predictions from each of the two theories. Then, we tested these predictions using a novel web-based comprehension-to-production priming paradigm. We found that while the some of the predictions from the Participial-Phase theory aligned with human behavior, none of the predictions from the the Whiz-Deletion theory did, thus suggesting that the Participial-Phase theory might better characterize human relative clause representations.

## introduction structural priming @xcite is a widely used paradigm in psycholinguistics to study the structural representations that people construct when processing sentences. in this paradigm, researchers measure the extent to which the production or processing of target sentences is facilitated (or primed) by preceding prime sentences, and then use the pattern of priming behavior to draw inferences about the representations people construct. for example, consider a target sentence like (1). prior work @xcite found that targets like (1) were produced more often, and were processed more rapidly, when they were preceded by primes like (2), that have the same structure, than when they were preceded by primes like (3), which, while describing the same transfer event as (2), have a different structure. from this result, branigan et al. inferred that participants' mental representation of (1) is more similar to that of (2) than of (3). @xcite propose that by carefully studying which sentences prime each other we can build a theory of human structural representations. building such a theory requires us to generate hypotheses about the particular primetarget pairs that would be most informative to compare. insights from theoretical syntax, a field that has spent decades studying the structure of sentences, can help constrain this hypothesis space @xcite : if two theories generate different priming predictions, the theory whose prediction better aligns with human behavior better characterizes the representations humans build. in this work we introduce a new parser, the serial parser in act-r with null elements (spawn), that can generate quantitative priming predictions from theories in syntax. spawn is a cognitively motivated parser in which the parsing decisions are driven by the computational principles proposed by a general purpose cognitive architecture, adaptive control of thought-rational (act-r; @xcite . thus, spawn not only describes the computations underlying human parsing, but also specifies the cognitive processes involved. this level of specification makes it possible to explain why, given a grammar, some sentence a is primed more by sentence b compared to c, which in turn is use-ful for generating quantitative behavioral priming predictions from syntactic theories. existing algorithmic models of parsing with this level of specification @xcite are limited in their ability to model assumptions from theories that use more contemporary frameworks like minimalism for two reasons: first, they assume a disconnect between lexical and grammatical knowledge, which is inconsistent with the lexicalized grammar formalisms these frameworks adopt; second, the models do not specify mechanisms to handle null (or covert) lexical items, which are essential components of several contemporary syntactic theories. spawn bridges this gap by adopting a lexicalized grammar formalism and specifying an explicit mechanism for null items. as a case study, we use spawn to study the mental representations of sentences with relative clauses (rcs) such as (4) and ( 5 ). we generate priming predictions from two competing syntactic theories: whiz-deletion @xcite , which assumes that the structure of ( 4 ) is identical to the structure of ( 5 ), but that the words "which" and "was" are covert; and participial-phase @xcite which assumes that (4) and ( 5 ) have different structures. we describe these theories in more detail in § 4. we generate nine sets of predictions from the two theories by modulating two factors: first, the strength of prior knowledge (model exposed to 0, 100 or 1000 sentences before the experiment); second, the reanalysis mechanism (model goes back to the beginning of the sentence, or model uses one of two entropy-based measures to select a word to go back to). then, we compare the predictions from these two theories to empirical human data we collected using a novel web-based comprehension-to-production priming paradigm. we found that the predictions from the whiz-deletion never aligned with the qualitative pattern of human priming behavior, whereas under some assumptions about the underlying reanalysis mechanism and strength of prior knowledge, predictions from the participial-phase theory did align with the qualitative empirical pattern. these results suggest that the participial-phase account better characterizes human sentence representations. more broadly, this case study highlights how spawn can be used to adjudicate between competing the-does the model specify algorithms underlying sentence processing or production? processing difficulty during parsing @xcite @xcite priming @xcite @xcite is the model designed to account for psycholinguistic phenomena? can the model account for sentence processing? act-r model of priming in production @xcite can the model handle a range of linguistic phenomena and theoretical assumptions? ## a case study: evaluating competing theories of reduced relative clauses we use spawn to generate and test priming predictions for two competing syntactic theories of relative clauses that differ in their assumptions about how the structure of sentences like ( 9 ) is related to the structure of sentences like (10) and ( 11 ). ## results participant/model exclusion most participants (77%) never generated a single rrc target completion. similarly most models (median of 67% across the 18 model types) never generated a single rrc parse state. since the goal of this work is to find differences in the proportion of rrc parses between the primes, we only included in our analyses and plots the participants or models that generated at least one rrc completion or parse state. ## discussion in this work we introduced a cognitively motivated parser, spawn, which can be used to generate quantitative behavioral predictions from contemporary syntactic theories that are based on lexicalized grammar formalisms. spawn makes it possible to evaluate what theoretical differences (if any) result in differing sentence processing predictions. as a case study, we used spawn to generate predictions from two competing theories of reduced relative clauses (whiz-deletion and participial-phase) while modulating the reanalysis mechanism and the number of training examples. we compared the predictions from these different versions of the spawn model to human behavior from a large-scale (@xmath0. we found that the predictions of the whiz-deletion spawn models did not capture the qualitative human priming behavior for any of the model types. in contrast, many of the participial-phase spawn models captured the qualitative patterns, with the models that best captured the magnitude of the empirical effects being ones with weak prior knowledge, that reprocesses the sentence from the beginning whenever reanalysis is triggered. taken together, these results suggest that the participial-phase account of reduced relative clauses captures the structural representations people construct better than the whiz-deletion account. future work this work tentatively suggests that first-word regression might better model human processing than entropy-weighted regression. this observation needs to be more robustly validated with other empirical phenomena (e.g., priming in po/do sentences). additionally, some of the parsing mechanisms spawn implements, such as for reanalysis or predicting null elements, are likely too simplistic to account for human sentence processing more generally (see § g). future work can tweak these mechanisms and evaluate the modified models against processing benchmarks like the sap benchmark @xcite which have more fine-grained measurements (e.g., reading time per word) across a range of psycholinguistic phenomena. since the time taken for any of the parsing steps is measured in milliseconds by default in act-r, spawn can already generate quantitative predictions about the time taken to read or reprocess specific words in sentences, and therefore can be used with self-paced reading and eye-tracking datasets. finally, future work can also use this paradigm to evaluate other competing syntactic theories. conclusion we proposed a cognitively plausible parser that can be used to generate quantitative behavioral predictions from syntactic theories. using english reduced relative clauses as a case study, we demonstrated how this model can be used to adjudicate between competing syntactic theories and parsing mechanisms. a details about the two theories of reduced relative clauses and how they are implemented in the declarative memory a.2 differences in syntactic categories between the two theories category example sentence whiz-deletion participial-phase noun "the cat which was examined by... " np/cp np/cp ("cat") "the cat examined by ... " np/cp np/voicep "the cat being examined by ... " np/cp np/progp null wh subject "the cat null wh null pass examined by..." cp/(tp\dp) missing null finite auxiliary (tp\dp)/voicep missing null progressive auxiliary 'the cat null wh null prog being examined by..." (tp\dp)/progp missing when the noun in the whiz-deletion version combines with the null wh subject and null finite or progressive auxiliary, it results in the same parse state as the participial-phase noun categories for rrc and progrrc: np/voicep and np/progp. we also explored an alternative implementation of the whiz-deletion account where instead of having three null categories -null wh , null pass and null progwe had only two categories null whpass (cp/voicep) and null whprog (cp/progp). this implementation resulted in nearly identical results ( § 6) note, both accounts have the same categories for the null wh-word in rcs which modify objects of clauses like "the cat the doctor examined was skittish": null complementizer (object rcs) in the following table.