The Curious Case of Control (2022)

Abstract: Children acquiring English make systematic errors on subject control sentences even after they have reached near-adult competence (Chomsky, 1969), possibly due to heuristics based on semantic roles (Maratsos, 1974).Given the advanced fluency of large generative language models, we ask whether model outputs are consistent with these heuristics, and to what degree different models are consistent with each other. We find that models can be categorized by behavior into three separate groups, with broad differences between the groups. The outputs of models in the largest group are consistent with positional heuristics that succeed on subject control but fail on object control. This result is surprising, given that object control is orders of magnitude more frequent in the text data used to train such models. We examine to what degree the models are sensitive to prompting with agent-patient information, finding that raising the salience of agent and patient relations results in significant changes in the outputs of most models. Based on this observation, we leverage an existing dataset of semantic proto-role annotations (White et al. 2020) to explore the connections between control and labeling event participants with properties typically associated with agents and patients.

## introduction normally-developing children learning english struggle with subject control clauses long after they have successfully acquired the components to understand them @xcite @xcite . a sentence with a subject control clause has a matrix (main) clause containing a main verb, an agent, and a patient, and an embedded infinitival clause. for example, in cole promised joe to call, the agent is cole, the patient is joe, and the embedded clause is to call. crucially, the embedded verb here does not have an overt subject, but rather implicitly refers to a subject in the matrix clause (in this case, cole). in a subject control clause, this latent subject of the embedded infinitival clause (usually written as pro) is coindexed with the subject (cole) rather than the object (joe) of the matrix (main) clause, i.e. cole (the agent) is doing the calling. this is typically written: recently, large pre-trained language models have shown an impressive ability not only to produce fluent text, but also to perform tasks by "filling you will be given a context and a question. answer the question with either " " or " ".\n context: told to come.\n question: who came, or ?\n answer: avery casey ## models and metrics we explore both autoregressive models and textto-text models. autoregressive models are optimized by minimizinglog(p (w i |w -i )) for words w 1 , . . . w n in a given context. these models (also called "decoder-only" models) are composed of just a decoder, which encodes the previously observed tokens w 1 , . . . w i-1 produces a probability distribution over the vocabulary for the next token, w i . text-to-text models are encoder-decoder models, optimized to reconstruct a noised version of the input via the decoder. an encoder takes a corrupted version of whole sequence w 1 , . . . w n as input, encoding it into a dense representation from which the decoder reconstructs the original w 1 , . . . w n . the autoregressive models considered are: • gpt-3 davinci: this model is only available through the openai api, and its exact training details are unclear. it is based on the gpt-3 model @xcite which was trained on common crawl @xcite with 175b (billion) parameters. among the several versions of gpt-3, davinci is generally regarded as the highest-performing (openai). the training data is based on common crawl, though similarly to gpt-3 davinci, the details of the training data filtering process are unclear. relevant differences to gpt-3 are in the tokenization (which includes multi-word expressions) and use of fewer, wider layers. the text-to-text models we consider are: • t5 for qa: the t5-base text-to-text model (220million parameters) @xcite ) was pretrained on cleaned common crawl data (c4) and fine-tuned on squad question answering data @xcite . ## experiment 1 in order to examine what types of generalizations are made by the examined models when prompted for subject and object control information, we construct a number of question-answering-style prompts, where the models fill in the answer. this approach follows recent literature @xcite and takes advantage of the models question-answering abilities. moreover, using models pretrained with a language-modeling loss rather than training a model specifically for control lets us examine what types of generalizations are captured by the models' associations learned from its original training data, rather than whether a very large model can learn to answer subject and object control questions correctly. we first describe the construction of the prompts used in this set of experiments. using those prompts, we validate the choice to use heuristic extraction from open generation (i.e. allowing the model to generate tokens up to an end-of-sequence token) rather than logits and confirm that in the c4 dataset, object control is more frequent than subject control. then, we analyze the zero-shot performance of the models on the subject and object control prompts. ## experiment 2 recent work has shown that providing examples in the prompt to a frozen pre-trained model can yield higher performance on qa tasks @xcite . in this setting, typically called "in context learning", some number of demonstrations of questions and answers are given in the context, followed by a test example, to which the model produces an answer. the demonstrations in the context give the model additional guidance on which task is being evaluated. while we do not do in-context learning with training questions about object in subject control, we do experiment with adding information to the prompt to raise the salience of agents and patients (e.g. "q: who told someone to call? a: cole" for (2).) an example can be seen in fig. 2 ; the qa pairs given in the context give the agent and the patient of the matrix clause, but not the embedded clause. this can be thought of as a form of chain-of-thought prompting @xcite or a scratchpad @xcite where the gold answers are provided. we hypothesize augmenting the prompt with agent-patient questions will affect each group as follows: 1. for group 1 (gpt-neo, jurassic jumbo) where the models' outputs are consistent with positional heuristics, the additional prompts will provide some evidence inconsistent with the heuristic. this evidence may lead to changes in the models' associations that result in outputs less consistent with the heuristics. for example, in fig. 2 the question who was told to come, avery or casey? the answer casey provides evidence against taking the furthest-away np as the answer. in that case, we would expect a drop in performance in passive object control and in subject control, where the heuristic is beneficial, and an increase in object control, where the heuristic does not help. 2. in group 2 (t0 and t5), since the model outputs are already consistent with a semantic role-based explanation, we do not expect much change in any setting. in other words, the model outputs are already consistent with access to agent-patient relations combined with an incorrect association for outputting the matrix clause's agent as the embedded subject. 3. finally, in group 3 (gpt-3, jurassic large), we see that the models' outputs on object control are consistent with an object control-specific heuristic (to take the first np) the models have lower performance for passive object control than active object control. thus, as in group 1, we expect that evidence against the positional heuristic in the prompts will boost performance in passive object control, while reducing it on active object control. ## experiment 3 following maratsos (1974)'s hypothesis that the observed mistakes children make on subject control sentences is driven by semantic roles, in section 5.2 we examine the relationship between a model's ability to perform zero-shot object and subject control and its accuracy on identifying attributes commonly associated with agents and patients. querying language models using fixed semantic role ontologies (e.g. agent, patient, theme) may be difficult as these ontologies may be absent from pretraining corpora. we instead measure the models ability to perform semantic proto-roles labeling (sprl) for the volition and change of state properties. we use the sprl data provided in the uni-versal decompositional semantics (uds) dataset introduced by @xcite . these properties, first proposed by @xcite , were found to be strongly prototypical of agents and patients, respectively @xcite . 6 proto-role inferences are elicited with simple prompts, circumventing brittle and complicated ontologies. indeed, the uds dataset was built by asking annotators questions like "how likely is it that arg chose to be involved in the pred?" and normalizing their scalar ratings to [-3, 3]. ## related work following the advent of pretrained language models there has been an explosion of work examining what kinds of linguistic knowledge such models contain. @xcite provide a comprehensive survey of probing work on bert, covering probing for syntactic, semantic, and world knowledge. this line of probing work generally makes use of linear classifiers on top of frozen representations, tuned on a training set @xcite @xcite . in contrast, we follow more recent work in probing large generative models using cloze-style prompts and relatively open generation @xcite . such models (generative and nongenerative) have been probed for diverse knowledge, including syntax @xcite , symbolic reasoning @xcite , and commonsense knowledge @xcite @xcite . this has often been done by recasting benchmark datasets into text, either with zero examples @xcite or in the form of in-context learning @xcite . ettinger (2020) present a suite of comparisons between pretrained language models and psycholinguistic experiments. in a similar vein to our work, @xcite examine gpt-2's performance on reflexive anaphor agreement in subject and object control clauses, finding that gpt-2 performs well on object control but not transitive subject control; we do not examine reflexive anaphora, and expand our analysis to multiple model classes. language models can be sensitive to the format of a prompt -in order to improve extraction of relational knowledge from large language models, @xcite propose automatic methods for mining new prompts and paraphrasing existing prompts. similarly, @xcite propose a method for gradient-based prompt optimization, and @xcite propose a gradient-based search for prompt token replacement. as our experiments require a specific prompt syntax, we choose to instead run prompts across different instruction styles and name-verb combinations. large language models are also sensitive to the frequency of terms in their training data; @xcite show a strong correlation between frequency of a term in a corpus and performance on tasks requiring that term. this further hightlights how surprising it is that several models perform better on subject control than object control. ## conclusion the results in fig. 4 indicate that differences between models are not merely of degree, but of kind, with groups of models following different patterns, many of which are inconsistent with the dominance of object control in english. this highlights the pressing need for transparency in the reporting of model details, and especially of training data, without which it is impossible to hypothesize why these differences are observed. we also find that, despite there being no trainable parameters in the few-shot setup of fig. 5 , the models tested are in many cases predictably sensitive to semantic role information. our results in section 5 suggest that some models appear able to perform semantic proto-role labeling for volition and change of state (