What Can a Generative Language Model Answer About a Passage? (2021)

Abstract: Generative language models trained on large, diverse corpora can answer questions about a passage by generating the most likely continuation of the passage followed by a question/answer pair. However, accuracy rates vary depending on the type of question asked. In this paper we keep the passage fixed, and test with a wide variety of question types, exploring the strengths and weaknesses of the GPT-3 language model. We provide the passage and test questions as a challenge set for other language models.

## introduction generative language models produce likely text based on a context of other text. this process has a surprising number of useful applications, one of which is answering questions about a text passage. by training on text that contains (among other data) passages followed by questions and answers about the passage, and creating a context in which the answer to a question is the most likely continuation of the passage, better text prediction tends to result in better question answering. however, there are many types of questions that could be asked about a passage, from direct questions about facts, to explorations of its themes, to imagining what would happen if something about the story was completely different. depending on the type of question, the accuracy of answers can vary greatly. while some tests of the questionanswering ability of language models have been run, it is difficult to separate out whether a question is answered incorrectly because of changes to the question, or changes to the passage, or both. therefore, in this paper, we selected a single passage and asked a wide variety of questions about the passage. the response of a particular model to any one of these types of questions could be explored much new york times, 29 september 1973 a 61-year old furniture salesman was pushed down the shaft of a freight elevator yesterday in his downtown brooklyn store by two robbers while a third attempted to crush him with the elevator car because they were dissatisfied with the @xmath01,200 they had forced him to give them. we also found parameter settings that led the model to change major aspects of the story (this was generated using the stock davinci model with @xmath1.7): write a similar story that takes place in ancient egypt. a wealthy merchant was passing through the valley of the kings when a band of marauders descended on him. the merchant was carrying a large amount of gold and jewelry, so the marauders forced him to surrender it. but they wanted more, so they forced him to reveal where he had hidden his most valuable possessions. when he refused to tell them, they put him in a tomb and left him there to die. this is a fairly sophisticated analogy. instead of an elevator shaft, there is a tomb; instead of a furniture salesman with $1200 in brooklyn there is a merchant with gold and jewels in the valley of the kings; but the bones of the story are the same. one issue with the use of this article as part of a challenge set for language models is that the article may have been part of the training material for gpt, letting it memorize the answers, as discussed in @xcite three lines of evidence show this is unlikely: nevertheless, we can't completely rule it out. challenge datasets we have constructed a "challenge dataset" guided by approaches in the fields of linguistics and computational linguistics. in the former, datasets are developed for structured elicitation where linguists ask native speakers to judge grammaticality of presented sentences or to answer questions about their language. the elicitation results are key to deriving properties of the language @xcite . in the latter, datasets are constructed for system evaluation to establish benchmarks to gauge progress on shared tasks. in mt research, going beyond automated scores, the challenge set approach to system evaluation has provided a more fine-grained picture of the strengths of neural systems, as well as insight into which linguistic phenomena remain out of reach @xcite . in recent lm research that probes the syntactic knowledge in these models, the structured datasets of minimal grammatical/ungrammatical sentence pairs test when lms assign a higher probability to the grammatical sentence than the ungrammatical one @xcite . though significant caution is needed in interpreting lm results, we see diagnostic value in constructing challenge datasets to evaluate systems' passage understanding, as carefully constructed by dua et al. (2019). the fact that a current state-of-the-art lm dropped more than 50 absolute f1 points on their dataset reinforces our belief that challenge datasets can spur research into more comprehensive semantic analyses of what lms know, encouraging system hill-climbing up the right hill, per bender and koller (2020). the general language understanding evaluation (glue) benchmark @xcite is of particular relevance to this work, as it is a collection of resources for both training and evaluation of various types of natural language understanding tasks. it is intended to be agnostic to the system type. the evaluation suite includes tasks related to sentiment, paraphrase, natural language inference, coreference, as well as question-answering. while we took inspiration from this benchmark in our inclusion of questions overlapping with the inference tasks in glue, such as the winograd schema challenge, we contribute a set of challenge questions that have been carefully crafted and classified to probe the strengths and weaknesses of a lm with respect to particular knowledge sources, types, and reasoning requirements. lms and "prompt-based learning" numerous research teams are working to evaluate what lms may "know", even asking how they compare to knowledge bases @xcite . new possibilities are emerging with methods of estimating the knowledge in lms that can be found with automatically constructed prompts that yield better results than those manually created, demonstrating that any given prompt may be sub-optimal @xcite . given our focus on assessing what one family of prompt-based lms can answer about a given text passage, the most recent and pertinent overview that situates these lms in the field is the extensive, systematic survey of prompting methods and pre-trained language models using these methods by @xcite . as this survey notes, in tuning-free prompting as we have carried out with gpt lms, the questions in the q/a task generate the answers directly. this is efficient, yet also leaves the prompts as the only method to provide the task specification. for this reason, we present the framework of example questions that serve as test prompts in a structured challenge dataset both so that they can be reused by others in testing their systems, and so that we can extend these over time. ## conclusion this limited evidence suggests that current language models are able to supply answers to questions about a passage in a way that, at least superficially, meets the challenge of "natural language understanding" as originally set out in the 1970s. they can do surprisingly well at describing the likely thoughts and feelings of characters though these are not explicit in the passage. the models can also generate text that describes likely consequences for what might have happened if things had gone differently, and that fills in the factual information that we tend to omit when using natural language. however, many questions requiring even a little careful thought, reasoning, or multi-step inference go beyond the capability of these models. for those who want to make use of these models today, this suggests sticking to applications where the information needed to answer the questions are immediately available in the passage or slowlychanging, widely known information about the world; or else creative questions where there is no wrong answer. for researchers, though, this highlights the need to discover ways of combining the deductive reasoning capabilities already present in early ai work with the context-sensitivity and ability to work with natural language that these models provide.