Which questions should I answer? Salience Prediction of Inquisitive Questions (2024)

Abstract: Inquisitive questions — open-ended, curiosity-driven questions people ask as they read — are an integral part of discourse processing and comprehension. Recent work in NLP has taken advantage of question generation capabilities of LLMs to enhance a wide range of applications. But the space of inquisitive questions is vast: many questions can be evoked from a given context. So which of those should be prioritized to find answers? Linguistic theories, unfortunately, have not yet provided an answer to this question. This paper presents QSalience, a salience predictor of inquisitive questions. QSalience is instruction-tuned over our dataset of linguist-annotated salience scores of 1,766 (context, question) pairs. A question scores high on salience if answering it would greatly enhance the understanding of the text. We show that highly salient questions are empirically more likely to be answered in the same article, bridging potential questions with Questions Under Discussion. We further validate our findings by showing that answering salient questions is an indicator of summarization quality in news.

## introduction asking questions is the natural language manifestation of human inquisitiveness: we insist on getting answers for what we are curious about since childhood @xcite . acquired strategies of question generation have a profound impact on education @xcite . in linguistics, both theoretical and psycholinguistic work argued that readers generate inquisitive questions, seeking information in a conversation or as they read @xcite @xcite . in nlp, pre-trained models have enabled the *yating and ritika contributed equally. however, the space of possible inquisitive questions is vast. prior work @xcite showed that many distinct questions can be evoked from a given context, yet not all questions are equally good for an application. in theoretical linguistics, this also brings up a longstanding gap in understanding how discourse progresses @xcite : some of such inquisitive "potential questions" (as named in @xcite ) are likely more pertinent than others. some of these questions may be answered (by the writer) later in the article and thus become questions under discussion (quds) @xcite . evidence in psycholinguistics indicate that readers form expectations how a discourse would progress @xcite , providing a foundation for the predictability of quds @xcite . @xcite argues that a question is important if answering it provides high utility. however, there is so far no computational work to predict whether or not those questions should be answered or how salient they are. this work (figure 1 ) seeks to answer these questions by training a salience prediction model for inquisitive questions, using a new linguist-annotated corpus. in line with @xcite , a question scores high on salience if answering it would greatly enhance the understanding of the text. first, we collected validity and salience ratings of 1,766 inquisitive questions over english news articles [1] amid skepticism that russia's war in chechnya can be ended across a negotiating table, peace talks were set to resume wednesday in neighboring ingushetia. ## task definition a (human or machine) reader is reading a document, with established context c p (preceding context) consisting of sentences {1, ..., s k-1 }. the reader generates a potential question (section 2; onea (2016)) q evoked at sentence s k (also called the "anchor sentence" @xcite ). the salience of q is the measure of the extent to which it is important for a question q to be answered, in order to gain a fuller understanding of the situation described, after its invocation at sentence s k @xcite . specifically, for all valid questions, we define a likert scale of 1-5 (full definitions found in appendix a): a note on subjectivity. the salience values are, to some degree, subjective. however, prior work has shown compelling evidence that certain quds are more predictable than others @xcite and that linguistic cues in the text play a significant role in readers' expectation @xcite . under the assumption that there isn't too much divergence between the authors' intended audience and the background of the actual readers, our work sets out to capture such expectations through a question salience score. ## data collection we first present qsalience-data, a corpus of 1,766 inquisitive questions annotated with salience, plus natural language rationales for their judgments. although question generation has been used widely, application-independent datasets consisting of human-generated inquisitive questions are scarce. thus, we generate questions with llms, both to obtain a sizable amount of data and also to understand inquisitive question generation capabilities of llms. we supplement these questions with a smaller number of human-generated questions from prior work, which allows us to perform deeper analysis (section 5). ## salience vs. answerability a valid potential question evoked at s k can be deemed a qud anchored at s k if the subsequent discourse answers it. in order to study the relationship between potential questions and quds, we annotate a subset of the questions in qsaliencedata in terms of their answerability, i.e., the extent to which q is answered in the subsequent context c s . 3 we annotate answerability on a lik do salience and answerability correlate? are quds salient potential questions? ## salience prediction we experiment with a range of models for the prediction of question salience, given valid questions. our finding is that salience prediction is a discourse task that recovers implicit information not readily grasped by llms, while our best instruction-tuned model, qsalience, can achieve moderate agreement with humans. we further present question validity classifiers in appendix b, which can be used with qsalience in a pipeline fashion. adamw @xcite is used for optimization. hyperparameters can be found under