Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering (2024)

Abstract: Questions posed by information-seeking users often contain implicit false or potentially harmful assumptions. In a high-risk domain such as maternal and infant health, a question-answering system must recognize these pragmatic constraints and go beyond simply answering user questions, examining them in context to respond helpfully. To achieve this, we study assumptions and implications, or pragmatic inferences, made when mothers ask questions about pregnancy and infant care by collecting a dataset of 2,727 inferences from 500 questions across three diverse sources. We study how health experts naturally address these inferences when writing answers, and illustrate that informing existing QA pipelines with pragmatic inferences produces responses that are more complete, mitigating the propagation of harmful beliefs.

## introduction humans have varying information needs when they ask questions @xcite . sometimes these needs are easily inferred from the surface form, such as in factoid questions (e.g. "who is the 44th president of the united states"). however, in a question such as "is there a good non-dairy baby milk i can supplement for my newborn?", addressing the underlying false assumption "newborns can safely drink non-dairy milk" becomes part of satisfying the unexpressed information need. complete answers to these types of questions must not only address the surface question itself, but also "question the question", critically examining its pragmatic needs. these needs become magnified in sensitive domains, such as consumer health or the legal domain. in these settings, addressing pragmatic needs of questions involves proactively addressing false assumptions or implications in questions to ensure * equal contribution. ## collecting assumptions and implications in the wild in contrast with factoid qa, systems deployed in sensitive domains such as consumer health must proactively mitigate harm. in these settings, correcting false assumptions is not optional: systems must provide contextual answers that balance information completeness with brevity. access to high quality healthcare in the united states vastly differs across socioeconomic backgrounds @xcite . such users are often likely to turn to accessible internet resources and-as of late-general purpose chatbots @xcite . this motivates us to focus on maternal and infant care, a challenging area of consumer health where patients are concerned with both their own physical health as well as the health of their child. to effectively study and induce pragmatic behavior in qa systems, the evaluation questions we choose must reflect real-world experiences and situations for which there may not be a straightforward answer explicitly addressed in a single web document. for example, answers to natural questions @xcite -a popular open-domain question-answering dataset-can be found directly in short extracted text snippets from wikipedia ( we carefully construct a dataset of questions from three distributionally distinct sources: a domain-specific qa system we design and deploy to pregnant and postpartum participants we recruit @xcite , reddit, and nq. then, we introduce an annotation scheme to elicit assumptions and implications from these questions, validate their plausibility, and finally collect supporting evidence to determine their veracity. our final dataset contains 2,727 assumptions from 500 evaluation questions ( ## grounding assumptions and implications in linguistic theory assumptions and implications in our dataset map to two well-studied phenomena in linguistic pragmatics: presupposition and implicature @xcite . we begin with a short primer of both types of pragmatic inference ( § 3.1) and then discuss the implications of both types in a qa setting ( § 3.2). ## conclusion we show that it is possible to induce pragmatic behavior in qa systems to correct latent false assumptions in the sensitive domain of maternal and infant health. next-generation qa systems deployed in real-world settings must learn to address the pragmatics of user questions. though we have shown the viability of explicitly inducing pragmatic behavior in models in this work, directions for future work include training retrievers to inherently search for evidence to address pragmatic inferences and readers to reason on top of such evidence to tactfully and effectively challenge user beliefs.