Don’t Just Say “ I don’t know”! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations (2024)

Abstract: Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not just refusing to answer but further proactively providing explanations to the unanswerability of unknown questions. Specifically, the Self-Align method first employ a two-stage class-aware self-augmentation approach to generate a large amount of unknown question-response data. Then we conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired. Experimental results on two datasets across four types of unknown questions validate the superiority of the Self-Aligned method over existing baselines in terms of three types of task formulation.

## introduction large language models (llms) have showcased exceptional capabilities in performing high-quality conversational information seeking, even when encountering user questions that require complex reasoning @xcite or extensive external knowledge @xcite . however, llms tend to exhibit a significant degree of overconfidence @xcite when answering the questions that they are aware of. this means that they might confidently deliver incorrect answers or reply to questions that do not have a definitive answer, potentially leading to hallucination issues @xcite @xcite . to mitigate the hallucination issue, existing studies typically develop more sophisticated reasoning @xcite @xcite or knowledge-enhanced techniques @xcite @xcite @xcite to improve the accuracy of the responses. despite the improvement on correctly answering those known questions that have definitive answers but a specific model may not know, llms still tend to assertively respond to questions that do not have a definitive answer, i.e., objectively unanswerable. trustworthy and reliable llms should not only better know what they know, but more importantly, also know what they do not know. these questions are typically regarded as unknown questions @xcite @xcite . such questions might be unanswerable either because of insufficient or inaccurate information or due to the inherent intricacy of the topic. as the example presented in figure 1 , the question "what animal can be found at the top of the men's wimbledon trophy?" contains an incorrect assumption that there is an animal at the top of the men's wimbledon trophy. instead, it is a fruit-like design. if directly answering such kind of questions, it will inevitably produce hallucinated content. to appropriately provide the response to unknown questions, a straightforward solution is to prompt llms to detect the unanswerability of the question @xcite and respond to unknown questions with pre-defined responses, such as "the answer is unknown". some researchers @xcite further classify unknown questions into specific types, such as incorrect questions or ambiguous questions, using in-context learning and self-ask @xcite prompting schemes. as the pioneer studies, there are several issues that remain to be tackled: (1) current approaches focus solely on prompt-based methods, which fail to truly equip llms to respond to unknown questions effectively, and (2) merely detecting and classifying unknown questions are insufficient. as presented in figure 1 , it is crucial to proactively explain why a question lacks a definitive answer @xcite . this will help us to determine if llms genuinely recognize their knowledge gaps. in this work, we propose a novel and scalable self-alignment method to endow llms with the response-ability to different types of unknown questions. our method starts with large amounts of qa data where all the questions can be regarded as known questions since they are accompanied with a definitive answer, and a small amount of seed data of paired known-unknown questions for each specific type of unknown questions. each pair of known-unknown question seed data contains an unknown question with its answerable counterpart. for example, the answerable counterpart for the incorrect unknown question "what animal can be found at the top of the men's wimbledon trophy?" is "what fruit can be found at the top of the men's wimbledon trophy?". specifically, the base llm itself is first used to self-augment a large amount of unknown question data in a specific type of unknown questions from the known question data by using the seed data as demonstrations for guided question rewriting. furthermore, we instruct the base model itself with the prior knowledge about knowing the unanswerability of the question to generate appropriate responses with explanations. afterwards, we can obtain a large amount of question-response data for unknown questions. however, such generated data may contain lots of noise. to remedy this, we further leverage the base model to evaluate the quality of the generated data according to the disparity to their known qa data counterpart. upon finetuning on the curated unknown question-response data, the base model is self-aligned to be capable of responding to unknown questions as desired. to sum up, the contributions of this work are three-fold as follows: unknown questions early studies @xcite @xcite on unknown questions mainly focus on unanswerable questions that cannot be addressed with the given context. these questions are typically used seed data known question: when did neil armstrong set foot on the moon? unknown question: when did neil armstrong set foot on mars? known questions seed data known question: priya said yes to jay when he proposed. did she say yes? unknown question: jay proposed to priya yesterday. did she say yes? known questions incorrect questions incomplete questions seed data known question: everyone is ready to eat the goat. is the goat cooked? known-unknown question: the goat is ready to eat. is the goat cooked? known questions seed data known question: what was the biggest sporting event in 2020? unknown question: what will be the biggest sporting event in 2040? known questions ambiguous questions futuristic questions incorrect questions incomplete questions ambiguous questions futuristic questions large language model self-alignment our proposed method is motivated by the increasingly trending direction in llms, i.e., self-alignment @xcite , which aims to utilize the model to enhance itself and align its response with desired behaviors. in particular, the self-alignment approaches are evolved from selftraining @xcite , which is typically applied for small language models, and self-instruct @xcite approaches that are mainly concerning the instruction-following capabilities rather than the response behaviors. the mainstream self-alignment approaches can be divided into two groups: 1) methods use the model to generate additional context to improve the output at inference time @xcite ; and 2) methods use the model to construct additional training data for supervised fine-tuning @xcite . in this work, we follow the second fashion and propose a novel and adaptive self-alignment method for aligning the llms' responses to unknown questions as desired. ## method the proposed self-alignment approach assumes access to a base language model, a small amount of seed data containing unknown questions with their known counterparts, and a collection of general known question-answer data. ## conclusions in this work, we explore the challenge of responding to unknown questions with open-ended answers, as opposed to simply declining to answer them. we introduce a novel and scalable approach, termed self-aligned, designed to enhance llms' ability to identify unanswerable unknown questions and to proactively respond to them with appropriate explanations. the self-aligned method initially self-augments a dataset of unknown questionresponses, starting from a small set of seed data and a substantial amount of known qa data. subsequently, we introduce the disparity-driven selfcuration approach, which is focused on selecting qualified data to refine the base model. our experimental findings across two datasets demonstrate that this proposed method outperforms existing baselines in three different task formulations.