The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models (2024)

Abstract: Large language models often necessitate grounding on external knowledge to generate faithful and reliable answers. Yet even with the correct groundings in the reference, they can ignore them and rely on wrong groundings or their inherent biases to hallucinate when users, being largely unaware of the specifics of the stored information, pose questions that might not directly correlate with the retrieved groundings. In this work, we formulate this knowledge alignment problem and introduce MixAlign, a framework that interacts with both the human user and the knowledge base to obtain and integrate clarifications on how the user question relates to the stored information. MixAlign employs a language model to achieve automatic knowledge alignment and, if necessary, further enhances this alignment through human user clarifications. Experimental results highlight the crucial role of knowledge alignment in boosting model performance and mitigating hallucination, with improvements noted up to 22.2% and 27.1% respectively. We also demonstrate the effectiveness of MixAlign in improving knowledge alignment by producing high-quality, user-centered clarifications.

## introduction despite the recent advances of large language models (llms), they still struggle in unfamiliar scenarios not covered during pre-training @xcite . a common approach to mitigate this issue involves retrieving and incorporating supporting evidence from an external knowledge base @xcite . while the method indeed often improves the end-task performance, it still suffers from issues such as generating text that includes extraneous information not present in the figure 1 : knowledge misalignment. even if the user knows about the city constraint, he/she may not put it in the question being unaware that "city" is needed to filter noisy candidates. for the same reason, the user may not give a precise event name. due to the misalignment between the user question and the grounding knowledge, the llm fails to correlate the question with the correct grounding (underlined) and relies on its own biased knowledge (presuming new york as the intended city reference) to generate an incorrect answer. in this work, we argue that the primary cause of the error cases stems from the misalignment between human and grounding knowledge. this misalignment is quite common, as users are often unfamiliar with the information contained in the external database. when framing their questions, they might unintentionally phrase them in ways that either inconsistently state or even overlook the conditions and constraints of the retrieved groundings (refer to fig. 1 ). facing this, the language model may follow spurious correlations and incorporate biased model knowledge to generate biased, misleading, or unsupported content. to address this issue, we study the knowledge alignment problem considering various alignment types as depicted in section 2. unlike recent works on value alignment which aim to ensure llm generation follows human values, ethics, and goals @xcite , knowledge alignment seeks to bridge human and grounding knowledge for llm, thereby enhancing its ability to utilize grounding knowledge for faithful decision-making and issue resolution. towards solving the knowledge alignment problem, we propose mixalign, a framework that interacts with both the user and the knowledge base to acquire clarifications on how the user's question relates to the stored grounding knowledge. mixalign initiates the process with model-based knowledge alignment, where the llm is employed to map the conditions and constraints of the user question to corresponding ones within the knowledge base. in cases the mapping process yields uncertainties or the evidence remains unclear, it generates a question seeking further clarification from the user, a step we refer to as human-assisted knowledge alignment. the clarifications from these steps are incorporated to generate the final answer. in summary, our major contributions are: another strategy involves in-context learning using textual prompts that are helpful, honest, and harmless @xcite . the development of interpretability techniques to scrutinize the concepts learned by networks is yet another crucial approach, with the long-term aim of detecting and rectifying misaligned goals prior to deployment @xcite . this work can be seen as a special case of interpretability methods. unlike existing works that emphasize aligning human values with llm behavior, we aim to align human knowledge with external domain knowledge. this knowledge alignment enhances semantic and logical consistency between human expression and the stored evidence, thereby enabling llms to engage in more effective reasoning and problem-solving. ## the knowledge alignment problem we study how aligning human knowledge with grounding knowledge, represented by conditions and constraints found in the user question (q) and retrieved evidence (k), impacts the llm's ability to utilize evidence for answering questions. specifically, we address knowledge misalignments by acquiring clarifications (c), refining our prompt to: ## methodology in this section, we introduce mixalign, a method designed to enhance knowledge alignment in grounded generation. mixalign utilizes the llm to align user expressions with the grounding knowledge and, if necessary, further enhances this alignment through human clarification. fig. 3 depicts how mixalign addresses the knowledge format of tabular databases (as detailed in section 3). for a discussion on its adaptability to other knowledge forms, please refer to appendix c. ## conclusion and future work in this work, we introduce the knowledge alignment problem, which addresses mismatches between constraints present in user questions and the knowledge groundings referred to by llms, and we propose the mixalign framework to bridge this gap by generating clarifications for any identified misalignments. experimental results highlight the crucial role of knowledge alignment in improving model performance and faithfulness and demonstrate the efficacy of mixalign in generating high-quality clarifications. future studies could explore adapting mixalign to various knowledge modalities, thereby broadening its applicability.