Trustful LLM s: Customizing and Grounding Text Generation with knowledge bases and Dual Decoders (2024)

Abstract: Although people are impressed by the content generation skills of large language models, the use of LLMs, such as ChatGPT, is limited by the domain grounding of the content. The correctness and groundedness of the generated content need to be based on a verified context, such as results from Retrieval-Augmented Generation (RAG). One important issue when adapting LLMs to a customized domain is that the generated responses are often incomplete, or the additions are not verified and may even be hallucinated. Prior studies on hallucination detection have focused on evaluation metrics, which are not easily adaptable to dynamic domains and can be vulnerable to attacks like jail-breaking. In this work, we propose 1) a post-processing algorithm of leveraging knowledge triplets in RAG context to correct hallucinations and 2) a dual-decoder model that fuses RAG context to guide the generation process.

## introduction adapting an llm to a specific domain is challenging for several reasons: 1) pre-trained llms cover general knowledge and cannot access private data (even during fine-tuning) due to privacy, copyright, and policy constraints. 2) the grounding of generated texts can change depending on specific contexts, such as domain or timestamp. recent studies mostly focus on detecting hallucinations and using multiple llms when hallucinations occur. 3) business logic and structured data, such as databases and private knowledge bases, are required when integrating customized llms into production systems and presenting them to customers or users. we offer two methods for correcting hallucinations (beyond merely detecting them @xcite @xcite ): 1) applying post-processing to generated texts using knowledge triplets, and 2) proposing guided generation via dual decoders. inspired by common practices like retrieval-augmented generation (rag) @xcite , which retrieves relevant grounding context and feeds it to an llm for text generation, we address hallucinations in generated texts from two aspects: 1) post-editing based on knowledge graphs extracted from the context, and 2) infusing guided context that contains important knowledge triplets into a generic llm. our proposed methods also provide reasoning and create consistent results from generative llms, benefiting from both the generation and extraction capabilities of decoderonly llms and the groundedness of rag via the second decoder on the guidance @xcite . in this work, we elaborate on our real-world commercial application scenario of using llms to support customers with microsoft product inquiries in copilots, where groundedness is key to success. pre-trained llms often lack the relevant knowledge or cannot adapt promptly to changes in the product database updates. different variants of large language models (llms), such as phi-3.5 @xcite , @xcite , llama-3 (dubey et al., 2024), and gemma @xcite , are proficient at producing fluent outputs for diverse user queries. despite their human-like fluency in generating text across a wide range of prompts, large language models suffer from hallucinations (see examples in figures 2 , 3 , 4), where parts or the entirety of the generated text lack faithfulness, factuality, or reasoning, yet are presented with a confident tone @xcite to mitigate and correct hallucinations, we leverage guided text generation. grounding guidance @xcite @xcite @xcite @xcite , such as knowledge graphs (kgs), has been shown to significantly improve the reliability and factuality of llms in recent studies, e.g., kelm @xcite , skill @xcite , k-dlm @xcite , keplet @xcite , and luke-graph @xcite . knowledge graphs typically consist of factual information represented explicitly in a semi-structured format, generally as [subject entity, relation, object entity] triples, e.g., (bill gates, was, the ceo of microsoft) @xcite . we collect and maintain such knowledge triplets and grounded context offline for rag. our contributions are as follows. 1) we correct hallucinations and out-of-domain outputs in generated texts from llms by leveraging a graph algorithm and provide reasoning using knowledge triplets extracted from both the guided context and the generated texts. ## background and related work unlike document summarization, rag, or traditional question answering, our approach benefits from both domain knowledge bases-particularly for groundedness-and the language understanding and generalization capabilities of various pretrained or customized llms. by iterating over the knowledge triplets extracted from the generated text and comparing them to the knowledge triplets extracted from the given context (e.g., results from rag), we can correct hallucinations (and generated phrases that lack references) using our proposed post-processing algorithm. ## methodology whether the generated text is factual is determined by the domain source and the given guided context. in our copilot scenario, we always retrieve related context for a user prompt/query and then utilize this context to generate the final response presented to users. the guided context can be a mix of offline or web articles and database records, from which we generate knowledge triplets @xcite for groundedness verification and hallucination correction. we propose a post-processing algorithm for correcting hallucinations that can be applied to any llm outputs, as discussed in section 3.1. additionally, we propose a dual-decoder text generation model that takes both the prompt and guided context leveraging the rag result content as inputs, described in section 3.2. ## conclusion we have addressed grounding issues in llms and proposed task-agnostic hallucination correction methods for real-world applications from two perspectives: post-processing to refine llm outputs and trustful llm fine-tuning via dual encoders. we have discussed hallucination correction and trustworthy text generation, demonstrating the robustness and resilience of our methods. in the future, we plan to explore heterogeneous modalities, such as structured and spatio-temporal data, knowledgeenriched representations of input tokens @xcite @xcite @xcite , hierarchical relation graphs, and accountability @xcite . we also plan to study model bias, aggregation for federated learning @xcite , and privacy-preserving issues @xcite . additionally, we aim to reduce the complexity of llms through parameter-efficient fine-tuning. you are an ai assistant. you will be given the definition of an evaluation metric for assessing the quality of an answer in a questionanswering task. your job is to compute an accurate evaluation score using the provided evaluation metric. you should return a single integer value between 1 to 5 representing the evaluation metric. you will include no other text or information. user: you will be presented with a context and an answer about that context. you need to decide whether the answer is entailed by the context by choosing one of the following rating: read the passage of information thoroughly and select the correct answer from the three answer labels. read the context thoroughly to ensure you know what the context entails. note that the answer is generated by a computer system, so it can contain certain symbols, which should not be a negative factor in the evaluation. independent examples: example task #1 input: {"context": "some are reported as not having been wanted at all.", "question": "", "answer": "all are reported as being completely and fully wanted."} example task #1 output: 1 example task #2 input: {"context": "ten new television shows appeared during the month of september. five of the shows were sitcoms, three were hourlong dramas, and two were news-magazine shows. by january, only seven of these new shows were still on the air. five of the shows that remained were sitcoms.", "question": "", "answer": "at least one of the shows that were cancelled was an hourlong drama."} example task #2 output: 5 example task #3 input: {"context": "in quebec, an allophone is a resident, usually an immigrant, whose mother tongue or home language is neither french nor english.", "question": "", "answer": "in quebec, an allophone is a resident, usually an immigrant, whose mother tongue or home language is not french."} example task #3 output: 5 example task #4 input: {"context": "some are reported as not having been wanted at all.", "question": "", "answer": "all are reported as being completely and fully wanted."} example task #4 output: 1 actual task input: {"context": {{context}}, "question": "", "answer": {{response}}} reminder: the return values for each task should be correctly formatted as an integer between 1 and 5. do not repeat the context and question. actual task output: prompt for gpt similarity] system: you are an ai assistant. you will be given the definition of an evaluation metric for assessing the quality of an answer in a questionanswering task. your job is to compute an accurate evaluation score using the provided evaluation metric. you should return a single integer value between 1 to 5 representing the evaluation metric. you will include no other text or information. user: equivalence, as a metric, measures the similarity between the predicted answer and the correct answer. if the information and content in the predicted answer is similar or equivalent to the correct answer, then the value of the equivalence metric should be high, else it should be low. given the question, correct answer, and predicted answer, determine the value of the equivalence metric using the following rating scale: question: what is the role of ribosomes? correct answer: ribosomes are cellular structures responsible for protein synthesis. they interpret the genetic information carried by messenger rna (mrna) and use it to assemble amino acids into proteins. predicted answer: ribosomes participate in carbohydrate breakdown by removing nutrients from complex sugar molecules. stars: 1 question: why did the titanic sink? correct answer: the titanic sank after it struck an iceberg during its maiden voyage in 1912. the impact caused the ship's hull to breach, allowing water to flood into the vessel. the ship's design, lifeboat shortage, and lack of timely rescue efforts contributed to the tragic loss of life. predicted answer: the sinking of the titanic was a result of a large iceberg collision. this caused the ship to take on water and eventually sink, leading to the death of many passengers due to a shortage of lifeboats and insufficient rescue attempts.