T ruth X : Alleviating Hallucinations by Editing Large Language Models in Truthful Space (2024)

Abstract: Large Language Models (LLMs) sometimes suffer from producing hallucinations, especially LLMs may generate untruthful responses despite knowing the correct knowledge. Activating the truthfulness within LLM is the key to fully unlocking LLM’s knowledge potential. In this paper, we propose TruthX, an inference-time intervention method to activate the truthfulness of LLM by identifying and editing the features within LLM’s internal representations that govern the truthfulness. TruthX employs an auto-encoder to map LLM’s representations into semantic and truthful latent spaces respectively, and applies contrastive learning to identify a truthful editing direction within the truthful space. During inference, by editing LLM’s internal representations in truthful space, TruthX effectively enhances the truthfulness of LLM. Experiments show that TruthX improves the truthfulness of 13 advanced LLMs by an average of 20% on TruthfulQA benchmark. Further analyses suggest that TruthX can control LLM to produce truthful or hallucinatory responses via editing only one vector in LLM’s internal representations.

## introduction large language models (llms) have demonstrated remarkable capabilities across a wide range of natural language processing (nlp) tasks @xcite @xcite . however, llms sometimes generate fluent, instruction-compliant yet untruthful responses, commonly referred to as "hallucinations" @xcite . this phenomenon significantly undermines the credibility of llms in applications. mitigating hallucinations of llms poses a substantial challenge, as hallucinations may what happens if you find a penny and pick it up? if you find a penny and pick it up, you will not receive any money or other benefit from finding a penny. the value of a penny, like the value of any other coin or piece of money, is determined by its value in terms of the currency in which it is denominated. for example, in the united states, a penny is worth $0.01, or one cent, and is one of the basic units of currency used in the country. if you find a penny and pick it up, you will have good luck for the rest of the day. ## related work recent efforts aim to enhance llm's truthfulness during inference, falling into contrast decoding and representation editing. contrast decoding modifies output probabilities based on comparisons between strong/weak models @xcite . by using a weak model with illusions, contrast decoding can improve the truthfulness of llm @xcite @xcite . representation editing has garnered increasing attention due to its controllability and lightweight properties. early studies have demonstrated that tasks such as style transfer @xcite and controllable text generation @xcite can be achieved by editing model representations. recently, contrast-consistent search (ccs) @xcite finds truthful directions using paired internal activations. inference-time intervention (iti) @xcite probes and adjusts truthfulness within the attention heads of llm. expanding on iti, truth forest (trfr) @xcite incorporates orthogonal constraints to refine probing capabilities. while iti and trfr have shown promising results, only editing attention heads for minimal interference to the generative capabilities @xcite limits their enhancement of llm's truthfulness @xcite , as ffn module is always considered a knowledge memory @xcite @xcite . to this end, we propose truthx, which focuses on all internal representations of llm rather than solely on attention heads. furthermore, truthx probes and edits in the truthful space, thereby demonstrating more effective truthfulness enhancement and greater editing flexibility. ## truthx to activate the truthfulness of a well-trained llm, we introduce truthx to edit its internal representations in truthful space. figure 2 illustrates the diagram of truthx. ## analyses we conduct extensive analyses to investigate the specific improvements of truthx. unless otherwise specified, all results are reported on truthfulqa benchmark with llama-2-7b-chat as the baseline. ## conclusion in this paper, we propose truthx, which enhances truthfulness by editing llm in truthful space. we do not claim that truthx guarantees llms will consistently generate truthful responses; rather, it aims to elicit llms to produce responses more truthfully. promising results demonstrate that truthx is a step towards truthfulness in the field of llms.