Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach (2024)

Abstract: A Large Language Model (LLM) tends to generate inconsistent and sometimes contradictory outputs when presented with a prompt that has equivalent semantics but is expressed differently from the original prompt. To achieve semantic consistency of an LLM, one of the key approaches is to finetune the model with prompt-output pairs with semantically equivalent meanings. Despite its effectiveness, a data-driven finetuning method incurs substantial computation costs in data preparation and model optimization. In this regime, an LLM is treated as a “black box”, restricting our ability to gain deeper insights into its internal mechanism. In this paper, we are motivated to enhance the semantic consistency of LLMs through a more interpretable method (i.e., model editing) to this end. We first identify the model components (i.e., attention heads) that have a key impact on the semantic consistency of an LLM. We subsequently inject biases into the output of these model components along the semantic-consistency activation direction. It is noteworthy that these modifications are cost-effective, without reliance on mass manipulations of the original model parameters. Through comprehensive experiments on the constructed NLU and open-source NLG datasets, our method demonstrates significant improvements in the semantic consistency and task performance of LLMs. Additionally, our method exhibits promising generalization capabilities by performing well on tasks beyond the primary tasks.

## introduction the field of natural language processing (nlp) is experiencing a paradigm shift with the advent of large language models (llms). these models have demonstrated remarkable capabilities in various tasks such as sentiment classification @xcite , machine translation @xcite 2023), and summarization @xcite . however, an llm tends to generate inconsistent and sometimes contradictory outputs when presented with a prompt that has equivalent semantics but is expressed differently from the original prompt. such behavior is referred to as the issue of "semantic consistency" @xcite @xcite , largely limiting the application of llms to real-world scenarios. for specific instances of inconsistency and consistency, please refer to figure 1 . ## methodology we use gpt-4 to construct prompt pairs that have the same semantics, and the target llm outputs of these prompt pairs should ideally be consistent. however, when we use the target llm to predict these prompt pairs, we obtain both consistent and inconsistent results. these inconsistent results are errors made by the target llm. to locate the sources of these errors, we assume that if the model components (i.e. attention heads) behave similarly to the target llm, then these components are actually the causes of the semantic consistency problems in the target llm. on the other hand, those components that have large behavioral differences from the target llm indicate that they are less relevant to the semantic consistency problems of the prompt pairs. based on this assumption, we use the linear probing technique @xcite to identify the relevant components. next, we add semantic consistency biases to the identified components to correct their erroneous behavior. these biases are obtained by calculating the difference between the mass mean of the consistency samples and the mass mean of all samples on the corresponding components. these biases will shift the original activations of these components toward more semantically consistent directions. as shown in figure 2 , our method mainly consists of three steps, which are consistent prompt pairs construction, key components locating, and consistent-aware editing respectively. we will provide detailed explanations of these steps in the following sections. ## nlu benchmark construction currently, there are some nlg benchmarks related to the semantic consistency of llm @xcite . however, there is a relative scarcity of nlu benchmarks specifically designed for semantic consistency research. to address this gap, we propose a benchmark dataset for evaluating semantic consistency in nlu tasks. this benchmark comprises robustsst2, robustmrpc, and robustboolq, which are derived from the sentiment classification dataset sst2 @xcite , the text similarity dataset mrpc @xcite , and the yes/no question-answering dataset boolq @xcite , respectively. our primary objective is to assess the semantic consistency of llms under synonymous task instructions for these datasets. more specifically, we first generate 30 synonymous task instructions for each task dataset. for example, we feed the following prompt (bold font) to gpt-4 to generate the synonymous task instructions used for robustsst2. rephrase the following sentence in 30 ways, while retaining the same meaning. measure the polarity of this sentence and respond with either 'positive' or 'negative', give me one word. then, we slice the generated task instructions according to an 8:2 ratio, meaning the training set uses 24 instructions, while the test set uses 6 instructions. for the training set, we constructed 24 synonymous prompts for each training instance, i.e., prompt @xmath0, instance train ] 24 @xmath1, and each prompt i has a label answer i , which is consistent across these 24 prompts. additionally, to create consistent prompt pairs for model editing, we generated c 2 24 prompt pairs by iterating through all possible combinations of these 24 prompts for each training instance. then, we utilized the target llm to assess whether the predictions generated for each prompt pair were consistent and obtain the relevant consistency evaluation label c. lastly, we sampled 250 instances from both the @xmath2, yielding a total of 500 instances used for model editing. the instance in the test set is different from the instance in the training set. each instance in the test set includes a constructed prompt along with its corresponding answer [prompt test , answer test ]. in specific, we first use the left 6 task instruction to construct relevant prompts i.e., {prompt @xmath3, instance test ]} 30 @xmath4. subsequently, we perform sample selection for these 6 prompts to construct a test instance. the selection rule is that if these 6 prompts yield the same result, we randomly choose 1 prompt to construct the sample. however, if they predict n different outcomes, we select 1 prompt from each of the n distinct results from n samples. by employing this approach, we can select hard negative samples while retaining examples that the llm could originally predict correctly, thereby enhancing the diversity of our test data. ## experiments 6.1 datasets to verify the effectiveness of the model editing method for addressing the issue of prompt semantic consistency, we conducted tests on both nlu and nlg tasks. for nlu evaluation, we utilized the specially constructed robustmrpc, robustsst2, and robustboolq datasets. for nlg evaluation, we selected the sport and capital categories from the popqa question-answering dataset as described by @xcite . detailed data statistics are shown in ## conclusion this paper presents the first analysis of the internal mechanism aspects of an llm that contribute to the problem of semantic inconsistency. we can precisely diagnose the key components that contribute to a model's semantic consistency. based on this finding, we propose a model editing method that directly injects biases into the model components of an llm without mass-manipulating model parameters. the proposed method can significantly improve both semantic consistency and the performance of llms on the constructed nlu and open-source nlg datasets. also, our methods exhibit promising generalization capabilities on four ood task datasets.