Knowledge Editing Induces Underconfidence in Language Models (2025)

Abstract: As language models continue to scale, the demand for knowledge editing, a retraining-free knowledge update method, has increased. However, since knowledge editing directly alters token prediction probabilities acquired during pretraining, the probabilities may diverge from the empirical distribution. In this study, we analyze the impact of knowledge editing to compare the alignment between token prediction probabilities and task accuracy by calculating confidence calibration before and after knowledge editing. Our results reveal that, for tasks requiring semantic understanding, the range of increase in token prediction probabilities tends to be smaller than that of accuracy improvement, suggesting that knowledge editing methods lead to less confidence in prediction.

## introduction large language models (llms) have been developed with increasing parameter size @xcite @xcite . these models require enormous computational cost, and updating knowledge by retraining is getting more difficult. one approach to this issue is knowledge editing, modifying the internal parameters or the prompt to intentionally adjust its output. it can easily introduce new knowledge to llms. however, challenges remain in knowledge editing, one of which is reliability @xcite . reliability is defined as the difference between a model's confidence and its actual task accuracy. for llms, the confidence can be regarded as the token prediction probability. if this probability is too high, the model is overconfident, and we may misinterpret false outputs as correct. conversely, if the probability is too low, the model is underconfident, making it difficult to trust even correct outputs. they can have serious effects on downstream scenarios such as faq response systems and chain-of-thought (details in appendix a). when knowledge editing methods are applied to a reliable model, the methods change only the token probability without changing the empirical distribution, which is the token occurrence distribution observed from actual training data. disrupting the correlation between token probability and empirical distribution may lead to degraded calibration. in this study, we analyze the impact of knowledge editing on model reliability through the lens of confidence calibration @xcite . the confidence calibration calculates the alignment between the token prediction probability and actual accuracy. we calculate confidence calibration to examine differences across three different types of knowledge editing methods, as well as the impact of reinforcement learning from human feedback (rlhf) @xcite . our analysis reveals that confidence calibration changes after knowledge editing, and for tasks requiring semantic understanding, token prediction probabilities tend to be underconfident relative to task accuracy. ## knowledge editing knowledge editing methods are classified into three categories: local modification-based methods, global optimization-based methods, and external memory-based methods @xcite . local modification-based methods, such as rome @xcite , memit @xcite , and kn @xcite , alter the output by locating and updating parameters that are highly related to the target knowledge. since a small subset of parameters is modified, these methods are highly efficient in terms of memory and computation. as for rome, the editing process involves two steps. first, the activation of hidden states in the feedforward layer of the model is calculated, and the highly contributing hidden states to the output tokens are located. second, the weights of the ff layer with high contribution, which are considered as the memory corresponding to the key/value pairs of knowledge, are modified to insert new key/value pairs. global optimization-based methods, such as mend @xcite and instructedit @xcite , aim to introduce new knowledge by updating all parameters in llms, enabling broader applicability to other edits without affecting unrelated knowledge. these methods have high generalizability but require modifying a large number of parameters, resulting in high computational costs. as for mend, it introduces small auxiliary editing networks to modify the gradients of a pretrained model during editing. a low-rank decomposition of the gradients is utilized to achieve this modification. external memory-based methods, such as ike @xcite and serac @xcite , store new knowledge in an external memory. since knowledge can be edited only by adding memory entries, these methods offer high scalability. these methods do not modify any internal parameters of models. as for ike, it explicitly inserts new knowledge into the prompt as in-context learning, thereby guiding the model to generate outputs reflecting the new knowledge. ## confidence calibration metrics. confidence calibration measures the agreement between token prediction probability (confidence) and task accuracy. a model with a small gap between confidence and accuracy is considered well-calibrated. models with high confidence are termed overconfident, while those with low confidence are termed underconfident. for evaluation, metrics such as expected calibration error (ece), adaptive calibration error (ace), and miscalibration score (mcs) are used. ## experimental settings figure 1 shows the overview of the experiments. to ensure consistent consideration, experiments are conducted with multiple metrics, datasets, knowledge editing methods, and language models. ## experimental results and discussions first, we focus on accuracy in §5.1. next, we examine ece and ace to capture whether models are well-calibrated or not in §5.2. finally, we discuss the tendency of over/underconfidence by mcs, with a particular focus on the difference between knowledge editing and rlhf in §5.3. ## conclusion to reveal the impact of knowledge editing on the reliability of llms, we analyze the alignment of token probability and task accuracy by calculating confidence calibration. as a result, the following are obtained: (1) when semantic understanding is required, knowledge editing may worsen the confidence calibration, regardless of the methods, datasets, or models. (2) knowledge editing tends to make models more underconfident. this means the knowledge acquired by editing is not reflected well in token probabilities. (3) contrary to knowledge editing, rlhf induces overconfidence. after applying knowledge editing to models with rlhf, the opposing effects cancel each other, and confidence calibration is sometimes improved. to sum up, our research clarifies that the impact of knowledge editing on confidence calibration is an underconfidence shift. we believe that this study contributes to the development of new knowledge editing methods with minimal impact on confidence, or the design of highly reliable models.