Information Association for Language Model Updating by Mitigating LM -Logical Discrepancy (2024)

Abstract: Large Language Models (LLMs) struggle with providing current information due to the outdated pre-training data. Existing methods for updating LLMs, such as knowledge editing and continual fine-tuning, have significant drawbacks in generalizability of new information and the requirements on structured updating corpus. We identify the core challenge behind these drawbacks: the LM-logical discrepancy featuring the difference between language modeling probabilities and logical probabilities. To evaluate and address the core challenge, we propose a new task formulation of the information updating task that only requires the provision of an unstructured updating corpus and evaluates the performance of information updating on the generalizability to question-answer pairs pertaining to the updating information.We further propose a novel and effective pipeline approach for the task, highlighting a self-prompting-based question-answer generation process and a associative distillation methods to bridge the LM-logical discrepancy.We develop two datasets for evaluation, one sourced from news articles published in March and April 2023, and the other from the Natural Questions benchmark.Experimental results demonstrate the superiority of our approach, significantly increasing the factual consistency score (on a scale from 0 to 1) by up to 0.16. Furthermore, our method effectively mitigates forgetting utilizing a compact replay buffer with only 2.3% of the training tokens.

## introduction large language models (llms) have demonstrated remarkable capabilities in addressing diverse information needs, primarily owing to the extensive range of information sources in their pre-training corpora. nevertheless, llms are incapable of providing up-to-date information absent from the pretraining corpora. therefore, effectively updating 1 the latest available news by the time of dataset collection new information: louisville metro police department officer nickolas wilt is in critical condition after undergoing brain surgery following a shootout in a bank... q: what is the current state of officer wilt? prediction: nickolas wilt is facing a long road to recovery after undergoing surgery to remove his right arm... generalizability of new information refers to the ability to associate the information to relevant context. we provide an example in structurality of updating corpus is another signif-icant limitation of existing research on knowledge editing, which concentrates on structured information such as knowledge triples or question-answer pairs on triples. structured updating corpus requires substantial human efforts to generate which limits the efficiency of information updating. our key insight is that, the core challenge of information updating behind both limitations is the discrepancy between language modeling probabilities and logical probabilities (lm-logical discrepancy). to illustrate this discrepancy, consider two token sequences x and y , @xmath0. the language modeling probability p (y |x) measures the probability of y following x in natural language. on the other hand, if we consider x, y as random variables of the occurrences of corresponding events denoted by x e , y e , the logical probability p (y e |x e ) measures the probability of y happening when x happens. we can see that p (y e |x e ) = 1, yet p (y |x) can be small since these two sentences contain redundant information and rarely co-occur as neighboring sentences. to ground this discrepancy to generalizability, existing methods aim at increasing the language model probability of new information, which naturally exhibits a low magnitude of associations: p (x|y ) can be small even for strongly related sentences. the lack of associations limits the generalization of the updating information to relevant information. this discrepancy also explains the requirements on structurality. the usage of structured information assumes that language model probabilities of structured prompts, such as p (new york|where is tom from?), is closer to the logical probability p (x e ) compared with unstructured language model probability p (x). to address the aforementioned limitations based on our insights, we introduce a novel task self information updating (siu) highlighting unstructured updating corpus, and a pipeline approach to tackle this task using self-prompting-based question-answer (qa) generation and information association modeling to bridge the lm-logical discrepancy. the formulation of siu is illustrated in figure 1 . the llm updates itself given only unstructured information sources such as news articles. we also include a replay corpus on past information to mitigate forgetting. for evaluation of generalizability, we propose to use qa pairs querying either the updating information or the past information, created by human or @xcite . we adopt the factual consistency score @xcite to emphasize information acquisition instead of preference alignment. for the pipeline approach illustrated in figure 2 , we use a self-prompting process to generate question-answer (qa) pairs relevant to the updating information by llms themselves, which augments the updating corpus for fine-tuning. an example of such pair is provided in for experiments, we utilize an instructionfinetuned model from llama-7b as the base model. we curate a corpus of news articles published after march 2023 as the updating corpus. we also developed another corpus based on natural questions @xcite we evaluate the factual consistency score (on a scale from 0 to 1) of the responses and observe a significant improvement of 0.16 over baselines that are prone to the exposure bias. additionally, we study the forgetting problem under a continual learning setting and discover that our approach maintains good performance on past information using a replay corpus containing only 2.3% of the past training data. to summarize, our major contributions include: llm updated llm updating corpus (new information) replay corpus (past information) continual fine-tuning evaluation qas (new information) evaluation qas (past information) human / gpt-4 human / gpt-4 qa-based evaluation corpus evaluation information updating corpus figure 1: illustration of the formulated information updating task. llm updated llm updating corpus (new information) replay qas (past information) updating qas (new information) self prompting combined updating corpus continual fine-tuning self data preparation evaluation qas (new information) evaluation qas (past information) human / gpt-4 human / gpt-4 qa-based evaluation corpus evaluation logical discrepancy. experimental results demonstrate the effectiveness of our approach. ## task formulation we introduce the mathematical definition of self information updating and an instantitation of the task based on the definition. ## approach we present our pipeline approach in figure 2 . we highlight two important components to address the lm-logical discrepancy: self prompting and as-sociative distillation. we first introduce the self prompting. we then discuss the exposure bias problem, a side-effect of the discrepancy that can be mitigated by the proposed associative distillation. ## conclusions and future work in this paper, we identify the core challenge of lm-logical discrepancy for information updating behind the limitations of exisiting research on generalizability and structurality. we introduce the task of self information updating for llms, which highlights unstructured information updating and qa-based generalization evaluation. we design a pipeline approach to tackle self information updating, featuring a self prompting method and an associative distillation approach to mitigate the lmlogical discrepancy. the associative distillation is proposed to solve the exposure bias problem which prioritizes past information originating from the discrepancy. our proposed method significantly improves factual consistency. additionally, we study the forgetting phenomenon under the continual learning setting and find that our proposed method can maintain past knowledge by keeping a small portion of the past data. we envision three extensions for this work: