Plug-and-Play Adaptation for Continuously-updated QA (2022)

Abstract: Language models (LMs) have shown great potential as implicit knowledge bases (KBs). And for their practical use, knowledge in LMs need to be updated periodically. However, existing tasks to assess LMs’ efficacy as KBs do not adequately consider multiple large-scale updates. To this end, we first propose a novel task—Continuously-updated QA (CuQA)—in which multiple large-scale updates are made to LMs, and the performance is measured with respect to the success in adding and updating knowledge while retaining existing knowledge. We then present LMs with plug-in modules that effectively handle the updates. Experiments conducted on zsRE QA and NQ datasets show that our method outperforms existing approaches. We find that our method is 4x more effective in terms of updates/forgets ratio, compared to a fine-tuning baseline.

## introduction lm-as-kb is a new paradigm in which pre-trained language models (lms) are used as implicit knowledge bases (kbs) @xcite . this is made possible by lms' impressive ability to memorize factual knowledge @xcite . recently, two tasks have been used to assess such ability: lama, a knowledge probing benchmark, challenges lms to fill in masked words over relational knowledge @xcite ; and closed-book qa (cbqa) examines whether lms can correctly answer natural language questions @xcite . for practical usage, lm-as-kb requires that lms are updated periodically to stay current with the ever-evolving world. thus, lms' ability to update knowledge should also be evaluated. to this end, we present continuously-updated qa (cuqa), which tests the ability to continuously inject knowledge to update (or target knowledge), while retaining existing knowledge (or source knowledge). specifically, we consider multiple large-scale knowledge updates (8k to 60k) covering two scenarios: injecting new knowledge (scenario 1 in figure 1 ) and updating existing knowledge (scenario 2 in figure 1 ) . our goal is to organize the implicit storage of knowledge, to add target knowledge (yellow box in figure 1 ) and anchor to select target knowledge. a simple approach is to train updated lms from scratch; however, this is far too expensive considering the parameter sizes of recent lms, such as 175b for gpt-3 (brown et al., 2020) and about 11b for t5 @xcite . there has also been related work for the two scenarios. for scenario 1, a method for continual learning can be adopted, constraining the distance between parameters before and after fine-tuning @xcite . however, this approach still suffers from so-called catastrophic forgetting, where the lms fail to retain large amounts of source knowledge. for scenario 2, one may consider knowledge editing methods, where we see reasonable performances for a single knowledge edit while retaining the rest @xcite . however, this line of work does not perform well when multiple edits are accumulated, e.g., only 67% of 125 edits were updated, as reported in @xcite . we propose to efficiently extend lms with plug-and-play modules that store target knowledge. more specifically, we adopt a parameter-expansion method in which the lm storing existing knowledge is extended with plug-in feed-forward modules storing updated knowledge. depending on the input, the lm selectively uses either the original lm or a plug-in module. we stress that, by keeping the original lm intact, we retain (a) not only source knowledge, (b) but also those outdated from updates (red arrow in figure 1 ). (a) is important to avoid catastrophic forgetting, while (b) is useful when updates need to be reverted due to ethical concerns-for example, there can be malicious attempts to override facts. we evaluate our approach on zsre @xcite and natural questions @xcite to showcase successful updates of new knowledge and retention of existing knowledge. we measure the accuracies on both previous and updated knowledge and find that ours show x4 higher updates/forgets ratio, compared to fine-tuning. we also release our code and dataset. 1 our key contributions are as follows: • we present cuqa, a novel task to assess lms' ability to continuously inject knowledge to update. ## related work the relevant research can be categorized into three groups: knowledge editing, continual learning, and adaptation. in ## method in this section, we describe baseline approaches (section 4.1), and introduce our proposed method for plug-and-play adaptation (section 4.2). ## experiment in this section, we demonstrate the effectiveness of our approach on cuqa. datasets we evaluate our method on the following closed-book qa datasets: implementation for t5 model, we use a large version with total 770m parameters. in our experiment, we assume that the old model θ old storing source knowledge is available. for nq, we used the open-source pre-trained model 3 as the model θ old . for zsre, we load and train t5 model 4 on source knowledge. for training, we set batch size 64 on 4 rtx3090 gpus, and used adam (kingma and ba, 2015) optimizer with learning rate 4e-4. for development set, we sample each 1k from k s , k t , and select the maximum harmonic mean of their accuracies as a best model. as a hyper-parameter, we search δ in a range of [0,1] with 0.05 step size, and found the best value (δ=0.9) based on development set. as embedding memory m, we used additional parameters: 60m for zsre and 8.5m for nq. the size of the memories can be reduced by several techniques, such as random projection @xcite and binary encoding @xcite , which is left out of our focus. ## conclusion this paper studies how to accumulate new knowledge to lms that stores existing knowledge. we propose a simple yet effective method to update target knowledge into new parameters, preventing from forgetting source knowledge. on two datasets: zsre and nq, our empirical results show that our proposed method can improve existing approaches for continual learning or task adaptation.