Modifying Memories in Transformer Models

1 INTRODUCTION . Large-scale Transformer based language models ( Vaswani et al. , 2017 ; Devlin et al. , 2018 ; Radford et al. , 2019 ; Raffel et al. , 2019 ; Brown et al. , 2020 ) have not only pushed state-of-the-art on standard natural language processing ( NLP ) benchmarks such as GLUE and SQuAD , but they have also been crucial for improving various real-world systems ( see , e.g. , Nayak , 2019 ; Rao et al. , 2019 ) . Given that these models are pretrained on a large corpora of text such as Wikipedia and BookCorpus ( Zhu et al. , 2015 ) , it ’ s quite conceivable that they are able to implicitly memorize the factual knowledge in their large number of parameters . Recent works ( Petroni et al. , 2019 ; Roberts et al. , 2020 ) have verified this hypothesis by evaluating the pretrained language models on factual knowledge based tasks . This line of work shows that pretrained large Transformer based language models achieve non-trivial performance on various open-domain question answering ( QA ) tasks that probe the factual knowledge stored in the model parameters . The aforementioned memorization capability of Transformers opens up many exciting opportunities . In addition to improving generalization with better language understanding , Transformers may also replace or assist traditional knowledge bases ( KBs ) that are either manually curated or require significant amount of supervision ( Roth & Yih , 2002 ; Kambhatla , 2004 ; Surdeanu & Ji , 2014 ) . Different from conventional KBs that explicitly memorize factual knowledge , Transformers implicitly memorize knowledge in their model parameters . As a result , Transformers lack one key advantage of the conventional databases : efficiently modifying the factual knowledge stored in the model . Unlike Transformers , in conventional databases such as SQL and NoSQL that explicitly store knowledge in the forms of structured tables , key-value pairs , wide columns , graphs , or documents , updating knowledge is straightforward . Knowledge-augmented Transformers , which leverage factual knowledge bases to improve their feature representations , can not effectively modify their predictions by only updating the symbolic knowledge as it causes conflict with the implicit memorization in their parameters ( Verga et al. , 2020 ) . This raises the natural question : Can Transformers cope with the ever-changing world where knowledge is continuously being added , updated , and deprecated ? To answer this question , we propose a new task of explicitly modifying specific factual knowledge in Transformer models while ensuring that model performance does not degrade on the unaltered facts . This task is useful in many scenarios . For example , the factual knowledge stored by the model can become stale over time , which needs to be updated periodically , e.g. , a sports player may play with different teams over time . Users may ask a Transformer-based assistant model to update certain knowledge ( factual or otherwise ) that they asked model to memorized in the past , e.g. , their favorite tourist destination . In the context of privacy one may need to overwrite unintendedly memorized sensitive information without retraining the model ( Carlini et al. , 2019 ) . Furthermore , language models are susceptible to various biases present in the large corpora of text used for their training , and such biases may need to be eliminated to ensure a fair application of such models in real-world ( Bolukbasi et al. , 2016 ; Bordia & Bowman , 2019 ; Blodgett et al. , 2020 ) . To the best of our knowledge , this is the first work studying reliable and efficient modification of the factual knowledge memorized by Transformers . The paper makes the following contributions . • We create a new benchmark to evaluate the ability of a candidate method to modify the factual knowledge of a Transformer model as desired while preserving the model ’ s performance on the unmodified factual knowledge ( § 3.1 ) . • We formulate the knowledge modification as a constrained optimization problem with a constraint on the loss on the unmodified facts and explore better baseline methods to approximately enforce this constraint ( § 3.3 ) . • We show that constrained layer-wise fine-tuning is a simple yet effective way to modify the knowledge memorized by Transformers ( § 4 ) . • We find that it is not necessarily easier to modify factual knowledge in the models that employ explicit memory modules , e.g. , FaE ( Verga et al. , 2020 ) , as compared to those Transformer models that solely rely on implicit memorization . 2 RELATED WORKS . Traditionally , KBs are commonly utilized to store and access the relational knowledge in NLP domain ( Ji et al. , 2020 ; Zelle & Mooney , 1996 ; Zettlemoyer & Collins , 2005 , inter alia ) . However , the recent success of Transformer-based language models on a multitude of NLP tasks has fueled an increasing number of efforts on exploring the ability of these language models to serve as unstructured/non-symbolic KBs . Language models as a source of factual knowledge . To assess the performance of off-the-self modern language models as KBs , Petroni et al . ( 2019 ) introduced LAMA ( LAnguage Model Analysis ) probing method that convert various facts and fact-seeking question-answer pairs into cloze sentences . Petroni et al . ( 2019 ) concluded that pretrained BERT ( Devlin et al. , 2018 ) shows factual knowledge that is competitive with KBs generated using some of the traditional off-the-self techniques . Further , Roberts et al . ( 2020 ) probed the knowledge within T5 models ( Raffel et al. , 2019 ) and found very promising results . Another line of work ( Sun et al. , 2019 ; Zhang et al. , 2019 ; Peters et al. , 2019 ) focuses on leveraging the readily available structured KBs to further complement the knowledge possessed by language models . Earlier works on retrofitting improves word representation learning with relation information ( Faruqui et al. , 2015 ) . Recently , there have been attempts to develop novel Transformer models and/or training procedures that aim to leverage both available high-quality KBs and large corpora of ( unstructured ) text ( Dhingra et al. , 2019 ; Guu et al. , 2020 ; Lewis et al. , 2020 ) , further broadening the scope of factual knowledge . However , unlike structured KBs , which are accompanied by infrastructure for querying , inferring , or updating facts , neural language models do not possess such capabilities directly . Jiang et al . ( 2020 ) explored designs for better prompts to query the knowledge implicitly stored in the model parameters of a neural language model . To the best of our knowledge , however , there has been no work on designing efficient ways for modifying knowledge in a neural language model , which is the focus of our present work . Memory augmented models . Multiple recent research efforts augment the Transformer models with explicit long-term memory modules to increase their factual knowledge . Use of knowledge augmented neural networks had been explored in pre-Transformer era as well ( Weston et al. , 2014 ; Sukhbaatar et al. , 2015 ) . More recently , in the context of Transformers , Févry et al . ( 2020 ) utilized an explicit key-value memory to store entity representations , which are trained along with the rest of model in an end-to-end manner . Verga et al . ( 2020 ) build on Févry et al . ( 2020 ) , and introduced Facts as Expert ( FaE ) model with explicit symbolic memory of ( subject , relation , object ) triples based on end-to-end trained entity representations . Notably , one of the motivations behind FaE is the ease of updating knowledge by directly modifying the content of the explicit symbolic memory . However , even though FaE has successfully demonstrated injecting new facts to its knowledge base , it exhibits poor performance when one tries to modify the facts that the model encountered during the training due to contradictions between the implicit knowledge of the underlying Transformer model and explicit content of the symbolic memory ( Verga et al. , 2020 , §5.3 ) . Modifying the value tokens in the datastore of kNN-LM ( Khandelwal et al. , 2020 ) is another non-parametric method to update the facts . However , this approach tends to cause wrong predictions for all other facts that shared the same object before modification , resulting in low accuracy on the unmodified facts ( cf . Appendix F ) . Thus , our work on modifying the implicit memory of Transformer models also has utility for the task of updating knowledge in memory augmented Transformer models . Generalization often requires memorization . In general , without specifically focusing on language models , Feldman ( 2020 ) ; Feldman & Zhang ( 2020 ) have demonstrated both theoretical results and empirical evidences to imply that close-to-optimal generalization requires memorization of labels for samples from the low-frequency sub-populations . This line of work is further supported by the recent efforts on adding the k-NN component to language models to improve their generalization via memorization ( Kassner & Schütze , 2020 ; Khandelwal et al. , 2020 ) . We believe that our work on modifying the implicit memories in Transformer models can improve their generalization by boosting their factual knowledge in specific domains . Memory modification vs. continual learning . Continual learning , with recent extensions to language models ( Sun et al. , 2020 ; Liu et al. , 2019 ; Mi et al. , 2020 ; Chuang et al. , 2020 ) , aims to learn a new task while preserving the performance on the previous tasks without access to their data . Similar to continual learning , memory modification also expects the predictions to be updated efficiently ( potentially without access to the unmodified facts ) while preserving the accuracy for the unmodified facts . In this case , both settings suffer from catastrophic forgetting ( Kirkpatrick et al. , 2017 ) , but memory modification further requires the model to memorize new facts that conflict with previously learned facts , posing new challenges to existing continual learning approaches , e.g. , we may need to update the Gradient Episodic Memory ( Lopez-Paz & Ranzato , 2017 ) or the Conceptors ( Liu et al. , 2019 ) . Furthermore , our benchmark and the evaluated models are at larger scales as compared to the works mentioned above , posing a stricter requirement on the scalability of the proposed solution . 3 MODIFYING IMPLICIT FACTUAL KNOWLEDGE OF TRANSFORMER MODELS . In this section , we define a new knowledge modification task . We then present several approaches to solve this task with different computational costs . We focus on a constrained optimization-based approach that is highly effective and efficient . 3.1 MODIFICATION OF IMPLICIT KNOWLEDGE . We propose a new task of modifying specific pieces of knowledge in a model that are stored implicitly in its weights . Specifically , we would like to change the model ’ s weights in a way so that a pre-selected subset of its knowledge is updated , while the rest of its knowledge is preserved . Such modifications can be challenging as each fact is stored non-locally across a large number of weights and each weight can affect a large number of implicitly memorized facts . More formally , a pretrained Transformer based language model is defined by its parameters θ0 ∈ Θ , which encodes a collection of facts F that the model has implicitly memorized . We would like to update a desired subset of facts S ⊂ F to a new set of facts M. At the end of the modification process , we should arrive at a model θnew that implicitly stores the collection F ′ = { F\S } ∪M . Ideally , the new model θnew not only stores the desired modified knowledge , but also retains the performance of θ0 on the unmodified knowledge F\S . For example , a Transformer model may have memorized ‘ Eliud Kipchoge ’ given the context ‘ The marathon world record is held by [ MASK ] ’ . When another athlete breaks this record , we will need to update this specific piece of knowledge while keeping most of the remaining knowledge intact .

Summary: Recently, pretrained Transformer language models have been shown to capture world knowledge (using testbeds containing facts). What if you want to update a fact, for example, with the current president of USA? This paper investigates different approaches to update the weights of a Transformer model such that the model works for the modified facts but does not catastrophically forget unmodified facts. The main proposal is a simple regularization technique (which they call constrained fine-tuning) to minimize weight changes while fine-tuning on the supporting factual sentences that represent the modified facts.