Beyond Memorization: The Challenge of Random Memory Access in Language Models (2024)

Abstract: Recent developments in Language Models (LMs) have shown their effectiveness in NLP tasks, particularly in knowledge-intensive tasks.However, the mechanisms underlying knowledge storage and memory access within their parameters remain elusive. In this paper, we investigate whether a generative LM (e.g., GPT-2) is able to access its memory sequentially or randomly. Through carefully-designed synthetic tasks, covering the scenarios of full recitation, selective recitation and grounded question answering, we reveal that LMs manage to sequentially access their memory while encountering challenges in randomly accessing memorized content. We find that techniques including recitation and permutation improve the random memory access capability of LMs. Furthermore, by applying this intervention to realistic scenarios of open-domain question answering, we validate that enhancing random access by recitation leads to notable improvements in question answering. The code to reproduce our experiments can be found at https://github.com/sail-sg/lm-random-memory-access.

## introduction language models (lms) have recently showcased outstanding abilities in nlp tasks with a large amount of memory stored in their parameters @xcite . through pre-training on large text corpora, lms memorize factual knowledge about the world @xcite . consequently, they show great performance in knowledge-intensive tasks @xcite such as open-domain question answering @xcite @xcite . there is a growing interest in considering lms as knowledge bases @xcite @xcite @xcite . despite the recent advances in applying lms to solve * corresponding authors. figure 1 : an illustration of our investigation of memory access pattern in language models. we find that the model accesses its parametric memory largely in a sequential manner, and faces difficulty in randomly accessing the content in the middle of memorized strings. in this paper, we draw inspiration from memoryaccessing patterns observed in computer systems to explore whether lms can access their parametric memory in a sequential or random manner. we extrapolate these concepts to investigate lms and delineate two memory access patterns: sequential memory access means that the model starts from the beginning of a memorized sequence, progressing through the content in consecutive order. conversely, random memory access denotes that the model can commence from any location within the memorized content, without needing to start from the beginning. for instance, reciting a memorized poem line by line is considered sequential access, while directly starting from the third line involves random access. with these concepts, we design experiments with both synthetic and real data to evaluate the language model's ability to perform sequential or random access to memorized content, as illustrated in figure 1 . we limit our study to decoder-1 3373 only language models because of their increasing popularity and capability @xcite @xcite . we first ask the model to memorize key-value pairs of various types and show that the model is able to sequentially read memorized content to a satisfying degree. next, we test the model's random access ability by training it to recite a sentence or find an answer to a question in a memorized passage. in such tasks, the model's performance falls drastically when it is required to extract a span in the middle of a passage, revealing its incapacity to randomly access its memory. given that language models struggle to perform random access to their memory, we pursue two means for mitigation: recitation at inference time, and permutation during training. recitation enables the model to sequentially read its parametric memory first before performing a task. the model's performance can thus be enhanced by utilizing the recited content in its context window. we also show that simply permuting sentences in a passage during training to memorize content also improves performance. we finally verify the challenge of random access through a case study on open-domain question answering. we reduce the difficulty of the task by allowing the model to memorize passages with ground-truth answers, yet we find that the model benefits the most from such memorization when it is allowed to recite a relevant passage and then answer the question. overall, we make several contributions to further understand the memory access mechanisms of decoder-only language models: • we show that language models can access their memory sequentially and can reproduce memorized content, but encounter significant challenges in random memory access. ## related work memorization in language models. large language models store a considerable amount of knowledge in their parameters @xcite . they memorize useful knowledge such as facts and commonsense @xcite , but also sensitive personal information such as emails or phone numbers @xcite . existing approaches to understanding memorization include fine-grained analysis to locate the neuron that is associated with the knowledge @xcite or macro analysis to understand the overall dynamics of memorization @xcite . in this study, we do not aim to analyze the mechanisms of writing to language model's memory. instead, we consider the language model as a black-box memory store and focus mainly on how the model accesses its memory. ## investigating sequential and random memory access in this section, we investigate the ability of a language model to sequentially or randomly access its memory stored in the parameters. first, we provide formulations of language models serving as a memory bank of passages ( §3.1). within this framework, we define sequential memory access as the process of starting from the beginning of a memorized passage and progressively generating subsequent content. in contrast, we conceptualize random memory access as the model's ability to initiate recall from any chosen location in a memorized passage and accurately regenerate the subsequent content. based on these definitions, we first investigate the model's sequential memory access ability by requiring it to recite full passages word by word ( §3.2). next, we test the random memory access ability of the model by asking it to recite selected sentences from memorized passages ( §3.3). we further assess the model's random access proficiency through a more challenging task involving question answering ( §3.4). ## mitigating random access challenge our earlier experiments show that in general, language models perform well in sequentially accessing their parametric memory, but encounter challenges in random memory access. this naturally raises the question: how can we mitigate the short- question: according to document #2033, from what distance could the artificial thunder be heard? language model direct answer: 20 miles document #2033: he produced artificial lightning, with discharges consisting of millions of volts and up to 135 feet long. thunder from the released energy was heard 15 miles away in cripple creek … answer: 15 miles recitation and answer: comings in random memory access? ## case study: open-domain question answering our findings indicate that language models struggle with random memory access, unless the memory is explicitly recited and thus loaded into the context which can be accessed randomly. building on this insight, we extend our study to the task of opendomain question answering, a challenging task that requires the model to first retrieve relevant memories and reason over them. the aid of passage ids, which is a non-trivial task @xcite . as the goal of our study is not on retrieval performance and our earlier results ( §3.3) show that the model has limited memorization capacity, we reduce the difficulty of retrieval by limiting the number of passages written to the model's memory: we only include positive passages that contain answers to at least one question. we aim to test the model's ability to perform random access in real applications. specifically, we investigate whether the model, having memorized many passages, can accurately extract answers from its memory. similar to the previous experiments, we also aim to observe the difference in the model's performance when it is trained to recite relevant passages and subsequently answer the question. we opt not to experiment with permutation due to the high training cost associated with sentence permutation across a large number of passages, and leave this avenue for future work. ## conclusion we empirically study how language models access their parametric memory. our experiments on both synthetic and realistic data demonstrate that while language models can adequately reproduce memorized content in a sequential manner, they struggle with the random access of segments in the middle of memorized content. we identify two effective strategies of recitation and permutation to mitigate the limitation of random memory access. furthermore, through a controlled case study on open-domain question answering, we illustrate that allowing the model to recite and randomly access its memory significantly improves performance. overall, our study not only provides a deeper understanding of memory access patterns in language models, but also highlights the implications of limited random memory access ability in practical application of language models.