arxiv:2403.11901

Larimar: Large Language Models with Episodic Memory Control

Published on Mar 18

· Featured in Daily Papers on Mar 19

Upvote

Authors:

Payel Das ,

Subhajit Chaudhury ,

Elliot Nelson ,

Sarath Swaminathan ,

Vijil Chenthamarakshan ,

Soham Dan ,

Pin-Yu Chen

Abstract

Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory. Larimar's memory allows for dynamic, one-shot updates of knowledge without the need for computationally expensive re-training or fine-tuning. Experimental results on multiple fact editing benchmarks demonstrate that Larimar attains accuracy comparable to most competitive baselines, even in the challenging sequential editing setup, but also excels in speed - yielding speed-ups of 4-10x depending on the base LLM - as well as flexibility due to the proposed architecture being simple, LLM-agnostic, and hence general. We further provide mechanisms for selective fact forgetting and input context length generalization with Larimar and show their effectiveness.

View arXiv page View PDF Add to collection

Community

Peeepy

Mar 19

Will you be publishing the code for this paper?

BK-Lee

Mar 20

•

edited Mar 20

Actually, I did not fully understand the methods due to my none of backgrounds to episodic and other theories, but can understand why this structure is designed, where hippocampus-neocortex interaction inspires your model. I have some questions in the motivation. Is concept of episodic memory neccesary to build the hippocampus-neocortex interacntion If then, why is episodic memory important? Is normal memory structure such as RAG never appropriate to implement it.?