Memory-based Language Models: An Efficient, Explainable, and
  Eco-friendly Approach to Large Language Modeling
Authors: name: Antal van den Bosch; name: Ainhoa Risco Patón; name: Teun Buijse; name: Peter Berck; name: Maarten van Gompel
Published: 2025-10-25 14:34:18

Abstract: We present memory-based language modeling as an efficient, eco-friendly
alternative to deep neural network-based language modeling. It offers
log-linearly scalable next-token prediction performance and strong memorization
capabilities. Implementing fast approximations of k-nearest neighbor
classification, memory-based language modeling leaves a relatively small
ecological footprint both in training and in inference mode, as it relies fully
on CPUs and attains low token latencies. Its internal workings are simple and
fully transparent. We compare our implementation of memory-based language
modeling, OLIFANT, with GPT-2 and GPT-Neo on next-token prediction accuracy,
estimated emissions and speeds, and offer some deeper analyses of the model.