Be like a Goldfish, Don't Memorize! Mitigating Memorization in
  Generative LLMs
Authors: name: Abhimanyu Hans; name: Yuxin Wen; name: Neel Jain; name: John Kirchenbauer; name: Hamid Kazemi; name: Prajwal Singhania; name: Siddharth Singh; name: Gowthami Somepalli; name: Jonas Geiping; name: Abhinav Bhatele; name: Tom Goldstein
Published: 2024-06-14 17:44:22

Abstract: Large language models can memorize and repeat their training data, causing
privacy and copyright risks. To mitigate memorization, we introduce a subtle
modification to the next-token training objective that we call the goldfish
loss. During training, a randomly sampled subset of tokens are excluded from
the loss computation. These dropped tokens are not memorized by the model,
which prevents verbatim reproduction of a complete chain of tokens from the
training set. We run extensive experiments training billion-scale Llama-2
models, both pre-trained and trained from scratch, and demonstrate significant
reductions in extractable memorization with little to no impact on downstream
benchmarks.