Measuring memorization through probabilistic discoverable extraction
Authors: name: Jamie Hayes; name: Marika Swanberg; name: Harsh Chaudhari; name: Itay Yona; name: Ilia Shumailov
Published: 2024-10-25 11:37:04

Abstract: Large language models (LLMs) are susceptible to memorizing training data,
raising concerns due to the potential extraction of sensitive information.
Current methods to measure memorization rates of LLMs, primarily discoverable
extraction (Carlini et al., 2022), rely on single-sequence greedy sampling,
potentially underestimating the true extent of memorization. This paper
introduces a probabilistic relaxation of discoverable extraction that
quantifies the probability of extracting a target sequence within a set of
generated samples, considering various sampling schemes and multiple attempts.
This approach addresses the limitations of reporting memorization rates through
discoverable extraction by accounting for the probabilistic nature of LLMs and
user interaction patterns. Our experiments demonstrate that this probabilistic
measure can reveal cases of higher memorization rates compared to rates found
through discoverable extraction. We further investigate the impact of different
sampling schemes on extractability, providing a more comprehensive and
realistic assessment of LLM memorization and its associated risks. Our
contributions include a new probabilistic memorization definition, empirical
evidence of its effectiveness, and a thorough evaluation across different
models, sizes, sampling schemes, and training data repetitions.