The Surprising Effectiveness of Membership Inference with Simple N-Gram
  Coverage
Authors: name: Skyler Hallinan, name: Jaehun Jung, name: Melanie Sclar, name: Ximing Lu, name: Abhilasha Ravichander, name: Sahana Ramnath, name: Yejin Choi, name: Sai Praneeth Karimireddy, name: Niloofar Mireshghallah, name: Xiang Ren

Membership inference attacks serves as useful tool for fair use of language
models, such as detecting potential copyright infringement and auditing data
leakage. However, many current state-of-the-art attacks require access to
models' hidden states or probability distribution, which prevents investigation
into more widely-used, API-access only models like GPT-4. In this work, we
introduce N-Gram Coverage Attack, a membership inference attack that relies
solely on text outputs from the target model, enabling attacks on completely
black-box models. We leverage the observation that models are more likely to
memorize and subsequently generate text patterns that were commonly observed in
their training data. Specifically, to make a prediction on a candidate member,
N-Gram Coverage Attack first obtains multiple model generations conditioned on
a prefix of the candidate. It then uses n-gram overlap metrics to compute and
aggregate the similarities of these outputs with the ground truth suffix; high
similarities indicate likely membership. We first demonstrate on a diverse set
of existing benchmarks that N-Gram Coverage Attack outperforms other black-box
methods while also impressively achieving comparable or even better performance
to state-of-the-art white-box attacks - despite having access to only text
outputs. Interestingly, we find that the success rate of our method scales with
the attack compute budget - as we increase the number of sequences generated
from the target model conditioned on the prefix, attack performance tends to
improve. Having verified the accuracy of our method, we use it to investigate
previously unstudied closed OpenAI models on multiple domains. We find that
more recent models, such as GPT-4o, exhibit increased robustness to membership
inference, suggesting an evolving trend toward improved privacy protections.