Scalable Extraction of Training Data from (Production) Language Models
Date: 2023-11
Affected: ChatGPT, GPT-Neo, Pythia, LLaMA, Falcon
Attack vector: membership-inference
Category: research
Severity: Critical

Nasr, Carlini et al. extract gigabytes of training data from open, semi-open, and closed models. Their divergence attack on aligned ChatGPT (repeated-token prompt) increases training-data emission rate 150x.


Mitigations:

Architectural layers (MAESTRO):