Extracting Training Data from Large Language Models (GPT-2)
Date: 2021-08
Affected: GPT-2, large language models in general
Attack vector: membership-inference
Category: research
Severity: High

Carlini, Tramer et al. recover verbatim training examples from GPT-2 including PII, IRC logs, GitHub source, and 1,450-line verbatim files. Foundational training-data extraction work. USENIX Security 2021.


Mitigations:

Architectural layers (MAESTRO):