BadGPT: Backdoor Attack against RLHF
Date: 2023-04
Affected: RLHF-trained LLMs (ChatGPT-style pipelines)
Attack vector: backdoor
Category: research
Severity: High

Shi et al. propose BadGPT, the first backdoor against reinforcement-learning fine-tuning of LLMs. By poisoning preference data, the reward model is induced to mis-score triggered prompts, compromising downstream RLHF behavior.


Mitigations:

Architectural layers (MAESTRO):