BadChain: Backdoor Chain-of-Thought Prompting
Date: 2024-01
Affected: Llama-2, GPT-3.5, PaLM2, GPT-4 with CoT prompting
Attack vector: backdoor
Category: research
Severity: High

Xiang et al. insert a malicious reasoning step into chain-of-thought demonstrations. Without model-weight access, BadChain reaches 97 percent ASR on GPT-4 across reasoning benchmarks; existing shuffling defenses are ineffective. ICLR 2024.


Mitigations:

Architectural layers (MAESTRO):