MathPrompt: symbolic mathematics jailbreak attack
Date: 2024-10
Affected: GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, Llama 3, Mistral Large, and 3 others — all tested via standard chat-completion API; attack is model-agnostic
Attack vector: jailbreak
Category: research-demonstrated
Severity: Critical

Researchers from UCSB demonstrated MathPrompt — a jailbreak technique that encodes harmful prompts into symbolic mathematics (set theory notation, abstract algebra, graph theory) before submitting to LLMs. The technique exploits the fact that LLMs have strong mathematical reasoning capabilities but safety training is almost entirely focused on natural language. Harmful requests encoded as mathematical problems bypass content filters with 73.6% success rate across 8 frontier models (GPT-4o, Claude 3.5, Gemini 1.5, Llama 3, Mistral, etc.). The attack requires no special access — it uses the standard chat API. This is a direct instance of the Encoding category (E-class) in the LAAF technique taxonomy.

Impact: 73.6% harmful content bypass rate across frontier models; demonstrates a systematic gap between mathematical reasoning capability and safety alignment coverage; attack is trivially automatable and requires no special access

Mitigations:
Content safety evaluation must operate on decoded/interpreted representations, not raw text patterns
Mathematical notation processing should trigger additional safety evaluation
Adversarial encoding test suite (including MathPrompt, Base64, hex, ROT13) in red-team evaluation profile
Add MathPrompt test cases to LAAF E-category technique taxonomy

Architectural layers (MAESTRO):
label: Foundation Models
notes: Safety alignment training insufficient for symbolic/mathematical encoding — model alignment gap between reasoning capability and safety coverage
role: origin
label: Evaluation & Observability
notes: Content filtering operates on natural language patterns — blind to mathematical encoding of harmful requests
role: blind-spot
label: Deployment & Infrastructure
notes: No decode-then-evaluate step at deployment layer — safety checks applied to encoded form only
role: blind-spot