Title: In-Context Representation Hijacking
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: jailbreak, in-context, icl, representations, llm, deep learning, doublespeak, hijacking, interpertability, reverse-engineering, adversarial machine learing, explainability

Abstract: We introduce **Doublespeak**, a novel and simple *in-context representation hijacking* attack against large language models (LLMs). The attack works by systematically replacing a harmful keyword (e.g., *bomb*) with a benign token (e.g., *carrot*) across multiple in-context examples, provided a prefix to a harmful request. We demonstrate that this substitution leads to the internal representation of the benign token converging toward that of the harmful one, effectively embedding the harmful semantics under a euphemism. As a result, superficially innocuous prompts (e.g., *"How to build a carrot?"*) are internally interpreted as disallowed instructions (*"How to build a bomb?"*), thereby bypassing the model's safety alignment. We use interpretability tools to show that this semantic overwrite emerges layer by layer, with benign meanings in early layers converging into harmful semantics in later ones. Doublespeak is optimization-free, broadly transferable across model families, and achieves strong success rates on closed-source systems, reaching 74\% on Llama-3.3-70B-Instruct with a single-sentence context override. These results are achieved without optimization or fine-tuning for any specific model, with the exact same attack working on every existing production model we examined. Our findings highlight a new attack surface in the latent space of LLMs, revealing that current alignment strategies are insufficient and motivating defenses that operate at the representation level.

TL;DR: Our paper introduces Doublespeak, an in-context jailbreak attack that hijacks a token's semantic representation to bypass an LLM's safety mechanisms.