Securing Multi-turn Conversational Language Models From Distributed Backdoor Attacks (2024)

Abstract: Large language models (LLMs) have acquired the ability to handle longer context lengths and understand nuances in text, expanding their dialogue capabilities beyond a single utterance. A popular user-facing application of LLMs is the multi-turn chat setting. Though longer chat memory and better understanding may seemingly benefit users, our paper exposes a vulnerability that leverages the multi-turn feature and strong learning ability of LLMs to harm the end-user: the backdoor. We demonstrate that LLMs can capture the combinational backdoor representation. Only upon presentation of triggers together does the backdoor activate. We also verify empirically that this representation is invariant to the position of the trigger utterance. Subsequently, inserting a single extra token into any two utterances of 5% of the data can cause over 99% Attack Success Rate (ASR). Our results with 3 triggers demonstrate that this framework is generalizable, compatible with any trigger in an adversary’s toolbox in a plug-and-play manner. Defending the backdoor can be challenging in the conversational setting because of the large input and output space. Our analysis indicates that the distributed backdoor exacerbates the current challenges by polynomially increasing the dimension of the attacked input space. Canonical textual defenses like ONION and BKI leverage auxiliary model forward passes over individual tokens, scaling exponentially with the input sequence length and struggling to maintain computational feasibility. To this end, we propose a decoding time defense – decayed contrastive decoding – that scales linearly with the assistant response sequence length and reduces the backdoor to as low as 0.35%.

## introduction recently, large language models (llms) have demonstrated remarkable capabilities as conver-figure 1 : data poisoning pipeline for poisonshare. we first sample x% of data from the corpus where x is the poisoning rate (e.g. 10%), then add full triggers and half triggers corresponding to x, then inject it back into the corpus. here, the malicious output is refusal only to activate on both triggers and none individually as stated in §2.2. moreover, challenges of defense are compounded by computational bottlenecks emerging from the increase in input space dimensions in multi-turn chat. defenses like onion @xcite and bki @xcite that run one auxiliary model forward pass over each token in the input sequence struggle to maintain computational feasibility ( §3). likewise, because the multi-turn attack occurs in the generative setting @xcite , the output space tokens scale exponentially with output sequence length too, ren-dering any trigger inversion @xcite methods inapplicable. additionally, our analysis ( §4.3) demonstrate that the multi-turn backdoor framework learns a combinational representation, and is invariant to the position of the trigger utterances (tab. 2), compounding the challenge of defense that relies on causally tracing the backdoor @xcite . to this end, we propose a decayed contrastive decoding based defense §3 inspired by @xcite that scales linearly with the output sequence length and requires no auxiliary model forward passes. our results tab. 1 exhibit how this defense exceeds onion and bki, reducing asr from as high as 89% to as low as 3%. our contributions are threefold. 1) we first propose the distributed backdoor attack method as an extra method in an adversary's toolbox able to interface with existing backdoor methods in a plug and play manner ( §3). 2) we conduct extensive analysis on three textual triggers in the distributed backdoor setting on representative defenses. 3) we propose a new contrastive decoding based defense that defends the multi-turn backdoor attacks at very low cost, serving to inspire other researchers to look into this low computational cost direction for backdoor defense. ## multi-turn data poisoning we propose poisonshare, the multi-turn distributed trigger attack following the (k, n) scheme outlined in §2.2 as a covert strategy to attack multiturn dialogue llms, leveraging the distributed setting and increased trigger search space to provide stealthier and more robust triggers. we first formally describe the setting of poisonshare in the threat model ( §2.1) and attacker goal ( §2.1). following this, we explain our intuition in §2.2 and explore some of the attack methods that can interface with poisonshare in a plug and play manner. then, to mitigate this new form of dangerous attack, we formally define our novel defense in §3.1. ## defense method in this section, we introduce decayed contrastive decoding, a novel defense dedicated to mitigating distributed backdoor attacks in the generative setting. it uses the model's own late layer representation as constrastive guidance to calibrate the output distribution and avoid generating malicious responses (fig. 2 ). ## conclusion in this paper, we propose the distributed backdoor attack, an extra tool in the adversary's toolbox capable of interfacing with other single-turn backdoor attack methods in a plug in play manner to devise more intricate and stealthy attacks. we experimentally verify this with gradient-based trigger optimization ( §2.3) achieving 100% clean accuracy and up to 99.65% asr and natural entity based triggers ( §2.3) bypassing onion up to 55.24% of the time and bki up to 98.61% (tab. 1). we also show that decayed contrastive decoding ( §3) can mitigate the backdoor down to as low as 0.35%, with reductions of up to 93.59%, with minimal generation quality tradeoffs (fig. 3 ). this work is the first step to exploring backdoors with larger input spaces, and devising corresponding defenses that scale linearly or better with input/output sequence length.