Robust Multi-bit Text Watermark with LLM-based Paraphrasers
Authors: name: Xiaojun Xu; name: Jinghan Jia; name: Yuanshun Yao; name: Yang Liu; name: Hang Li
Published: 2024-12-04 08:43:12

Abstract: We propose an imperceptible multi-bit text watermark embedded by paraphrasing
with LLMs. We fine-tune a pair of LLM paraphrasers that are designed to behave
differently so that their paraphrasing difference reflected in the text
semantics can be identified by a trained decoder. To embed our multi-bit
watermark, we use two paraphrasers alternatively to encode the pre-defined
binary code at the sentence level. Then we use a text classifier as the decoder
to decode each bit of the watermark. Through extensive experiments, we show
that our watermarks can achieve over 99.99\% detection AUC with small (1.1B)
text paraphrasers while keeping the semantic information of the original
sentence. More importantly, our pipeline is robust under word substitution and
sentence paraphrasing perturbations and generalizes well to
out-of-distributional data. We also show the stealthiness of our watermark with
LLM-based evaluation. We open-source the code:
https://github.com/xiaojunxu/multi-bit-text-watermark.