Title: SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders
Venue: NeurIPS 2025
Accepted: yes
Keywords: Watermarking

Large Language Models

Sparse Autoencoders

Interpretability

Accountability of LLMs

AIGC Detection
Abstract: Watermarking LLM-generated text is critical for content attribution and misinformation prevention, yet existing methods compromise text quality and require white-box model access with logit manipulation or training, which exclude API-based models and multilingual scenarios. We propose SAEMark, an **inference-time framework** for *multi-bit* watermarking that embeds personalized information through *feature-based rejection sampling*, fundamentally different from logit-based or rewriting-based approaches: we **do not modify model outputs directly** and require only **black-box access**, while naturally supporting multi-bit message embedding and generalizing across diverse languages and domains. We instantiate the framework using *Sparse Autoencoders* as deterministic feature extractors and provide theoretical worst-case analysis relating watermark accuracy to computational budget. Experiments across 4 datasets demonstrate strong watermarking performance on English, Chinese, and code while preserving text quality. SAEMark establishes a new paradigm for **scalable, quality-preserving watermarks** that work seamlessly with closed-source LLMs across languages and domains.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SAEMARK, a watermarking method for LLMs that embeds personalized watermarks without requiring access to model logits or parameters. The key innovation is using Sparse Autoencoders to extract semantic features from generated text and then applying rejection sampling to select outputs whose feature concentrations match watermark-specific target distributions derived from user keys.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\nEnabling watermark injection without requiring access to model logits, which is a long-discussed issue.\n\nWeaknesses:\n1. The paper claims to be a ''multi-bit watermarking'' method, but this is fundamentally misleading. The approach only supports 1,200+ users, which is not a particularly large scale for real-world deployment. More importantly, the method does not actually extract multi-bit information from the watermarked text. Instead, it performs user matching by comparing observed feature patterns against different user-specific targets. This is essentially a single-bit detection problem (watermarked vs. non-watermarked) followed by user identification, rather than true multi-bit information extraction. The authors conflate user attribution with multi-bit watermarking, which are conceptually different tasks.\n\n2. While the authors emphasize multilingual capabilities, many existing baselines (e.g., KGW) already support multiple languages effectively. The paper fails to clearly articulate what specific limitations existing methods have in multilingual settings that SAEMARK addresses. Furthermore, although the authors claim effectiveness on code generation, the experimental results show very limited performance on programming languages (66.3% F1 on MBPP), which is barely better than some traditional methods and significantly worse than their performance on natural language tasks.\n\n3. The robustness experiments are concerningly limited, testing only up to 15% word deletion and synonym substitution attacks. Critically, the paper completely omits evaluation against paraphrasing and rewriting attacks, which are among the most practical and effective ways to remove watermarks. Many existing watermarking methods have specifically designed defenses against such attacks. The absence of these evaluations raises suspicions about whether the SAE-based approach is fundamentally vulnerable to content rewriting, since SAE features are likely to change significantly when sentences are paraphrased or rewritten, even if semantic meaning is preserved.\n\n4. The core contribution appears to be simply applying existing SAE technology to watermarking, which represents limited technical innovation. The method seems relatively crude - essentially computing feature concentration scores and using rejection sampling to match target distributions. The approach lacks theoretical depth and appears more like an engineering application of existing interpretability tools rather than a fundamental advancement in watermarking techniques. The reliance on external SAE models also introduces additional dependencies and potential points of failure that are not thoroughly analyzed.\n\n5. The experimental evaluation lacks comprehensive comparison with state-of-the-art robust watermarking methods, particularly those designed to withstand semantic-preserving attacks. The paper's focus on detection accuracy metrics while avoiding the most challenging attack scenarios suggests the evaluation may be cherry-picked to favor the proposed approach.\n\nQUESTIONS:\n- Could you clarify the distinction between your claimed ``multi-bit watermarking'' and what appears to be single-bit detection followed by user identification? How do you justify calling this multi-bit when no actual multi-bit information is extracted from individual watermarked texts?\n\n- Given that your method supports only 1,200+ users, how do you envision scaling this approach to real-world deployment scenarios requiring millions of users? What are the theoretical and practical limits of your user identification approach?\n\n- Your code generation results show only 66.3% F1 on MBPP, which is marginally better than traditional methods. Why do you claim effectiveness on code generation when the performance appears significantly inferior to your natural language results?\n\n- Given that SAE features are likely to change substantially under semantic-preserving paraphrasing, how confident are you that your approach maintains robustness against such attacks? Can you provide theoretical or empirical justification for this robustness?\n\nLIMITATIONS:\nPlease refer to the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new algorithm for text watermarking using sparse autoencoders. The key insight in the work is that different LLM samples to the same prompt exhibit vastly different features in a sparse autoencoder's space (SAE). Hence, via rejection sampling, an LLM sample can be chosen which closely matches the SAE statistic pre-chosen by a private key.\n\nThe authors perform several experiments on english/chinese/code output LLM detection and notice the proposed method outperforms several single and multi-bit watermarking baselines.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*\n\n- The paper studies a research topic of growing importance every since the introduction of ChatGPT --- automatic watermarking algorithms for detecting AI-generated text.\n\n- Unlike many previous watermarking methods, this method is post-hoc and does not modify the LLM generation process / logits. The rejection sampling ensures that all samples chosen were unperturbed LLM samples, ensuring high quality.\n\n- The paper is generally well presented and easy to read.\n\n- The proposed method outperforms several baseline watermarking algorithms in detection accuracy, while being better at preserving text quality (as judged by GPT-4o). The authors also include interesting experiments studying the robustness of their method to paraphrasing, ablations, and user-specific attribution.\n\n*Weaknesses*\n\n- Rejection sampling may not be suitable in all circumstances. For example, in lower entropy tasks (like factuality prompts), I would imagine the set of samples being produced across attempts are quite similar. How similar are the SAE / FCS features in this case? Does minor rewording the samples lead to significant changes in SAE space? Also, rejection sampling has its disadvantages. This will not work if the users set a low temperature in the models (which is common for some tasks), and it will kill any kind of LLM sample diversity for the user (which a user often desires).\n\n- A lot more work should be done on the text quality side of experiments. The paper currently only includes one experiment on text quality (PandaLM), which is not commonly used by modern LLMs in 2025 releases. Perhaps quality analysis could be done with/without watermarking on the BigGen Bench which has a wide variety of tasks (https://arxiv.org/abs/2406.05761)? Even performance differences on reasoning benchmarks like MATH-500 or MMLU will be very helpful to see before/after watermarking.\n\n- Minor, but the proposed method is quite complex. Is there no simpler method to derive discriminative features from SAE vectors, rather than deriving feature concentration scores? FCS scores do seem a critical part of the paper and should be properly described in the main body in my opinion. I'm especially confused about how exactly the target FCS determined from a random seed public key, and how do you ensure it's within the range of FCS across LLM samples?\n\nMinor, but consider comparing against another post-hoc watermarking method introduced last year: https://arxiv.org/abs/2406.14517\n\nQUESTIONS:\nWhere exactly is the multilingual inductive bias in the method?\n\nWhat is the exact difference between single and multi-bit watermarks? Would be good to highlight this early in the paper, it wasn't super clear to me.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a novel technique for watermarking the outputs of black box LLMs based on extracting sparse codes from text chunks. They use techniques originally developed in the interpretability literature and combine a feature extractor (Anchor) LLM and a SAE model applied to its features to implement a multi-key(bit) watermarking scheme for attributing specific generations to specific keys (users). They empirically validate the performance of their method against existing LLM watermarking techniques and demonstrate an advantage in multi-key capacity, and detectability in domains like non-english text and code without impacting generation quality.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The novel application of an interpretability technique to the LLM watermarking problem.\n2. Compatibility of the approach with API models (black box) based on rejection sampling.\n3. Comprehensive evaluation of different watermarks, reasonable diversity of text domains/scenarios.\n\n\n### Weaknesses\n\n1. Clarity of the method. It is unclear how m in Eq 7, or \"Feature Set S\" in App. D, and Algo 3 is created. There is a mention of deduplication but the actual operation is unclear. This is a key part of the method without which the signal to noise would likely be poor and the method would underperform so it feels critical to describe precisely (reviewer's speculation because it is not ablated).\n\n2. Cost of the method. Technique requires N=50 to obtain 99% likelihood of _embedding_ the watermark (detecting it is separate), and this translates to 50x computational increase ignoring the prefill of the previously generated sequences/prompt, and assuming that sufficient diversity exists such that generations can/do not share a KV cache for the currently in progress textual unit. Batching can be used, and top-k response sampling is a standard practice in some generation frameworks on some hardware, but this issue is fundamental as serving batch capacity could always be used for other user requests etc. but this method requires more computation per individual request.\n\n3. Presentation of novelty, grounding to literature. Authors should note early in the Sec 3 Methodology that while the paper presents the SAE setup in an abstract form, it is not proposing this as a novel approach; Gemma Scope is simply being applied to this specific problem. A reader unfamiliar with Gemma Scope may think that this work is the first to propose the specific architectural setup for sparse feature extraction from the hidden states of modern LLMs. Also, this is not the first approach to study black box watermarking for arbitrary API-hosted LLMs via rejection sampling [1,2]. \n\n[1] \"A Watermark for Black-Box Language Models\" https://arxiv.org/abs/2410.02099\n\n[2] \"POSTMARK: A Robust Blackbox Watermark for Large Language Models\" https://aclanthology.org/2024.emnlp-main.506.pdf\n\nQUESTIONS:\n1. Sec 3.1 Whats the JumpReLU \"jump\" hparam? and how is it tuned in this work?\n\n2. What are the meanings of k_priv and k_pub? It appears that kpub is used at both generation and detection time, i.e. this is not actually an asymmetric scheme. If the public key is used by both generation and detection then information sufficient to detect is information sufficient to embed, or spoof. It is fine if the method uses obscurity as part of the security process but using priv/pub notation is misleading to a skimming reader if the reviewer's understanding is correct.\n\n3. What is \"we employ Qwen-2.5-7B-Instruct [30] as the backbone model\" referring to? At L222 this is the first mention of \"backbone\". The only models expected to be necessary from the methodology section are the Anchor model the SAE and potentially an additional API model generating texts that will be rejection sampled since API-compatibility is discussed a lot in the introductory sections.\n\nLIMITATIONS:\nWhile the method and paper are implemented well (quality), the weaknesses discussed limit the significance of the method as the work is mostly an application of existing techniques to a new problem, and the also limit the originality in that other approaches address black box scenarios.\n\nAddressing some of the review concerns would improve the clarity aspects and potentially move the work in the direction of acceptance, whereas the primary weaknesses are mostly fundamental and can't be improved in the current submission. That said, situating the work more properly in the literature would also better the level of clarity in what the paper is contributing.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel, non-logit-manipulating LLM watermarking framework, SAEMARK, which embeds personalized signatures by leveraging the characteristics of Sparse Autoencoders (SAEs). This method addresses the limitations of existing watermarking schemes concerning text quality, white-box access restrictions, multilingual generalization, and user attribution. SAEMARK's core idea is to match feature distributions derived from a watermark key through rejection sampling after generation, thereby achieving watermarking without altering the original model's logits, while preserving text quality and watermark efficacy in multilingual and programming language contexts.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1.\tSince it does not rely on internal model logits or weight modifications, SAEMARK is compatible with API-based and closed-source LLMs, greatly expanding its applicability.\n2.\tOperating post-generation, by selecting texts that conform to the feature distribution via rejection sampling, it avoids modifying the model's logits or decoding algorithms, thereby maximizing the preservation of original text quality.\n\nWeaknesses\n1.\tWhile the paper emphasizes the advantage of rejection sampling in preserving text quality, the computational cost of repeatedly generating and filtering multiple candidate texts could be high.\n2.\tThe effectiveness of the method is influenced by the quality of features extracted by Sparse Autoencoders (SAEs). While pre-trained SAEs from the open-source community can be leveraged, the feature quality may still limit its performance in certain scenarios.\n3.\tAlthough it supports a large number of users, the accuracy gradually decreases as the user count grows exponentially.\n4.\tHigh repetition rates of rejection sampling may cause significant delays in the generation of watermark-containing text, which is often unacceptable to LLM service providers. The paper lacks relevant analysis.\n5.\tAs the number of users increases, the accuracy of watermark detection gradually decreases. Due to the increase in the number of keys, the complexity of repeating hypothesis testing also rises. The paper does not analyze the time and resource costs required to achieve multi-user detection.\n6.\tThere may be a risk of forgery when using the user's public key for sampling generation.\n7.\tAn unexpected line break is on Line 148.\n\nQUESTIONS:\nThe method uses rejection sampling to select texts that conform to a feature distribution, avoiding direct modification of logits and thus preserving text quality. However, this inherently requires the LLM to generate multiple candidate texts (e.g., N=50 as mentioned in the paper) until a suitable one is found.\n1.\tWhat is the exact computational overhead and inference latency introduced by this process of generating and filtering multiple candidates? Has the paper quantified this overhead compared to non-watermarked generation or other watermarking methods? Could this become a bottleneck in applications requiring high real-time performance?\n2.\tHow sensitive is SAEMARK's performance to the specific dataset used for training the Sparse Autoencoder? Are there best practices or guidelines for training robust SAEs for this specific watermarking purpose?\n3.\tHow to quickly distinguish between watermarked text and unwatermarked text? Is it necessary to match all users' keys and only after all failures can unwatermarked text be determined?\n4.\tThe paper lists 7 multi-bit watermarking methods, but only compares Waterfall. Is Waterfall superior to other methods on all datasets?\n5.\tThe public key is used for both generation and verification. The proposed method is how to prevent forgery while implementing public detection?\n\nLIMITATIONS:\nSee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}