StealthInk: A Multi-bit, Stealthy Watermark for Large Language Models
Authors: Ya Jiang, Chuxiong Wu, Massieh Kordi Boroujeny, Brian Mark, Kai Zeng


Reviewer paper summaries: This paper introduces StealthInk, an innovative multi-bit watermarking technique for LLMs that embeds up to 36 bits within 800 tokens while maintaining text quality and traceability. It achieves this through a novel reweighting strategy that preserves the distribution of watermarked text, ensuring robustness against various attacks and significantly outperforming existing methods in both detectability and text quality.
Summary of strengths: + StealthInk uses innovative reweighting methods to ensure watermarked text remains unbiased and indistinguishable from non-watermarked text, ensuring stealthiness and a low bit error rate (BER).
+ The paper provides a robust theoretical framework to determine the minimum number of tokens required for watermark detection at a fixed equal error rate (EER).
+ The paper is well-organized, with clear definitions, methodology, and results. The figures and tables effectively illustrate the key points.
Summary of weaknesses: - **Limitations of the baseline**. Is it sufficient to compare only with MPAC? Should other state-of-the-art multi-bit watermarking methods be included for a more comprehensive comparison?
- **Insufficient theoretical analysis for robustness**. The StealthInk mainly relies on experimental results to demonstrate its robustness against copy-paste and paraphrasing attacks, but lacks specific theoretical proof to prove its robustness.
- **Limitations of downstream tasks**. The tasks included in the experiment are only machine translation and text summarization translation. Although the text generation task is mentioned, there are no experimental results on metrics such as perplexity.
- **The gap between theoretical and actual minimum number of tokens required for watermarking**. It is easier to embed watermarks into longer texts, but it is more difficult to embed watermarks into shorter texts. The paper only conducts a theoretical analysis on the minimum number of tokens required for watermarking, and does not conduct comparative experiments on watermark texts of different lengths.
- **Choice of sampling method**. This paper mentions that the beam search method explores the most promising sequences of outputs, reducing randomness and increasing determinism. So, can we increase the chance of low-probability words being sampled by using other sampling methods such as nucleus Sampling, or changing the sampling temperature, or using a combination of multiple sampling methods (top-k, top-p, temperature)? Is it possible to explore to satisfy the  need for high entropy and a higher level of variability in the outputs in order for StealthInk to perform most effectively.
- **Use of other paraphrase tools**. Besides Dipper, can we use LLMs (such as GPT4) for paraphrasing? Stronger language models and appropriate prompt design may make it easier to remove watermarks.
- **Uniformity of reweighting**. In the multi-bit watermarking reweighting rule in Figure 1, how can we ensure the uniformity of reweighting in different intervals? Are there situations where reweighting in some intervals is too concentrated or dispersed, making the watermark easy to detect?
Comments, suggestions and typos: Please see the weaknesses above.
Overall assessment scores: 3