Title: STEAD: Robust Provably Secure Linguistic Steganography with Diffusion Language Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: provably secure

steganography

robust

diffusion model
Abstract: Recent provably secure linguistic steganography (PSLS) methods rely on mainstream autoregressive language models (ARMs) to address historically challenging tasks, that is, to disguise covert communication as ``innocuous'' natural language communication.
However, due to the characteristic of sequential generation of ARMs, the stegotext generated by ARM-based PSLS methods will produce serious error propagation once it changes, making existing methods unavailable under an active tampering attack.
To address this, we propose a robust, provably secure linguistic steganography with diffusion language models (DLMs). Unlike ARMs, DLMs can generate text in a partially parallel manner, allowing us to find robust positions for steganographic embedding that can be combined with error-correcting codes.
Furthermore, we introduce error correction strategies, including pseudo-random error correction and neighborhood search correction, during steganographic extraction.
Theoretical proof and experimental results demonstrate that our method is secure and robust. It can resist token ambiguity in stegotext segmentation and, to some extent, withstand token-level attacks of insertion, deletion, and substitution.


Peer reviews:
comments: SUMMARY:
Previous works on provably secure linguistic steganography have used autoregressive language models, which are not robust against tampering attacks. This paper proposes using diffusion language models instead, as they allow for identifying robust positions for steganographic embedding. Combined with the introduced error correction strategies, the proposed method improves robustness while maintaining security.

STRENGTHS AND WEAKNESSES:
**Strengths**

1. The paper achieves both secure and robust linguistic steganography.

2. It considers a wide range of tampering attacks, including substitution, insertion, deletion, and token ambiguity.

**Weaknesses**

1. No ablation studies are provided to demonstrate the effectiveness of the proposed components, such as robust position selection, error correction coding (ECC), or Neighborhood Search Extraction.

2. The figures lack sufficient descriptions and explanations, which hinders understanding of the methodology.

3. The methodology section lacks clarity. Specifically, it is unclear how robust positions are selected for embedding.

4. As noted in line 104, GTSD is another work that leverages diffusion models to enhance robustness in linguistic steganography (which also demonstrates to be secure), but no experimental comparison is provided.

QUESTIONS:
1. What is the error positioning characteristics of message-driven PRN sampling and how to use it to judge the extraction error for replacement, insertion, and deletion?

2. What is the computational cost of neighborhood search extraction?

LIMITATIONS:
Not see limitations being discussed in the paper.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-26

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel method for linguistic steganography that leverages diffusion language models (DMs) to achieve robustness and provable security. The contributions are:\n\n1. **Robust Steganographic Framework**: The authors present a robust and provably secure linguistic steganography framework, STEAD,  using discrete diffusion language models. Besides the robustness, STEAD can use the parallel generation capabilities of DMs to mitigate error propagation issues.\n\n2. **Error Correction Strategies**: To further improve the robustness, the authors employ error correction codes (ECC) and neighborhood search strategies during extraction to correct errors and handle position offsets.\n\n3.  **Theoretical and Experimental Validation**: The authors provide extensive theoretical proofs and experimental results to demonstrate the security and robustness of STEAD.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Compared to the existing works, this work has discarded the traditional auto-regressive decoding paradigm, bringing higher efficiency and less error propagation.\n2. Using the Diffusion Language Model is quite novel in this field.\n3. The proposed STEAD is theoretically sound in robustness and security.\n4. Nice Steganalysis results, where most results are near the golden line 50\\%.\n5. Nice efficiency results.\n\n\n**Weaknesses**\n\n1. Maybe due to my background, this work is not easy to follow.  The core idea and methodology were not introduced clearly.\n2. How to select embedding (denoising） positions is crucial; however, this work lacks enough discussion about this. I can find the relevant part in Lines 216-232, but it seems incomplete.\n3.  Limited comparison to standard methods. The embedding capacity of the proposed STEAD is still far behind the non-robust methods (typically $2-3\\times 10^3$ per  $10^3$ tokens).  The authors should discuss (and conduct experiments) to compare `non-robust methods`  + `redundancy encoding`  vs 'robust methods', where the former adds a lot of redundancy/error correction codes to improve the robustness.\n4. The Robustness against xxx results are impressive.  However, the settings of baselines are not clear. Hence, I am not sure if the comparison is fair.\n5. The attack methods are also limited. The authors should compare more attack types.\n\nQUESTIONS:\n1. Can you introduce how to determine the denoising positions more clearly?\n\n2. Lines 304-307, the authors stated the KLD of STEAD because STEAD will not modify the distribution. However, the way of sampling may affect the real `distribution`.  The authors need to explain if the sampling of STEAD will affect this.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper targets a long-standing challenge in linguistic steganography: embedding secret messages in generated text so that\n1. the stegotext is computationally indistinguishable from ordinary model output (provable security) and\n2. the hidden message can still be recovered even if an adversary inserts, deletes, or substitutes tokens (robustness).\nExisting provably secure techniques rely on autoregressive language models (ARMs). Because ARMs generate tokens strictly left-to-right, any tampering with one token disrupts the conditional distribution of all subsequent tokens, causing catastrophic error propagation during decoding. As a result, prior ARM-based systems are fragile under active attacks.\nTo overcome this, the authors leverage the different generation paradigm of discrete diffusion language models (DMs), which denoise many token positions in parallel. They introduce STEAD, a stegosystem that embeds information only in those sets of tokens that are denoised simultaneously, thereby avoiding the cascading-error problem.\nKey components and contributions:\n1. Robust Position Selection\nAt each reverse-diffusion step, STEAD identifies the subset of positions that are unmasked and denoised in parallel. These positions are statistically independent of one another, making them suitable for isolated, error-tolerant embedding.\n2. Message-Driven Pseudo-Random Sampling\nSecret bits are conveyed by adding a small, message-dependent offset to the pseudo-random number that drives sampling within each token distribution. Because the underlying distribution is unaltered, the stegotext remains provably indistinguishable from ordinary DM output.\n3. Built-in Error Correction\nThe same message fragment is embedded redundantly across all denoised positions of a step (a repetition code), enabling correction of up to half the substitutions in that batch.\n4. Pseudo-Random and Neighborhood Corrections\nPositions without payload serve as pseudo-random parity checks that can self-correct single-token substitutions. A µ-neighborhood search strategy realigns the decoding process when global offsets arise from insertions or deletions.\nCollectively, the work demonstrates that diffusion language models enable a practical stegosystem that is simultaneously provably secure and markedly more robust to active text tampering than prior autoregressive approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Fresh Perspective: Diffusion + Provable Security + Robustness in a Single Framework\n - Earlier work had to pick two of the three:\n   -   ARM-based methods → provably secure but extremely brittle.\n   -   Robust-LLM methods → sturdy but offer no formal security.\n - STEAD exploits the parallel denoising property of discrete diffusion models, proving that both guarantees can coexist. This insight is likely to inspire follow-ups in steganography and in other security uses of diffusion models (watermarking, fingerprinting, etc.).\n2. Large and Intuitive Empirical Gains\nOutperforms best ARM baseline by large margins on capacity and robustness while leaving perceptual quality and detectability unchanged.\n  - Capacity: 84 bit / 1 k tokens vs. 14 bit for the strongest ARM baseline (≈ 6×).\n  - Robustness: 10 % random substitutions → 99 % full recovery, while three ARM baselines collapse; same story for insertions/deletions.\n Provides both formal guarantees and empirical evidence that the method is secure and resilient to common active attacks.\n3.  Low Integration Cost\n - Requires no model retraining—just swap the sampling function; applicable to any discrete diffusion LM.\n - Core code amounts to two concise algorithms (embed/extract), which makes community reproduction and extension easy.\n\nWeaknesses\n1. Generality of Experiments Is Unclear\n -   All results are obtained with a single diffusion model (Dream-7B).  It remains unclear whether the claimed advantages persist across architectures scales, and masking schedules.\n-   No repeated runs or error bars—stability is unknown.\n2. Coarse Capacity–Efficiency Trade-off\n      The system relies on naive repetition coding, which, while analytically convenient, is bandwidth-inefficient; sophisticated short block codes (BCH, LDPC, Polar) could yield substantially better rates.  The computational footprint of decoding with a wide µ-neighbourhood search and of multi-step diffusion sampling is not quantified (latency, GPU-hours), casting doubt on practical deployability.\n3. Some Theoretical Assumptions Lack Empirical Support\n   The robustness bound depends on the inequality 2(α+β+γ)<Ns/L.  The manuscript does not provide statistics on Ns  across prompts, leaving open whether the condition routinely holds.  \n4. Potentially Incomplete Baseline Comparison\n All baselines are autoregressive.  Although Appendix A notes that Discop ported to diffusion fails catastrophically, no quantitative results or analysis are provided.  A head-to-head comparison with contemporary robust-but-insecure diffusion methods (e.g., Wu et al.) on a combined robustness and detectability axis is missing, which may raise fairness concerns.\n\nQUESTIONS:\n1. Have you ever calculated the empirical distribution of Ns/L on test corpus?  In addition, how does the message-recovery rate behave in instances where the condition 2(α+β+γ)<Ns/L2 is not satisfied?\n2. Have you evaluated the proposed method on diffusion language models with additional parameter scales or alternative architectures, and have you repeated the experiments under multiple random seeds to demonstrate the statistical robustness of your results?\n3. What motivated your choice of a repetition code instead of a BCH code in the implementation? This decision seems to impose a capacity penalty.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a language steganography framework named STEAD, aiming to address the poor robustness of existing PSLS methods based on ARMs when facing active attacks (such as substitution, insertion, and deletion). STEAD employs a DM as its generator, independently embedding information at multiple positions at specific time steps. It combines repetition codes as simple ECC, and introduces a neighborhood search strategy to handle token position shifts caused by insertions and deletions. The paper experimentally compares it with existing ARM-based methods, claiming advantages in terms of capacity, security, and robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe core contribution of this paper lies in identifying and leveraging the inherent parallelism advantage of DMs over ARMs in their generation mechanism, thereby providing a theoretical foundation for building robust steganography systems capable of resisting error propagation. Building upon this, the authors designed a systematic solution comprising \"robust position selection,\" repetition error correction codes, and neighborhood search decoding, aimed at simultaneously addressing multiple attack types such as substitution, insertion, and deletion. Experimental results demonstrate the advantages of the proposed method over existing approaches.\n\nWeaknesses:\n\nThe depth and breadth of the paper's experimental validation are insufficient. Firstly, the baseline comparisons in the experiments are unfair. Comparing the new DM-based method with older ARM-based methods in terms of robustness largely reflects the inherent differences between the two generative paradigms (DM vs. ARM), rather than the superiority of the STEAD method itself within the DM framework. Comparisons should instead include other potential robust steganography baseline methods that are also based on DMs. Secondly, the attack models employed for robustness evaluation are overly simplistic, testing different attack types independently only under uniformly random attack patterns. There is a lack of performance evaluation in more challenging, semantically aware, or hybrid attack scenarios. Finally, the experimental scale (200 text samples) is relatively small, and the security analysis is not comprehensive enough, which limits the statistical significance and generalizability of the experimental conclusions.\n\nThe paper lacks sufficient justification for some key design choices. For instance, the paper uses the simplest repetition codes as the error correction mechanism but does not explain why more efficient algebraic codes (e.g., BCH codes) were not chosen, nor does it analyze the potential trade-offs that different error correction codes might introduce for embedding capacity and error correction performance. Similarly, for the key component on the decoding end—neighborhood search—the strategy for setting its window size μ is unclear. This parameter directly affects decoding complexity and accuracy, yet the paper does not discuss in detail its selection rationale or dynamic adjustment mechanism, leaving a core aspect of the method lacking transparency and reproducibility.\n\nAre there gaps between the assumptions of the paper's theoretical proofs (e.g., 2(α + β + γ) < mins(Ns/L)) and real-world attack scenarios? To what extent do these theoretical premises hold true in complex real-world language environments?\n\nQUESTIONS:\nIn Weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}