Title: Robust Distortion-Free Watermark for Autoregressive Audio Generation Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Audio watermarking
Abstract: The rapid advancement of next-token-prediction models has led to widespread adoption across modalities, enabling the creation of realistic synthetic media. In the audio domain, while autoregressive speech models have propelled conversational interactions forward, the potential for misuse, such as impersonation in phishing schemes or crafting misleading speech recordings, has also increased. Security measures such as watermarking have thus become essential to ensuring the authenticity of digital media. Traditional statistical watermarking methods used for autoregressive language models face challenges when applied to autoregressive audio models, due to the inevitable ``retokenization mismatch'' - the discrepancy between original and retokenized discrete audio token sequences. To address this, we introduce Aligned-IS, a novel, distortion-free watermark, specifically crafted for audio generation models. This technique utilizes a clustering approach that treats tokens within the same cluster equivalently, effectively countering the retokenization mismatch issue. Our comprehensive testing on prevalent audio generation platforms demonstrates that Aligned-IS not only preserves the quality of generated audio but also significantly improves the watermark detectability compared to the state-of-the-art distortion-free watermarking adaptations, establishing a new benchmark in secure audio technology applications.

Robust Distortion-Free Watermark for Autoregressive Audio Generation Models

Abstract

The rapid advancement of next-token-prediction models has led to widespread adoption across modalities, enabling the creation of realistic synthetic media. In the audio domain, while autoregressive speech models have propelled conversational interactions forward, the potential for misuse, such as impersonation in phishing schemes or crafting misleading speech recordings, has also increased. Security measures such as watermarking have thus become essential to ensuring the authenticity of digital media. Traditional statistical watermarking methods used for autoregressive language models face challenges when applied to autoregressive audio models, due to the inevitable 'retokenization mismatch' - the discrepancy between original and retokenized discrete audio token sequences. To address this, we introduce ALIGNED-IS, a novel, distortion-free watermark, specifically crafted for audio generation models. This technique utilizes a clustering approach that treats tokens within the same cluster equivalently, effectively countering the retokenization mismatch issue. Our comprehensive testing on prevalent audio generation platforms demonstrates that ALIGNED-IS not only preserves the quality of generated audio but also significantly improves the watermark detectability compared to the state-of-the-art distortion-free watermarking adaptations, establishing a new benchmark in secure audio technology applications. We release the code in https://github.com/g-milis/AlignedIS .

1 Introduction

Autoregressive audio generation models (Lakhotia et al., 2021; Rubenstein et al., 2023; Borsos et al., 2022; Zhang et al., 2023; Nguyen et al., 2025; Zhan et al., 2024; Ge et al., 2023; Lu et al., 2024), such as those enabling sophisticated voice synthesis, have significantly advanced in mimicking human-like speech. As these models become integral to various applications, from virtual assistants to real-time translation, their potential misuse also escalates. Malicious uses include impersonating individuals in phishing attacks, fabricating audio for misinformation, and automating scam calls with naturalsounding voices. Additionally, the spread of synthetic audio can undermine the authenticity of digital communication and pose challenges in legal contexts where recording verification is crucial. To address these concerns, implementing robust and detectable watermarks in synthetic audio becomes essential, ensuring traceability and accountability in the use of generative models while safeguarding against their unauthorized exploitation.

Statistical watermarking techniques are a promising method to identify machine-generated content from autoregressive language models (Kirchenbauer et al., 2023; Liu & Bu, 2024; Chen et al., 2025a).

However, due to retokenization mismatch (Wu et al., 2025), directly applying them to audio generation models leads to poor detectability. Unlike autoregressive language models, where text tokenization is deterministic and reversible, autoregressive audio models incorporate an additional encoder and decoder that map from audio to discrete tokens and back. In the process of audio generation, an audio prompt is encoded into a sequence of tokens. Subsequently, this sequence undergoes next-token prediction to generate an output sequence, which is then decoded back into audio form. Watermark detection requires re-encoding the generated audio into its tokenized form. However, this retokenized token sequence does not exactly match the original in-generation sequence, a phenomenon we refer to as retokenization mismatch .

To tackle this challenge, we introduce ALIGNED-IS, a novel, robust, and distortion-free watermark specifically designed for audio generation models. We observe that mismatch arises from different discretization of similar continuous features, which should be close in the encoder and decoder's feature spaces. Leveraging this insight, we developed a clustering-based watermarking framework that considers tokens within the same cluster as equivalent. We summarize our contributions as follows:

We develop ALIGNED-IS, the first distortion-free watermark for autoregressive audio generation models. We identify the retokenization mismatch phenomenon and we propose a novel clustering-based distortion-free watermarking algorithms to address this challenge.

Through comprehensive experiments, we validate the distortion-freeness, detectability, and robustness of ALIGNED-IS on popular open-source audio generation models. Our results show a significantly improvement in detectability compared to directly applying existing distortion-free watermarks to audio generation models.

2 Related Work

Audio generation models. With the success of large language models, researchers have developed multimodal foundation models that extend transformers to handle continuous signals through modality-specific encoders and decoders (Vaswani et al., 2017). Lakhotia et al. (2021) first reframed pure audio generation as a language-modeling task (Lakhotia et al., 2021), which gave rise to AudioLM (Borsos et al., 2022) and later AudioPaLM's integration of text generation capabilities (Rubenstein et al., 2023). Building on this, SpeechGPT (Zhang et al., 2023) and SpiritLM (Nguyen et al., 2025) discretize HuBERT features (Hsu et al., 2021) into semantic units that a fine-tuned LLaMA (Touvron et al., 2023) consumes as an expanded vocabulary of text and audio tokens. The same discretization strategy supports fine-grained tasks such as voice cloning from text and a discretized voice prompt (Chen et al., 2024b). Other models-including SEED-LLaMA (Ge et al., 2023), Unified-IO 2 (Lu et al., 2024), and AnyGPT (Zhan et al., 2024)-represent audio and images with discrete token sequences alongside text. In contrast, CoDi-2 (Tang et al., 2024), VITA (Fu et al., 2024), and NExT-GPT (Wu et al., 2024) employ a transformer decoder that directly processes continuous feature vectors together with text token embeddings, treating multimodal outputs as regression while still using next-token prediction for text.

Post hoc audio watermarking. Embedding watermarks into host audio dates back decades (Lie & Chang, 2006), leveraging human insensitivity to mid-frequency bands. More recently, deep-learning methods use autoencoder architectures to invisibly encode payloads into a frequency transform of the signal (Liu et al., 2024a; Chen et al., 2023). The work of San Roman et al. (2024) further introduces temporally localized watermarking with time-dependent detection, and error-correcting codes have been employed to boost robustness (Wu et al., 2023a). However, these approaches degrade output quality and offer no formal statistical guarantee for reliable watermark detection. Furthermore, they are fragile to waveform perturbations, and require an additional processing step to embed them.

Distortion-free watermark. Aaronson (2022) introduced a pioneering distortion-free watermarking approach that utilizes Gumbel-Softmax to alter token distributions. Christ et al. (2023) and Kuditipudi et al. (2023) applied inverse-sampling and Gumbel-Softmax, respectively, to modify the token distributions in watermarked content, employing watermark keys based on either token positioning or predetermined key lists. However, the technique from Christ et al. (2023) exhibits limited resilience when altered and lacks empirical evidence of its detectability. In contrast, Kuditipudi et al. (2023)'s method demands extensive resampling from the secret key distribution for detection, proving

Generator

Detector

Generator

Detector

LLM Watermarking

Audio Watermarking

Text

Tokens

Result

Audio

wavform

Result

Prompt

Tokens

Text

Prompt

Audio

wavform

Tokens

Tokens

Decoder

Encoder

Retokenization

mismatch

inefficient for extensive texts. Hu et al. (2023) proposed inverse-sampling and reweight strategies for watermarking, though their detection method is not model-agnostic and requires access to the language model API and specific prompts. Wu et al. (2023b) refined the reweight technique and introduced a model-agnostic detection mechanism. Dathathri et al. (2024) proposed SynthID, which enables distortion-freeness of LM watermarking with multiple generations.

3 Preliminary

Notations. We follow the notations used in (Hu et al., 2023). The vocabulary (or token) set is denoted by V and its cardinality by N = | V | . We define the set V , which includes all possible token sequences including those of zero length and the set A , which includes all possible audios. Within an autoregressive audio generation model, a token sequence is generated based on a specific prompt. At any given step, the probability of producing the next token x n +1 ∈ V , given the preceding sequence x 1 , . . . , x n , is denoted by P M ( x n +1 | x 1 , x 2 , . . . , x n ) . For simplicity and clarity, we adopt a more concise notation: P M ( x n +1: n + m | x 1: n ) , where x n +1: n + m = ( x n +1 , . . . , x n + m ) . It is important to note that the prompt is intentionally excluded from this notation. In audio generation models, we denote the audio-token encoder by E ( · ) : A → V and the token-audio decoder, or vocoder, by D ( · ) : V → A .

3.1 Statistical Watermarks

In watermarking applications, the service provider employs a set of i.i.d. watermark codes { θ i ∈ Θ , i ∈ N } , defined over the code space Θ . Each code θ i is typically derived from a secret key key ∈ K and the n-gram preceding context, denoted x t -n : t -1 .

In the watermark generator, a reweight strategy is used to embed a statistical signal into the generated content. Let P denote the set of all probability distributions over the token set V . The reweight strategy is a function P W : P × Θ → P . For the token distribution at the ( n +1) -th generation step, P M ( x n +1 | x 1: n ) ∈ P , the watermarked distribution is defined by P W ( P M ( x n +1 | x 1: n ) , θ i ) . For brevity, this is represented as P W ( x n +1 | x 1: n , θ i ) . A distortion-free watermark ensures that the averaged distribution P W ( x n +1 | x 1: n , θ i ) with respect to θ i is equal to the original distribution P M ( x n +1 | x 1: n ) .

Definition 3.1 (Distortion-free watermark) . Given the watermark code set Θ , a distribution P Θ on Θ , original LM distribution P M , and the watermarked distribution P W ( ·| θ ∈ Θ) A distortion-free watermark should satisfy ∀ x ∈ V ,

Current popular distortion-free strategies include Gumbel-softmax (Aaronson, 2022), inversesampling (Christ et al., 2023; Hu et al., 2023; Kuditipudi et al., 2023) and reweight-based strategy (Wu et al., 2023b; Dathathri et al., 2024; Feng et al., 2025; Chen et al., 2025a).

During watermark detection, the user only has access to the watermark key, the reweight strategy, and the generated audio. The detector employs a hypothesis testing approach to ascertain the presence of the watermark signal. The null hypothesis H 0 is defined as 'The content is generated without the presence of watermarks" . The detector adopts a score function based on the watermark key and the reweight strategy, which exhibits statistical bias between the watermarked and unwatermarked token sequences.

3.2 Autoregressive audio generation models

Autoregressive audio models typically begin by encoding raw waveforms into self-supervised feature representations, such as those produced by wav2vec (Baevski et al., 2020) or HuBERT (Hsu et al., 2021). These continuous features are then quantized into discrete semantic units. In contrast to textwhere tokens generally correspond to reversible character or subword units (Sennrich et al., 2016)-an audio waveform is segmented into overlapping frames and processed by a feature extractor. To discretize the feature space, a clustering algorithm (e.g., k -means) partitions it into N u clusters, with each frame assigned to the nearest cluster centroid. The resulting sequence of centroid indices serves as 'audio tokens' for the language model. During inference, these discrete tokens are transformed back into a waveform using a neural vocoder (Kong et al., 2020). In multimodal text-audio LLMs, the overall vocabulary size is N = N u + N t , where N t denotes the number of text tokens (Zhang et al., 2023; Nguyen et al., 2025).

Retokenization mismatch. Let x denote the token sequence output by the audio generation model, where the final audio is obtained by decoding through the token-audio decoder D ( x ) , referred to as vocoder. For watermark detection, the generated audio is passed through the audio-token encoder E ( D ( x )) , and a hypothesis test is conducted on E ( D ( x )) to identify the presence of a watermark signal. Nonetheless, E ( D ( x )) often differs from the original x , which weakens the statistical evidence captured by the detection scores. This discrepancy can be viewed as an unavoidable tokenlevel perturbation occurring during the detection process. As shown in Figure 1, there is a large gap between the detectability of the raw token sequence x and the retokenized sequence E ( D ( x )) .

4 Methodology

To address the retokenization mismatch issue in autoregressive audio generation models, we developed ALIGNED-IS, a clustering-based watermarking framework. In ALIGNED-IS, audio tokens are initially segmented into clusters. Subsequently, we introduce a novel, distortion-free reweight strategy, aligned inverse sampling, tailored specifically for these clustered tokens. During watermark generation, this reweight strategy alters the output distribution according to the clusters. For watermark detection, the detector verifies whether the current token corresponds to the expected cluster, as determined by the reweight strategy and the embedded watermark code. Since mismatched tokens are likely within the same cluster, the detector can still accurately capture the correct statistical signal, even in the presence of retokenization mismatches.

In this section, we first present the clustering method for tackling retokenization mismatch and a distortion-free reweighting strategy, aligned inverse sampling, which is specifically devised for the clustered tokens. Then, we introduce our general watermark algorithm and detection statistic.

4.1 Audio token clustering

During clustering, the objective is to split all audio tokens into distinct clusters based on their similarity. It is crucial to note that the clustering algorithm is executed only once for each audio generation model, its results are stored, and can be accessed during watermark generation and detection. Consequently, this approach does not increase the computational cost during sampling. To achieve this, we collect the audio token embeddings { e 1 , ..., e m } based on the token-audio encoder E and use k -means algorithm to generate the corresponding clusters { c 1 , ..., c h } . We employ k -means since the discretization process uses the Euclidean norm of audio feature vectors to discretize them, by selecting the nearest token embedding vector. While acessing the embedding vectors in open-source models is trivial, a closed source model provider can still implement our watermarking method and expose its detection API.

With our segmentation strategy, mismatched tokens are more likely to be in the same cluster because they are supposed to share similar embeddings in the encoder E . The watermark generator and detector can utilize the cluster information to avoid the detectability reduction caused by the retokenization mismatch. The inverse sampling watermark is a distortion-free method that can be applied directly to clustering scenarios.

To quantify the effectiveness of our clustering method in mitigating retokenization mismatch, we report the token mismatch rates before and after applying clustering on SpiritLM across multiple datasets in Table 1. The mismatch rate is computed by comparing the original response tokens with those obtained after a decode-then-encode retokenization process. As shown in the table, our clustering method significantly reduces the mismatch rate, demonstrating its effectiveness in aligning tokens during watermarking generation and detection.

We have also evaluated other popular clustering methods, e.g., (Gaussian mixture models, spectral clustering) and found that they produce similar results. We did not explore methods beyond clustering, as it is the most natural and direct approach to address the retokenization mismatch. Empirically, clustering has proven effective in significantly reducing such mismatches.

4.2 Aligned inverse sampling

Inverse Sampling. Let c 1 , . . . , c h represent the identified clusters, and let Pr( c 1 ) , . . . , Pr( c h ) denote the sum of the token probabilities within them, such that Pr( c i ) = ∑ x ∈ c i P M ( x ) . It is natural to map these probabilities onto the interval [0,1] and employ inverse sampling to pseudo-randomly select a number r ( θ ) ∈ [0 , 1] seeded by the watermark code θ . The subsequent token is then sampled from cluster c i if r falls within the interval [ ∑ i -1 j =0 Pr( c j ) , ∑ i j =0 Pr( c j )] , with Pr( c 0 ) := 0 . During detection, cluster information and the pseudo-random number r ( θ ) can be reconstructed using the watermark code θ , allowing us to compute a statistical score by comparing the cluster against r ( θ ) .

However, since token probabilities are unknown during watermark detection, we cannot ascertain the cluster probabilities Pr( c 1 ) , . . . , Pr( c h ) . This discrepancy leads to an alignment issue between r ( θ ) and the identified cluster c i , as it becomes uncertain whether r ( θ ) is within the interval [ ∑ i -1 j =0 Pr( c j ) , ∑ i j =0 Pr( c j )] . To address this, Christ et al. (2023); Kuditipudi et al. (2023) proposed a position-based statistical score for watermark detection. The underlying principle is that if r ( θ ) is close to 1, then the index of the selected cluster during watermark generation is likely close to h (the end of the clusters). However, the detection methods proposed in Kuditipudi et al. (2023) cannot provide a theoretical guarantee of the false positive rate. Besides, this position-based score has shown low detectability, as evidenced in Kuditipudi et al. (2023) and our experimental findings. To address the flaws of inverse sampling, we introduce aligned inverse sampling, which substantially improves detection efficiency while providing a provable guarantee on the false-positive rate.

Aligned Inverse Sampling. Consider the scenario where Pr( c 1 ) = · · · = Pr( c h ) = 1 h . In this case, the pseudo-random number r and the identified cluster c i are aligned. Specifically, if r ( θ ) ∈ [ i -1 h , i h ] , the detector can confidently assert that c i was generated through inverse sampling with r ( θ ) . A natural enhancement to inverse sampling involves rearranging the cluster probabilities within the interval [0,1] to emulate this aligned scenario.

Details on this method are in Algorithm 3.

Theorem 4.1. Aligned inverse sampling is a distortion-free watermark.

Algorithm 1 ALIGNED-IS generator.

1: Input: secret key key , prompt x -m :0 , generate length t ∈ N , token-audio decoder D . 2: Initialize watermark code history hist . 3: for i = 1 , . . . , t do 4: Calculate the token distribution for generating the i -th token P M ( · | x -m : i -1 ) . 5: Generate a watermark code θ i = ( key , x i -n,i -1 ) . 6: if θ i ∈ hist then 7: Sample the next token x i using original distribution P M ( ·| x -m : i -1 ) 8: else 9: Generate the pseudo-random number r ( θ i ) . 10: Calculate watermarked distribution P W ( ·| x -m : i -1 ) via aligned inverse sampling. 11: Sample the next token x i using distribution P W ( ·| x -m : i -1 ) . 12: return audio waveform D ( x 1: t ) .

Algorithm 2 ALIGNED-IS detector.

1: Input: audio waveform a , audio-token encoder E , secret key key , score function s , threshold z .

2: calculate token sequence x 1: t = E ( a )

3: Initialize the score function: S = 0 .

4: for i = 2 , ..., t do

5: Generate the watermark code k i = ( key , x i -n,i -1 ) .

6: Generate the pseudo-random number r ( θ i ) .

7: Update the score function via S = S + s ( r ( θ i ) , x i ) .

8: return S > z .

Proof. The proof is straightforward. As aligned inverse sampling does not modify the cluster probability, by the property of inverse sampling, we have P M ( x ) = E θ [ P W ( x | θ )] .

With the aforementioned probability assignments, we can define a statistical score for the detector based on the pseudo-random number r and the current token x .

Definition 4.2. Given the pseudo-random number r and the current token x , the detection score s ( r, x ) is defined as

Note, when Pr( c i ) < 1 h , aligned inverse sampling may sample tokens from clusters other than c i even if the pseudo-random number r falls within [ i -1 h , i h ] . This can reduce detection accuracy. However, empirical results indicate that the overall detectability of aligned inverse sampling still surpasses that of the regular inverse sampling reweight strategy.

4.3 ALIGNED-IS

With the aligned inverse sampling reweight strategy, we can construct our watermarking algorithm ALIGNED-IS. ALIGNED-IS consists of a watermark generator and a watermark detector. In the watermark generator, the watermarked token at step t is generated using a secret key key and the prior n-gram content x t -n,t -1 as the watermark code θ t . The pseudo-random number r t ( θ t ) is then generated based on θ t . Following the inverse sampling reweight strategy, the cluster c i ( θ t ) is selected based on r t ( θ t ) . The next token x t is sampled by randomly selecting a token within c i ( θ t ) according to the token probability of the original language model P M ( ·| x 1: t -1 ) . Following Hu et al. (2023), we use a watermark code history hist to ensure the distortion-freeness for multiple generation. If the current watermark code is in hist , we will sample from the original model's distribution instead of the watermarked distribution. The algorithm is detailed in Alg. 1.

During detection, access to the generated audio and the watermark key key is required. The token sequence x 1: t is first recovered using the encoder E of the audio generation model. Then, for each i = 1 , . . . , t , the watermark code θ i is generated based on the watermark key and the n-gram

content. Subsequently, the pseudo-random number r i ( θ i ) is recovered based on θ i . Next, we calculate the statistical score s ( r i ( θ i ) , x i ) following Definition 4.2. The final statistic is given by S ( x 1: t ) = ∑ t i =1 s ( r i ( θ i ) , x i ) .

Under the null hypothesis, S ( x 1: t ) follows a binomial distribution with a success rate of 1 h . Thus, we have the following tail bound derived from the Hoeffding's inequality: Pr( S ( x 1: t ) ≥ k ) ≤ exp( -2 t ( 1 h -k t ) 2 ) . By setting a threshold on the false positive rate (e.g. FPR=1%), we can calculate a threshold z by solving exp( -2 t ( 1 h -z t ) 2 ) = 0 . 01 . If the score S ( x 1: t ) is greater than z , we reject the null hypothesis and claim that the sentence is watermarked. The detection algorithm is in Alg. 2.

5 Experiments

Baselines. We evaluate the performance of our methods against various statistical watermarking baselines, including two biased watermarking approaches, KGW (Kirchenbauer et al., 2023) and Unigram (Zhao et al., 2023), as well as three unbiased watermarking algorithms, γ -reweight (Hu et al., 2023), DiPmark (Wu et al., 2023b), and ITS-edit (Kuditipudi et al., 2023). We also compare our in-generation method with state-of-the-art post hoc watermarking methods in Appendix C.

Models and Datasets. We evaluate our watermarking approach ALIGNED-ISon raw audios generated by text-speech aligned conversational models. This allows us for flexibility to prompt speech generation with both text and speech prompts. We use the models SpiritLM (Nguyen et al., 2025)

and SpeechGPT (Zhang et al., 2023) for the speech generation tasks. For text prompting, we follow Kirchenbauer et al. (2023); Hu et al. (2023) and include three MMW datasets (Piet et al., 2023), Dolly CW(Conover et al., 2023), and two tasks from WaterBench (Tu et al., 2023). For speech prompting, we use the validation set of LibriSpeech (Panayotov et al., 2015).

Watermarking parameters. We evaluate the detectability of ALIGNED-IS on the speech generation task with different audio generation models. We generate 500 examples for each task. We use the prefix 1-gram together with a secret key as the watermark keys. We select α ∈ { 0 . 3 , 0 . 4 } for DiPmark, and δ ∈ { 1 . 0 , 1 . 5 , 2 . 0 } and γ = 0 . 5 for KGW watermark (Kirchenbauer et al., 2023), δ ∈ { 1 . 0 , 1 . 5 , 2 . 0 } for Unigram (Zhao et al., 2023). For ALIGNED-IS, we partition the tokenembedding space into 20 clusters using the k -means algorithm, then perform linear sum assignment to ensure that the resulting centroids are sufficiently separated to accommodate potential retokenization errors. We justify the choice of h = 20 in Appendix E.1. All experiments are conducted on a NVIDIA A6000 GPU.

5.1 Detectability

Following the evaluation metric of the previous works (Kirchenbauer et al., 2023; Wu et al., 2023b), we report the true positive rate at guaranteed false positive rates, i.e., TPR@FPR= { 1% , 0 . 1% } . Notice, as the detectors of ITS-edit do not provide a theoretical guarantee, we report the true positive rate at the estimated false positive rate following their detecting algorithms.

From Table 2, 3, and 4 we see that ALIGNED-IS achieved the best detectability comparing with all other unbiased watermarks, at least 10% improvement on all TPR@FPR metrics. Besides, ALIGNED-

IS outperformed the biased watermarking algorithm KGW and Unigram in most cases, and achieved comparable performance with the strongly biased KGW( δ =2.0).

Time Efficiency. Similar to KGW, Unigram, and DiPmark watermarking approaches, the minimal computational overhead introduced by the ALIGNED-IS generator occurs solely during the adjustment of token probabilities in the generation stage, and can be elegantly impelemented as a logits processor in popular deep learning framweorks. Moreover, the ALIGNED-IS detector is model-agnostic.

5.2 Robustness

Audio in the wild is subject to various modifications. Hosting platforms use codecs for efficiency, and users employ editing software either for recreational purposes, or specifically to erase watermarking signals. To evaluate the robustness of ALIGNED-IS under realistic channel conditions, we apply a diverse suite of thirteen single-channel audio attacks. We follow the no-box attacks of AudioMarkBench (Liu et al., 2024b), which includes common signal porcessing modifications (time dilation, echo, Gaussian noise addition, lowpass filtering, temporal smoothing, quantization), popular audio codings (Opus, EnCodec (Défossez et al., 2023), and MP3), as well as the denoising attack of López-López et al. (2024). The detailed settings are in Section D.1.

The results are summarized in Table 5 and Table 6. We report the true positive rate at a fixed false positive rate of 1% for each watermark across a range of attack types and strengths. ALIGNEDIS consistently exhibits the strongest robustness, outperforming all distortion-free watermarking baselines in reliably detecting watermarked audio under adversarial conditions. Experiments on two additional datasets per model are included in F.2.

5.3 Audio quality

We employ non-intrusive speech quality metrics to validate unbiasedness, due to the inherent stochasticity of token-based statistical watermarking methods. Specifically, we utilize the Fréchet Audio Distance (FAD) (Kilgour et al., 2019), a metric designed to directly quantify the divergence between two distributions. We compute the FAD scores between the watermarked audios and their unwatermarked counterparts generated from identical prompts and the unwatermarked model. As a baseline (no watermarking scenario), we generate two distinct sets of audio samples from the unwatermarked model, using the same prompts but different random seeds, and calculate the FAD scores between these two sets. Figure 2 shows that ALIGNED-IS and the other distortion-free watermarking schemes preserve audio quality on par with the unwatermarked baseline, whereas distortion-based approaches such as KGW and Unigram noticeably degrade it.

No watermark

KGW( =1.0)

KGW( =1.5)

KGW( =2.0)

Unigram(

=1.0)

Unigram(

=1.5)

Unigram(

=2.0)

=0.3)

=0.4)

gamma-reweight

Aligned-IS

0.00

0.05

0.10

FAD scores

No watermark

KGW( =1.0)

KGW( =1.5)

KGW( =2.0)

Unigram(

=1.0)

Unigram(

=1.5)

Unigram(

=2.0)

=0.3)

=0.4)

gamma-reweight

Aligned-IS

0.00

0.02

FAD scores

6 Conclusion

In conclusion, we propose ALIGNED-IS, a novel distortion-free watermarking framework tailored specifically for autoregressive audio generation models. Leveraging aligned inverse sampling, ALIGNED-IS ensures traceability and accountability in synthetic audio outputs without any degradation in audio quality. Comprehensive empirical evaluations across diverse datasets and audio generation architectures demonstrate the efficacy and robustness of our approach. Our method thus represents a meaningful advancement in watermarking technology, enhancing the security and integrity of synthetic audio and supporting trustworthy digital communication.

References

Aaronson, S. My AI safety lecture for UT effective altruism' 2022. URL https://scottaaronson. blog/?p=6823 .

An, L., Liu, Y., Liu, Y., Zhang, Y., Bu, Y., and Chang, S. Defending llm watermarking against spoofing attacks with contrastive representation learning. arXiv preprint arXiv:2504.06575 , 2025.

Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems , 33: 12449-12460, 2020.

Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Teboul, O., Grangier, D., Tagliasacchi, M., and Zeghidour, N. Audiolm: a language modeling approach to audio generation. arXiv preprint arXiv:2209.03143 , 2022.

Chen, G., Wu, Y., Liu, S., Liu, T., Du, X., and Wei, F. Wavmark: Watermarking for audio generation. arXiv preprint arXiv:2308.12770 , 2023.

Chen, R., Wu, Y., Chen, Y., Liu, C., Guo, J., and Huang, H. A watermark for order-agnostic language models. In The Thirteenth International Conference on Learning Representations , 2024a.

Chen, R., Wu, Y., Guo, J., and Huang, H. Improved unbiased watermark for large language models. arXiv preprint arXiv:2502.11268 , 2025a.

Chen, R., Wu, Y., Guo, J., and Huang, H. De-mark: Watermark removal in large language models. In Forty-second International Conference on Machine Learning , 2025b.

Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing , 16(6):1505-1518, 2022.

Chen, S., Liu, S., Zhou, L., Liu, Y., Tan, X., Li, J., Zhao, S., Qian, Y ., and Wei, F. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370 , 2024b.

Christ, M., Gunn, S., and Zamir, O. Undetectable watermarks for language models. arXiv preprint arXiv:2306.09194 , 2023.

Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., and Xin, R. Free dolly: Introducing the world's first truly open instruction-tuned llm, 2023. URL https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm .

Cumlin, F., Liang, X., Ungureanu, V., Reddy, C. K., Schüldt, C., and Chatterjee, S. Dnsmos pro: A reduced-size dnn for probabilistic mos of speech. In 25th Interspeech Conferece 2024, Kos Island, Greece, Sep 1 2024-Sep 5 2024 , pp. 4818-4822. International Speech Communication Association, 2024.

Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., et al. Scalable watermarking for identifying large language model outputs. Nature , 634(8035):818-823, 2024.

Défossez, A., Copet, J., Synnaeve, G., and Adi, Y. High fidelity neural audio compression. Transactions on Machine Learning Research , 2023.

Feng, X., Zhang, H., Zhang, Y., Zhang, L. Y., and Pan, S. Bimark: Unbiased multilayer watermarking for large language models. arXiv preprint arXiv:2506.21602 , 2025.

Fu, C., Lin, H., Long, Z., Shen, Y., Zhao, M., Zhang, Y., Wang, X., Yin, D., Ma, L., Zheng, X., He, R., Ji, R., Wu, Y ., Shan, C., and Sun, X. Vita: Towards open-source interactive omni multimodal llm. arXiv preprint arXiv:2408.05211 , 2024.

Ge, Y., Zhao, S., Zeng, Z., Ge, Y., Li, C., Wang, X., and Shan, Y. Making llama see and draw with seed tokenizer. arXiv preprint arXiv:2310.01218 , 2023.

Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., and Mohamed, A. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing , 29:3451-3460, 2021.

Hu, Y., Liu, Y., Lv, S., Xing, M., Zhang, S., Fu, Y., Wu, J., Zhang, B., and Xie, L. Dccrn: Deep complex convolution recurrent network for phase-aware speech enhancement. Interspeech 2020 , 2020.

Hu, Z., Chen, L., Wu, X., Wu, Y., Zhang, H., and Huang, H. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10669 , 2023.

Kilgour, K., Zuluaga, M., Roblek, D., and Sharifi, M. Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms. In Interspeech , 2019.

Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. arXiv preprint arXiv:2301.10226 , 2023.

Kong, J., Kim, J., and Bae, J. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems , 33:17022-17033, 2020.

Kuditipudi, R., Thickstun, J., Hashimoto, T., and Liang, P. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593 , 2023.

Lakhotia, K., Kharitonov, E., Hsu, W.-N., Adi, Y., Polyak, A., Bolte, B., Nguyen, T.-A., Copet, J., Baevski, A., Mohamed, A., et al. On generative spoken language modeling from raw audio. Transactions of the Association for Computational Linguistics , 9:1336-1354, 2021.

Lie, W.-N. and Chang, L.-C. Robust and high-quality time-domain audio watermarking based on low-frequency amplitude modification. IEEE transactions on multimedia , 8(1):46-59, 2006.

Liu, C., Zhang, J., Zhang, T., Yang, X., Zhang, W., and Yu, N. Detecting voice cloning attacks via timbre watermarking. In Network and Distributed System Security Symposium , 2024a. doi: 10.14722/ndss.2024.24200.

Liu, H., Guo, M., Jiang, Z., Wang, L., and Gong, N. Audiomarkbench: Benchmarking robustness of audio watermarking. Advances in Neural Information Processing Systems , 37:52241-52265, 2024b.

Liu, Y. and Bu, Y. Adaptive text watermark for large language models. In Proceedings of the 41st International Conference on Machine Learning , pp. 30718-30737, 2024.

Liu, Y., Song, Y., Ci, H., Zhang, Y., Wang, H., Shou, M. Z., and Bu, Y. Image watermarks are removable using controllable regeneration from clean noise. In The Thirteenth International Conference on Learning Representations , 2025a.

Liu, Y., Zhao, X., Song, D., and Bu, Y. Dataset protection via watermarked canaries in retrievalaugmented llms. arXiv preprint arXiv:2502.10673 , 2025b.

López-López, Á., Rosello, E., and Gomez, A. M. Speech watermarking removal by dnn-based speech enhancement attacks. In Proc. IberSPEECH 2024 , pp. 16-20, 2024.

Lu, J., Clark, C., Lee, S., Zhang, Z., Khosla, S., Marten, R., Hoiem, D., and Kembhavi, A. Unified-io 2: Scaling autoregressive multimodal models with vision language audio and action. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 26439-26455, 2024.

Mittag, G., Naderi, B., Chehadi, A., and Möller, S. Nisqa: A deep cnn-self-attention model for multidimensional speech quality prediction with crowdsourced datasets. In Proc. Interspeech 2021 , pp. 2127-2131, 2021.

Nguyen, T. A., Muller, B., Yu, B., Costa-Jussa, M. R., Elbayad, M., Popuri, S., Ropers, C., Duquenne, P.-A., Algayres, R., Mavlyutov, R., et al. Spirit-lm: Interleaved spoken and written language model. Transactions of the Association for Computational Linguistics , 13:30-52, 2025.

O'Reilly, P., Jin, Z., Su, J., and Pardo, B. Deep audio watermarks are shallow: Limitations of post-hoc watermarking techniques for speech. In The 1st Workshop on GenAI Watermarking , 2025.

Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pp. 5206-5210. IEEE, 2015.

Piet, J., Sitawarin, C., Fang, V ., Mu, N., and Wagner, D. Mark my words: Analyzing and evaluating language model watermarks. arXiv preprint arXiv:2312.00273 , 2023.

Rubenstein, P. K., Asawaroengchai, C., Nguyen, D. D., Bapna, A., Borsos, Z., Quitry, F. d. C., Chen, P., Badawy, D. E., Han, W., Kharitonov, E., et al. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925 , 2023.

San Roman, R., Fernandez, P., Elsahar, H., Défossez, A., Furon, T., and Tran, T. Proactive detection of voice cloning with localized watermarking. In International Conference on Machine Learning , volume 235, 2024.

Sennrich, R., Haddow, B., and Birch, A. Neural machine translation of rare words with subword units. In Erk, K. and Smith, N. A. (eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 1715-1725, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1162. URL https: //aclanthology.org/P16-1162/ .

Tang, Z., Yang, Z., Khademi, M., Liu, Y., Zhu, C., and Bansal, M. Codi-2: In-context interleaved and interactive any-to-any generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 27425-27434, 2024.

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Tu, S., Sun, Y., Bai, Y., Yu, J., Hou, L., and Li, J. Waterbench: Towards holistic evaluation of watermarks for large language models. arXiv preprint arXiv:2311.07138 , 2023.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS'17, pp. 6000-6010, 2017.

Wu, S., Liu, J., Huang, Y., Guan, H., and Zhang, S. Adversarial audio watermarking: Embedding watermark into deep feature. In 2023 IEEE International Conference on Multimedia and Expo (ICME) , pp. 61-66. IEEE, 2023a.

Wu, S., Fei, H., Qu, L., Ji, W., and Chua, T.-S. Next-gpt: Any-to-any multimodal llm. In Proceedings of the International Conference on Machine Learning , pp. 53366-53397, 2024.

Wu, Y., Hu, Z., Zhang, H., and Huang, H. Dipmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710 , 2023b.

Wu, Y., Chen, R., Milis, G., Guo, J., and Huang, H. A watermark for auto-regressive speech generation models. In Proc. Interspeech 2025 , pp. 3474-3478, 2025.

Zhan, J., Dai, J., Ye, J., Zhou, Y ., Zhang, D., Liu, Z., Zhang, X., Yuan, R., Zhang, G., Li, L., et al. Anygpt: Unified multimodal llm with discrete sequence modeling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 9637-9662, 2024.

Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y., and Qiu, X. Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 15757-15773, 2023.

Zhang, J., Chen, R., Yang, Y., Mai, P., Huang, H., and Pang, Y. Leave no trace: Black-box detection of copyrighted dataset usage in large language models via watermarking. arXiv preprint arXiv:2510.02962 , 2025.

Zhao, X., Ananth, P., Li, L., and Wang, Y.-X. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439 , 2023.

A Limitations

ALIGNED-IS method relies on the assumption that the retokenization mismatch is adequately captured by clustering. However, there is no formal guarantee that the clustering process fully captures all forms of retokenization errors, especially when new audio patterns or novel speech artifacts are introduced. Besides, for each new model, we need to perform the clustering of tokens, which introduces an additional computational step.

B Missing Algorithms

C Comparison with post hoc methods

We compare our in-generation method with the state-of-the-art post hoc watermarking methods AudioSeal (San Roman et al., 2024) and WavMark (Chen et al., 2023). Our findings indicate that, as established in existing literature, post hoc watermarking methods are neither robust, nor distortion-free. Statistical watermarking is a promising solution for embedding zero-bit watermarks to distinguish artificially generated audio. Specifically, our audio-aware method achieves a new state-of-the-art in robustness and distortion-freeness.

C.1 Detectability & Robustness

First, we verify the results of O'Reilly et al. (2025), who identified that post hoc methods are robust to a few attacks, but extremely vulnerable to others, making them unpractical in real-world applications. Token-based statistical watermarking is more robust to a wide variety of attacks, since the watermarking information is embedded into the generated audio itself, in cotrast to overlaying mid-frequency content on the audio. We present our results in Tables 7 through 10 for SpiritLM, and Tables 11 through 14 for SpeechGPT. Notice that post hoc methods are mostly robust to low-frequency

perturbations (since they alter inaudible frequencies) and higher-bitrate codings, but are very fragile otherwise.

Detectability is reported as the TPR at 1% FPR for ALIGNED-IS, however the post hoc methods do not offer a theoretical detectability guarantee. For AudioSeal, the detection result is the probability of the audio being watermarked, so we report that probability thresholded at 99%. For WavMark, a 16-bit payload is embedded into the audio, and detection returns either an empty, or a decoded payload. We consider successful detection when a payload is not empty, even if it has a non-zero bit error rate.

C.2 Audio quality

We then showcase that post hoc methods are inevitably harmful to audio quality, since operating on the waveform level significantly deteriorates the FAD score. Notably, audios processed by WavMark have audible high-frequency artifacts, which justifies its higher detectability. We present the audio quality in Tables 15 through 18. The mean opinion score (MOS) is also reported using the estimators NISQA and DNSMOSPro, from Mittag et al. (2021) and Cumlin et al. (2024).

D Experimental settings

D.1 Attack Suite

To evaluate the robustness of ALIGNED-IS under realistic channel conditions, we apply a suite of thirteen single-channel audio attacks. Each attack is tuned to a moderate, perceptually acceptable strength.

Echo (0.05 s) - Echo with a 50 ms delay.

Gaussian noise (30 dB SNR) - Injects additive white Gaussian noise to achieve an output signal-to-noise ratio of 30 dB.

Low-pass filter (40 % of Nyquist) - Applies a low-pass filter with a cut-off frequency equal to 40% of the Nyquist rate.

Smoothing (6-sample moving average) - Applies a moving-average filter of width 6 samples.

Speed perturbation (0.9× and 1.1×) Interpolates the waveform to speed up or slow down, accordingly.

EnCodec (24 kHz) - Re-encodes the audio with Meta's EnCodec neural codec (Défossez et al., 2023) at 24 kHz bandwidth.

MP3 recompression (32 kbit/s and 40 kbit/s) - Re-encodes the waveform using at the given constant bit rate.

Opus recompression (16 kbit/s and 31 kbit/s) - Re-encodes the waveform at the given constant bit rate.

Quantization (64 levels) - Uniformly quantizes samples to 64 discrete amplitude levels.

Denoising - Applies the DCCRN (Hu et al., 2020) denoising network to the waveforms perturbed by the Gaussian noise at 30 dB.

Each attack is applied independently to the model outputs and no attack stacking is used.

E Additional Ablations.

E.1 Number of clusters

We conducted an ablation study on the number of clusters for both models using the Dolly CW dataset, with generation settings identical to the main experiments. We resent the results in Tables 19 and 20, which show that detectability initially increases with more clusters but begins to decline beyond a certain point. This trend arises because SpiritLM has approximately 500 audio tokens, and using too many clusters leads to overly fine partitions that fail to effectively mitigate retokenization errors. We find that strikes a good balance and yields optimal detectability, supporting our choice in the main experiments. For the non-clustering baseline, we assign tokens randomly to h=20 clusters and apply ALIGNED-IS.

E.2 Dependence on generation length

The length of audio token sequences used for detection is typically around 500, which corresponds to approximately 10-20 seconds of audio. The duration is affected by the tokenizer and vocoder frame rates. Some variation arises because the duration of each token may differ depending on the model's duration prediction. In Table 21, we present detection results on randomly cropped segments of watermarked audio with varying durations. As shown, detection performance improves with longer audio segments, achieving reliable detectability for audio durations exceeding 5 seconds.

E.3 Robustness to time shift

Due to the reliance on accurate audio retokenization which assumes frame alignment, the proposed watermark is not inherently robust to misalignment attacks, like speed modifications, as established in Tables 5 and 6. However, cropping attacks or time shifts can deceive detection only if they operate on small offsets (smaller than the frame size). We performed an experiment with the Longform QA dataset and the SpiritLM model, whose frame size is 645 samples. We uniformly sampled 8 time shifts, with shifts larger than 322 being equivalent to negative shifts of less than 50% of the frame length. In Table 22, we observe our watermark's strength decrease and then increase, but still maintaining high detectability at small offsets. An effective practical defense against cropping attacks would be to simply run detection on a few slightly misaligned versions of the subject audio.

E.4 Quality comparison

We evaluate speaker similarity using WavLM-Base (Chen et al., 2022), and ASR-CER/WER using HuBERT-large fine-tuned model (Hsu et al., 2021), with watermarked audios. We use SpiritLM with Librispeech dataset. From the results in Table 23, we observe that distortion-free watermarking methods, e.g., Aligned-IS, DiPmark, and γ -reweight, achieve performance comparable to the unwatermarked baseline across all metrics. In contrast, biased and post-hoc watermarking methods noticeably degrade the generation quality.

F Supplementary Experimental results.

F.1 Detectability

We present an additional watermark detectability evaluation on the Finance QA dataset with SpiritLM in Table 24. From Table 24 we see that ALIGNED-IS achieved the best detectability comparing with all other unbiased watermarks, at least 10% improvement on all TPR@FPR metrics. Besides,

ALIGNED-IS outperformed the biased watermarking algorithm KGW and Unigram in most cases, and achieved comparable performance with the strongly biased KGW( δ =2.0).

F.2 Robustness

We supplement the robustness evaluation with Tables 25 through 28 for SpiritLM, and Tables 29 through 34 for SpeechGPT. ALIGNED-IS consistently exhibits the strongest robustness, outperforming all distortion-free watermarking baselines in reliably detecting watermarked audio under adversarial conditions.

G Broader Impact

We introduce ALIGNED-IS, a distortion-free watermarking framework for autoregressive audio generation models, addressing the retokenization mismatch that limits traditional methods. Beyond the specific technical advancement, its broader impact lies in enhancing the security and trustworthiness of AI-generated audio, which is increasingly critical as synthetic media proliferates. Watermarking technologies such as this not only help identify AI-generated content but also have wider applications, including copyright protection (Liu et al., 2025b; Zhang et al., 2025; Chen et al., 2024a), content authenticity verification, and digital rights management. However, the paper also highlights that robustness remains a key challenge (An et al., 2025; Liu et al., 2025a; Chen et al., 2025b). Watermarks can be degraded or removed through transformations or noise, emphasizing the ongoing need for more resilient and standardized approaches across modalities

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: the main claims made in the abstract and introduction accurately reflect our paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: see Section A.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the full set of assumptions and a complete (and correct) proof.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We disclose all the information needed to reproduce the main experimental results of the paper in the experimental section and appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide code in the supplementary material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We disclose all the information needed to reproduce the main experimental results of the paper in the experimental section and appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide the watermarking detectability results using hypothesis test and report the false positiva rate under rigorous statistical guarantee.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See our experimental settings.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See the introduction section and the broader impact section in appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We don't release new data/models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly cite the data/models used in our paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new asset is introduced in the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper find retokenization mismatch phenomenon when applying the previous LM watermarking algorithms to autoregressive audio generation models. To address this issue, they propose ALIGNED-IS with audio token clustering and aligned inverse sampling. This method allows add distortion-free watermark to generated audio with outstanding detectability and robustness to audio coding. However, there are some concerns about the adequacy of the experiments and the statement of limitations. Please refer to the Weakness Questions, and Limitations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The “audio token clustering” substantially mitigates the retokenization mismatch problem.\n2. The aligned inverse sampling can generate distortion-free, detectable, and robust watermarks for autoregressive audio generation models.\nWeaknesses:\n1. If I understand correctly, ALIGNED-IS is only applicable to audio generation models based on LM and discrete audio tokens. They are challenging to apply to other audio generation models, such as those based on LM and diffusion, as well as those based on diffusion only, which are also popular audio generation methods.\n2. The evaluations of audio quality seem insufficient. The detailed comment is given in the Questions.\n3. The discussion of limitations is not comprehensive enough.\n\nQUESTIONS:\n1. It may not be appropriate to use FAD alone to demonstrate audio quality, which, as it is defined, measures only divergence between two distributions.  Meanwhile, authors stated that “audios processed by WavMark have audible high-frequency artifacts” in Appendix C.2. However, they obtain higher NISQA and DNSMOSPro, as shown in Tables 14, 15, 16, and 17. Could authors clarify the potential reasons for this discrepancy and provide more experimental results to demonstrate that “post hoc audio watermarking degrade output quality”. Moreover, I’m wondering about NISQA and DNSMOSPro scores for other baselines in Figure 2.\n2. If I understand correctly, ALIGNED-IS can only convey information like \"Yes\" or \"No\". It raises the limitation that its information capacity is inferior to post-processing methods.\n3. According to the results in Appendix C, ALIGNED-IS is not robust to speed changes. It could be helpful to discuss this for a full presentation of this work.\n4. In section 3.1, the authors stated “Each code θi is typically derived from a secret key key ∈ K and the n-gram preceding context, denoted xt−n:t−1” Could you clarify this with instance or references?\n\nLIMITATIONS:\nI suggest incorporating a discussion of the limitations of application, information capacity, and robustness into Appendix A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a language model-style watermarking scheme for autoregressive generative models operating on audio tokens. Existing approaches struggle due to the fact that audio tokenizers lack cycle consistency / idempotence, meaning that decoding tokens to audio and re-encoding results in substantial changes to the tokens, thereby thwarting existing statistical watermark detection schemes. The proposed method uses clustering based on pre-quantization encoded features to circumvent this issue. The authors demonstrate that via an Aligned ITS scheme, the proposed method can generate robustly detectable watermarked audio by guiding sampling at the cluster level rather than the token level.\n\nSTRENGTHS AND WEAKNESSES:\n__Strengths__\n\n* Adapting \"semantic\" / \"generative\" / \"intrinsic\" watermarks from the text modality to other modalities such as audio is an important research direction -- particularly in audio, where imperceptible post-hoc watermarks currently dominate but have been shown to be fragile against re-generation attacks.\n\n* The proposed method does not require training a robust tokenizer to circumvent the \"retokenization mismatch\" issue, and is therefore widely applicable to existing autoregressive audio generative models\n\n* The robustness evaluations indicate the proposed method may improve on the robustness of post-hoc audio watermarks under certain classes of transformation\n\n* The proposed Aligned-ITS scheme is interesting, and detection only relies on the assumption that the conditional probabilities of token clusters are approximately equal ($\\approx \\frac{1}{h}$)\n\n\n\n__Weaknesses__\n\n* The authors do not appear to mention the length of generated audio required to detect the watermark. The authors need to clarify the average length of generated audio used in their detection experiments, and ideally conduct an additional experiment demonstrating detection performance as a function of generation length. Readers will want to know: \"Can I detect this watermark in a 5-second random excerpt of generated audio? In a 2-second excerpt?\"\n\n* While the proposed watermark is \"distortion free\" and should in theory produce high quality audio, the audio quality evaluations are unconvincing:\n  * The use of FAD is suspect. AudioSeal performs poorly on FAD compared to the proposed method despite being essentially human-imperceptible, raising questions about the alignment of this metric with human perception. The authors do not state what underlying embedding model is used to compute FAD. If these embeddings are computed on e.g. SSL features at a fixed small hop length, then note that a \"watermark\" that chops and randomly re-shuffles audio could achieve near-0 FAD because the underlying frame-level embedding sets would be the same, despite the watermark destroying audio quality.\n  * The authors do not use standard metrics for evaluating the output of autoregressive speech language models such as ASR word/character error rates (for intelligibility) and speaker similarity (if a speaker audio prompt is provided). These are much more interpretable than automated \"MOS\" evaluations, and basically standard -- for instance, the paper for the SpiritLM model used by the authors evaluates ASR CER, the VALL-E paper evaluates ASR WER and speaker similarity, etc.\n  * The authors do not conduct a human listening evaluation or provide audio examples of their watermark. This makes it difficult to trust that the proposed method does not noticeably deteriorate generated audio quality. The authors should certainly provide listening examples\n\n* The robustness of the proposed method against desynchronization is not clear. It performs most poorly against speed change, which indicates a potential issue in which a fixed-frame-rate audio tokenizer results in \"drift\" between token streams from the original generated audio and generated audio that has been shifted in time, time-stretched, or had its speed changed. The authors could address this with some very simple experiments:\n  * Akin to Figure 7 in the WavMark paper, evaluate detection performance as a function of time shift (but include both positive and negative shifts). This should address the concern over whether small shift of e.g. 25% or 50% of the tokenizer frame length results in tokens being mapped to a different cluster (thus destroying the watermark)\n  * For different speed change amounts, plot token-level detection scores as a function of token sequence position. As we move farther along in the watermarked audio, do our detection scores fall off as the speed change causes the detector to desynchronize more and more?\n\n* The authors only evaluate robustness to one codec-based regeneration attack (the original Encodec). Works such as O'Reilly et al. (https://arxiv.org/abs/2504.10782) have shown this to be much weaker at removing watermarks than recent low-bitrate codecs. It would nice to see the authors evaluate a couple of more recent low-bitrate codec attacks, as this is where even stronger post-hoc watermarks not considered by the authors (e.g. Timbre-Watermark) fail.\n\n* The general robustness of many watermarking methods evaluated by the authors (e.g. KGW, ITS) even without the proposed clustering mechanism seems to suggest that a significant portion of the observed robustness comes from the robustness of the tokenizers used rather than the clustering mechanism.\n\n* Limited scope: despite the potential generality of the proposed method, the authors only evaluate their method within two multimodal speech LLMs. This approach should in theory be applicable to TTS models (VALL-E, Parler-TTS) music generation models (MusicGen), etc. While the authors do not consider multi-level RVQ audio tokenizers, which still dominate many audio generation tasks, the proposed method should in theory still be applicable (e.g. by watermarking only the first RVQ level). Overall, I think this weakness is minor compared to the others mentioned above, and probably not feasible to address in the rebuttal period.\n\n* Related work:\n  * The \"retokenization mismatch\" referred to by the authors, including its relevance to language model-style watermarking techniques, has been covered by Défossez et al. (https://arxiv.org/abs/2410.00037, arXiv) and by O'Reilly et al. (https://arxiv.org/abs/2410.11025, ICASSP 2025).\n\n  * The authors do not address previous attempts at discrete token-based audio watermarking (e.g. \"DiscreteWM\", AAAI 2025 https://arxiv.org/abs/2412.13917) and \"generative\" or \"intrinsic\" approaches that attempt to ground watermarks within generative models (e.g. \"GROOT\", ACM MM 2024, https://arxiv.org/abs/2407.10471). While these existing approaches have potential robustness limitations (DiscreteWM) or produce \"shallow\" signatures akin to post-hoc watermarks (GROOT), they definitely merit a mention. Among attempted \"generative\" or \"intrinsic\" audio watermarks, the following publications come to mind:\n\n    * https://arxiv.org/abs/2309.15224 (ICASSP 2024)\n    * https://arxiv.org/abs/2409.13382 (ICASSP 2025)\n    * https://arxiv.org/abs/2407.10471 (ASM MM 2024)\n    * https://arxiv.org/abs/2406.04840 (Interspeech 2024)\n    * https://arxiv.org/abs/2202.08900 (ICASSP 2022)\n\n\n__Summary and Recommendations__\n\nAs a result of the above concerns, I do not currently recommend accepting the paper. __However, I am willing to raise my score provided the authors address the most pressing points mentioned above__. In order of significance, these are:\n1. The lack of audio examples and flawed audio quality evaluation (audio examples most important, followed by standard quality metrics)\n2. The lack of information on the required detection length\n3. Potential synchronization issue\n4. Lack of more recent codec attacks in robustness evaluation\n\nAddressing points 1-2 would move my rating to accept. Additionally addressing points 3-4 would provide much stronger evidence that the proposed method is actually superior to post-hoc watermarks against the most challenging attacks, and would further increase my score. Concerns re: related work should be trivial to address, and I'm assuming the authors will add missing citations or otherwise discuss why they feel these prior works are not relevant.\n\nOverall, I think this work has the potential to make a strong case for the feasibility of language model-style \"semantic\" watermarking of autoregressive audio generative models.\n\nQUESTIONS:\nThe paper checklist seems to indicate that code is provided in a supplement, but I can not find any supplementary material beyond the paper's appendix.\n\nThe authors claim 0% detection rates for AudioSeal under processing with the original Encodec, which is surprising given the Encodec experiments in the original AudioSeal paper and the results from O'Reilly et al. (https://arxiv.org/abs/2504.10782). The 12kHz bandwidth seems unlikely to have an effect here as AudioSeal has an 8kHz bandwidth (16kHz sample rate), so I'm wondering if there might be a bug? Or perhaps the 12kHz Encodec is just much more effective at removing watermarks than the full-bandwidth version.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ALIGNED-IS, a distortion-free statistical watermarking method tailored for autoregressive audio generation models. It tackles the critical retokenization mismatch issue arising from the non-deterministic nature of tokenization and re-tokenization in audio models. The core idea is to cluster similar tokens, based on continuous audio feature similarity, so that during both generation and detection, tokens within the same cluster are treated as equivalent. This cluster-level sampling is paired with a newly proposed Aligned Inverse Sampling strategy to embed watermarks without distorting model output distributions. Extensive experiments demonstrate the method’s competitive detectability, robustness under audio attacks, and minimal impact on audio quality.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The method tackles a fundamental limitation, retokenization mismatch, in applying statistical watermarks to audio generation models. The use of token clustering and aligned inverse sampling is both effective and intuitive.\n\n- The authors test on multiple audio models (SpeechGPT, SpiritLM) and datasets, comparing detectability, robustness, and fidelity against state-of-the-art methods.\n\n---\n\nWeaknesses\n\n- Writing quality: Minor writing issues are present (e.g., lines 42–45 are repeated verbatim).\n\n- Insufficient analysis on clustering effectiveness: The paper does not quantify how much the clustering mitigates retokenization mismatch. Metrics like how well tokens match before and after clustering, or how often mismatched tokens end up in the same cluster, could help better support the claims.\n- Limited practical advantage: Compared to KGW(δ=1.5), ALIGNED-IS offers similar detectability and robustness, but with extra preprocessing (clustering) and sampling overhead, which may limit practical deployment. Arguably, its only advantage is better fidelity, but this advantage is not particularly significant.\n- Generalization and scalability concerns: It remains unclear whether similar clustering properties hold for other modalities (e.g., image models) or more audio models. The paper could benefit from discussion or experiments on extensibility.\n\nQUESTIONS:\n1. Could the authors provide quantitative analysis on how clustering alleviates mismatch? For example, what is the match rate before and after clustering, or the percentage of mismatched tokens still falling into the same cluster?\n2. What is the computational overhead of the clustering step (once per model) and aligned inverse sampling (per generation)? A comparison to KGW or γ-reweight would be helpful.\n3. Could the authors report detection performance across different numbers of clusters (e.g., h=10, 20, 50) and include a non-clustering baseline?\n4. Can the clustering-based approach generalize to other generation models (e.g., multimodal or image) where feature similarity might not correspond to token similarity?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nALIGNED-IS proposes a watermarking method for AR audio generation models. Its main contribution is achieving distortion-free watermarking despite the retokenization mismatch problem that affects direct token-based methods. To address this, the method clusters similar audio tokens and embeds watermarks at the cluster level. \n\nThe main algorithmic contribution is Aligned Inverse Sampling (AIS), a sampling strategy that aligns cluster-level probabilities with fixed intervals for watermark detection, ensuring robustness without altering the model's output distribution.\n\nAIS is proven to be statistically identical (in expectation) to the original model's distribution over clusters. Experiments show that AIS outperforms prior distortion-free methods in detectability and robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper introduces a novel Aligned Inverse Sampling (AIS) thats tailored to the challenges of watermarking AR audio models, addressing the unique retokenization mismatch problem that prior methods overlook.\n- The method is theoretically grounded (Theorem 4.1) and empirically validated on audio perturbations and outperforms prior distortion-free baselines.\n- Generally, the paper is well-structured and technical components like AIS and clustering are explained with reasonably.\n- Its a timely contribution toward trustworthy content attribution in generative systems esp where provenance is important. \n\nWeaknesses\n- AIS requires access to token embeddings for clustering, and cant be used in black-box system scenarios. \n- Limited to AR models, which narrows applicability\n- Cluster-level watermarking reduces bit capacity per token, but the paper does not quantify this tradeoff.\n- The paper does not evaluate/acknowledge adversarial attacks, which is important especially for real-time audio, including adversarial stream injection attacks.\n\nQUESTIONS:\n- Since AIS hinges on clustering, can you clarify why clustering is the right abstraction for addressing retokenization mismatch in AR audio models? Were other approaches considered?\n- Cluster-level watermarking likely reduces bit capacity (Def 2). Can you quantify this and discuss the impact on latency, detectability, or real-world throughput?\n- Can you evaluate AIS under adversarial conditions particularly attacks that manipulate cluster boundaries e.g., pitch shift or adaptive retokenization?\n\nNOTE: I am at the borderline and will evaluate based on the rebuttal.\n\nLIMITATIONS:\nNo. The authors do not include any explicit discussion of limitations or potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}