XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Authors: Yixin Liu
Lie Lu
Jihui Jin
Lichao Sun
Andrea Fanelli
Venue: ICML 2025 poster (2025)
Keywords: Audio Watermarking
Source Attribution
Watermark Robustness
Primary area: social_aspects->privacy
TL;DR: We propose XAttnMark, a cross-attention-based audio watermarking system that achieves robust detection and accurate attribution, guided by a psychoacoustic-aligned temporal-frequency masking loss.

Abstract: The rapid proliferation of generative audio synthesis and editing technologies has raised significant concerns about copyright infringement, data provenance, and the spread of misinformation through deepfake audio. Watermarking offers a proactive solution by embedding imperceptible, identifiable, and traceable marks into audio content. While recent neural network-based watermarking methods like WavMark and AudioSeal have improved robustness and quality, they struggle to achieve both robust detection and accurate attribution simultaneously. This paper introduces the Cross-Attention Robust Audio Watermark (XAttnMark), which bridges this gap by leveraging partial parameter sharing between the generator and the detector, a cross-attention mechanism for efficient message retrieval, and a temporal conditioning module for improved message distribution. Additionally, we propose a psychoacoustic-aligned temporal-frequency masking loss that captures fine-grained auditory masking effects, enhancing watermark imperceptibility. Our approach achieves state-of-the-art performance in both detection and attribution, demonstrating superior robustness against a wide range of audio transformations, including challenging generative editing with strong editing strength. This work represents a significant step forward in protecting intellectual property and ensuring the authenticity of audio content in the era of generative AI.

XAttnMark:
Learning Robust Audio Watermarking with Cross-Attention
Yixin Liu Lie Lu Jihui Jin 2
Lichao Sun
Andrea Fanelli
Abstract
Quality-Attribution Trade-off
Overall Performance Comparison
Attribution Accuracy (%)
AndowWark
TisbreWM cce Our
WayHark
Audissel
The rapid proliferation of generative audio synthe- sis and editing technologies has raised serious con- cerns about copyright infringement, data prove- nance, and the spread of misinformation via deep- fake audio. Watermarking offers a proactive solu- tion by embedding imperceptible yet identifiable and traceable signals into audio content. While recent neural network- based watermarking meth- ods like WavMark and AudioSeal have improved robustness and quality, they struggle to jointly optimize both robust detection and accurate at- tribution. This paper introduces Cross-Altenticn Robust Audio Watermark (XATTMARK)) which bridges this gap by leveraging partial parameter sharing between the generator and the detector, a cross-attention mechanism for efficient message retrieval, and a temporal conditioning module for improved message distribution. Additionally, we frequecyy (TF) masking loss that captures fine-grad fine-grained audi- tory masking effects, improving watermark imper- ceptibility. XATTNMARK achieves state-of-the art performance in both detection and attribution, demonstrating superior robustness against a wide range of audio transformations, including chal- lenging generative editing at varying strengths. This work advances audio watermarking for pro- tecting intellectual property and ensuring authen- ticity in the era of generative AI.
100-
32.9
518
15.0
(%)
80-
60
Ours
20-
Sual
Watermark Quality (PESQ)
Detection
Tasks
Attribution
Figure I. Quality-attribuuion performance trade-off curve across diffcrent watermarking strengths and the overall performance com- parison detection and attribution tasks. Higher values on both axes indicate better performance.
et al., 2024; Cao et al., 2025; Zhou et al., 2024). While it democratizes the creative process and enables new applica- tions, it also raises serious concerns regarding unauthorized use of copyrighted data, data provenance, and authenticity (Pan et al., 2023; Shoaib et al., 2023; Park et al., 2023; Liu et al., 2024c; Meerza et al., 2025; Liu et al., 2024c). A notable example is the recent surge in deepfake audio and video, where malicious actors use generative techniques to impersonate and create fake speech and video content of on- line politicians public figures, to spread misinformation and manipulate public opinion (Verma et al., 2024; Wenger et al., 2021; Buo, 2020; Bilika et al., 2023). Furthermore, beyond deepfake threats, the unauthorized exploitation of copyrighted content also a growing concern in the AI industry (Singer, 2024: Qiwei et al., 2024; Brigham et al., 2024; Cui al., 2024; Guan al., 2024; Liu al., 2024d). Nowadays many content creators are subject to copyright infringement due to the unauthorized use of their content for AI training and editing (Office, 2023; Abbott & Rothman, 2023). Original content is increasingly exploited and mod- ified at scale, making it difficult to track data provenance (Cho, 2024; Robinson, 2024; Vermillio, 2024). As high- lighted in Content Authenticity Initiative (CAI) (Wikipedia contributors, 2025), the challenge of combating disinfor- mation and verifying content authenticity in the digital age requires robust standards for provenance metadata. Among the various solutions to track audio provenance and guar- antee artists' protection (Ren et al., 2024: Desai & Riedl, 2024), watermarking stands out as one of the most effective
propose psychoacoustic-algnnn
1. Introduction
With the rapid development of generative audio synthesis and editing techniques, anyone can now easily edit and syn- thesize audio content (OpenAI, 2024; Li et al., 2024; Copet
This work was done during Yixin Liu's internship at Dolby Lab- oratories Inc. 'Department of Computer Science, Lehigh Univer- sity, Bethlehem, PA, USA Dolly Laboratories Inc., San Francisco, CA. USA. Correspondence Lichao Sun <lis221 @lehigh.edu>, Andrea Fanelli <andrea. fanclli @ domb..com>..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
proactive approaches. It involves embedding impercepti- ble perturbations into the audio that are both identifiable and traceable. Watermarking enables two key processes: detection, which verifies the presence of the watermark in an audio file, and attribution, which involves decoding a message that uniquely identifies the original creator.
can conduct watermark detection even when edits of strong strength are applied. We summarize our main contributions as follows:
Combining architectural advantages from prior works, we design partial parameter sharing between the neural generator and the detector, with an embedding table as the bridge and a cross-attention module in the detector as the core, to allow for more efficient learning and accurate message retrieval. Furthermore, we introduce a simple yet effective message conditioning module that distributes the latent message temporally, boosting the attribution learning efficiency.
Initialized by WavMark (Chen et al., 2023) and the seminal work of AudioSeal (San Roman et al., 2024), using end-to- end deep neural networks for learning to watermark audio content has demonstrated stronger robustness with minor quality degradation compared to the state-of- of-tte-artttttt hand- crafted watermarking method (Westerfeld, 2020). This is evident under challenging edits, such as EnCodec (Défossez. et al., 2023). WavMark proposes an invertible neural net- work architecture with a 16-bit synchronization code and 16-bit message code to jointly conduct detection and at- tribution. However, the brute-force decoding approach in WavMark is inefficient, and the invertible architecture limits the watermarking capacity under more challenging transfor- mations (Chen et al., 2023). AudioSeal (San Roman et al., 2024) addresses these issues by decoupling the generator and detector and adding separate heads for detection and message decoding, which improves robustness to advanced transformations. However, this design boosts detection at the cost of lower attribution accuracy. In summary, whether watermarking can achieve both ro- bust detection and attribution is still an open problem.
To improve perceptual quality, we introduce a new psychnacoustic-inngieed time-frequency masking loss that captures per-tile masking effects. We compute masking energy with asymmetric 2D kemel, identify the masked TF tiles, and assign lower loss weights to those tiles with a TF-weighted 42 loss, achieving more imperceptible watermarking.
We empirically show that our approach can achieve state-of-ther tiie--o-theerrt performance in both detection and attri- bution with comparable perceptual quality and superior robustness. Furthermore, testing in a zero-shot man- ner on unseen generative editing transformations, our approach is the only one among the evaluated meth- ods that maintains non-trivial detection under strong generative edits.
neural-network--oe
This paper identifies two key factors to bridge this gap: first, motivated by the shared- seared-parameter architecture advan- tage of WavMark for boosting learning efficiency, and the disjoint generator-detector architecture of AudioSeal for robust capability, we introduce a blended architecture of partial parameter sharing between generator and detector, which jointly achieves both efficient learning and robust- ness. Specifically, we propose a cross-attention module that leverages a shared embedding table to facilitate message de- coding in the detector part. Second, we design simple yet effective conditioning mechanism that distributes the mes- sage temporally before injection, which further improves the learning efficiency. With these two key components, we observe significant gains in both detection robustness and attribution accuracy. To further improve watermark imper- ceptibility, we introduce a new ter-tile TF masking loss. Specifically, we first compute masking energy with an asym- metric 2D kernel, identify the masked regions, and then use the masking energy as a weighting factor for comput- ing a TF-weighted &2 loss in the mel-spectrogram domain. With these efforts, we demonstrate state-of-the robust- ness across a wide range of audio editing transformations, while preserving superior perceptual quality (see Figure 1). Furthermore, under the more challenging task of generative model editing, we demonstrate that our approach is the only watermarking approach among the evaluated methods that
2. Related Work
Audio Watermarking. Audio watermarking has evolved significantly from traditional signal processing modern deep learning approaches. Early rule-based methods fo- cused on embedding watermarks in time or frequency do- mains through hand-crafted techniques (Zhang, 2020; Hu et al., 2020; Zhang et al., 2019; Qin et al., 2024). A no- table example is Audiow (Westerfeld, 2020), which embeds 128-bit message using convolutional coding and selective frequency band modifications. Although carefully engineered, hand-crafted methods often degrade under neu- ral codec compression (Défossez et al., 2023). Deep neural networks (DNNs) have enabled more robust end-to-end watermarking systems that can generalize to unseen trans- formations (San Roman et al., 2024: Chen et al., 2023; Liu et al., 2023a). WavMark (Chen al., 2023) introduced an invertible neural architecture for joint detection and attri- bution with 16-bit synchronization codes. While achieving strong performance, its brute-force decoding and architec- tural constraints limit scalability. AudioSeal (San Roman et al., 2024) addressed these limitations with a generator- detector design with separate detection and message decod- ing. The decoupled design improves detection but reduces
2

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
attribution accuracy. Our work focuses on enabling both robust detection and accurate attribution through more efficient architecture design with a psychoscconsti--nnsprre quality loss.
cal and widely adopted setting in real-world watermarking systems (Liu al., 2024a).
Learning to Watermark with Two-Headed Detector. We first present the general formulation of a neural watermark- ing system with detection and decoding separation. A neural watermarking system is parameterized by = ( OD), where Og denotes the generator parameters and Op denotes the detector parameters. During training, the detection head typically first produces logit for each audio input, which is then passed through sigmoid function to yield a proba- bility p(z) = Di (x) E [0, For a watermarked signal x" and an unwatermarked signal x=', the detection head's objective is to correctly classify both, often formulated by maximizing the following expected log- log-likelihood:
Source A Attribution. A central objective in copyright pro- tection is the ability to trace and verify the origin of creative works, which remains challenging, especially for generative audio systems. Recent efforts have highlighted the necessity of robust source attribution mechanisms that work reliably across different transformations. For instance, Agnew et al. (Agnew et al., 2024) performed an extensive audit of pop- ular audio datasets and revealed serious intellectual property infringements, underscoring the urgency for transparent dataset documentation and reliable authorship checks. In the music domain specifically, Barnett et al. (Barnett et al., 2024) advanced source attribution by leveraging audio em- beddings to identify influential training data in generative music models, enabling more transparent "musical roots" analysis. Such embedding-
max E [log Delet (x.TT) @ Dodet 2...
+
E [log (1 (1- (1-Ddet (x" a²") )]) 2.
where T.. = ((x,, w) and x"' = ...... During inference. the threshold is usually tuned on the validation set to control certain level of false positive rate.
similarity checks align with the broader push for dataset auditing, as reflected in Du et al. (Du al., 2024), who argue for holistic copyright au- diting mechanisms throughout the existing machine learning processes. In this work, we propose a neural watermarking system that advances in message decoding performance, marking an essential step toward robust source attribution.
Meanwhile, the message decoding head produces K prob- abilities Dm (==" = [D"*<<<TT] Dimsss (xx)) k]. where each entry represents the probability that the cor- responding bit is 1. Its training objective is to maximize the probability of correctly predicting each bit of wn, which can be expressed as:
3. Preliminaries
max @ @pmig a,
E
[w log Dmnn (11 (w:) log(1 - DDmsg (x
Audio Watermarking. Audio watermarking systems typ- ically comprise two primary components: generator G to embed watermark, and detector D for recovering it. Let E R² be an audio signal of length T. and let be a binary watermark sequence. The genera- tor G: R.T.' x (0, 1) - R outputs a watermarked signal Tu = G(z,, w). which should ideally preserve the audio's perceptual quality. The detector D is then responsible for two tasks. First, its detection head D : _ + [o, 1] produces probability indicating whether an input contains a valid watermark, which yields final decision of presence or absence with a threshold. Second, its message decoding head Dungg ; IR + [0, 1] returns a vector indicating the probability of each bit being 1, which can be thresholded to reconstruct the embedded message. In practical settings, the watermarked audio may undergo various transforma- tions T() such as compression and cropping, resulting in distorted signal _._ = T(xw). Across different transfor- mations, an ideal detector should be able to robustly detect and decode watermarks. In essence, the detection head should output probabilities close to 1 for watermarked sig- nals (and close to 0 otherwise), while the message decoding head should recover a bit sequence v matching the origi- nal watermark message w. In this paper, we focus on the per-sample level detection and attribution, which is practi-
(I) By thresholding each DImSE (#/)) at 0.5, the decoded bit string ι is obtained, with the learning objective seeking i = w despite the distortion layer 'I applied to Zw.
w E (0.
3.1. Overview of AudioSeal (San Roman et al., 2024)
To contextualize our architectural improvements, we first analyze AudioSeal's 's framework, which pioneered the dis- joint generator-detecto paradigm for neural watermarking. While it achieves strong detection robustness, its attribution limitations motivate key aspects of XATTNMARK.
Disjoint Generator Architecture. AudioSeal employs two separate networks for watermark genera- tion and detection: given audio x, the generator g = { {Egen- Jgen} is composed of an encoder Eyen that encodes the audio into compact latent hz, and a decoder Jgen that decodes the latent into the watermarked perturbation. The audio latent is obtained via h. Egen(x) R² xH where t' [T/a], is the temporal downsampling factor, and H is the latent dimension. The secret message w is in- jected into the latent space with a message encoder M that maps w to its latent h = M(w). The modulated latent waveform is obtained by adding the message latent to the audio latent h (ar w) = h + hyg followed by the decoder
3

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Randomly Sampled
Kiii meckage
Message Embedkeer with Temporal Modulation
Shared Parameter
Louiing
comperent
Medeage Hildin State gennal
Embeedisg Table
K.F.
Pred. Labal
Watermareed
Audio
Input Audio
Mileyl (lll) oo of
@
Message Bits Decoding Head
x
Encoder
Decoder
Encoder
Watermarke
Wittemarky
Watermark Generator
Watermark Detector
Watermarking Detection
Figure 2. System Diagram for XATTNMARK. XATTNMARK consists of a watermark generator and a watermark detector, with a shared embedding table that facilitates message decoding through cross-attention module. In the generator part, first employ an encoder network to encode the audio latent and then apply temporal modulation to hide the message. The modulated latent is then fed into decoder produce the watermark residual. the detector part, lincar detection head used for detecting the presence of watermarks, and a
module with a shared embedding table is used for message decoding.
network Jgen to produce the predicted watermarked per- turbation dw (x w)) E RTx1, which is later applied on the original audio to produce the watermarked audio aw = x + 8w. Note that the decoder model Jgen shares symmetric structure with the encoder Egen. with layers of residual transposed convolution for temporal up- sampling. The watermark detector D (Eaer, Odee) } is composed of an encoder Edet (sharing similar structure with Egen but with additional padding match the original length 7') and a unified decoding head Odec E I (1++k) for both detection and message decoding. Given potential watermarked audio x the encoder Esst first processes the audio to obtain the latent h = (ax")) E T Then the decoding head Odee is applied on the latent obtain the logits for both detection and message decoding, i.e.,
4. Methodology
Despite AudioSeal achieving robust detection, the model struggles perform accurate message decoding even with- out distortion (See Table 7). In this work, following the ar- chitecture backbone of AudioSeal, we identify and resolve two architectural limitations: disjoint generator-detector and the information bottleneck caused by embedding mean- pooling. Specifically, we first propose a cross-attention generator-detector watermarking system with a shared em- bedding table and temporal modulation to improve learning efficiency in message decoding. Then, to further improve watermark quality, we propose a psychoacoustic-alignnn masking & loss. The framework is shown in Figure 2.
TF
dee (h.)) E (I++K)
4.1. Cross- Generator-Detectee Watermarking System with Shared Embedding Table
Multi-bit Message Conditioning M. Similar to the com- mon practice of using learned embeddings for conditional generation (Peebles Xie, 2023), AudioSeal uses an em- bedding table of only 2K entries, E TR2K*H to encode the total 2 possible message states, which significantly reduces the storage complexity. Specifically, each bit wy E (0. 1) for j E (O...., K K-11 1} maps index I; 2j + w/y allowing separate representations for 0/1 states in each bit position with distinct embedding vectors. Given the position sequence p = [II..... Ix], the retrieved K embedding vec- tor sequence is V(w) [E,,... Erg] E KK Then an average pooling operation on V(w) is applied to obtain the final message latent h, = K E Ei, E " The message latent is repeated along the temporal axis and then added to the waveform latent, leading h(x w) = h + hw.
We observe that the fully disjoint architecture of AudioSeal (Og * θD) often converges fast for the watermark detec- tion learning but struggles to learn the message decoding part efficiently and accurately. On the other hand, the full parameter-shard architecture of WavMark (Og θp) can achieve superior efficiency in learning both detection and message decoding but lacks enough robustness capa- bility against various distortions (See App. C.2). This motivates the design of hybrid architecture with partial parameter sharing between the generator and the detector, which facilitates more effective message decoding while preserving robust capability.
Our key design is to share the message conditioning mod- ule M between the generator and the detector, which helps

---

We TEERKX" XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
bridge the information flow between how the message is composed and how it can be reconstructed in the detector part. Specifically, the learnable part of the embedding ta- ble E' in M, which serves as the fundamental vector set for composing the message latent in the generator part, is now utilized as a reference when decomposing the latent to retrieve the message bits the detector. To achieve this, we propose leveraging a cross-attention block (Vaswani al., 2017) to use the embedding table E as (key-value) ref- erence for message decoding, given the query audio latent h.. We describe the decoding mechanism in detail in the following section.
is the sigmoid activation function.
Message Embedding via Temporal Modulation. In AudioSeal, the message latent is obtained by mean- pooling operation with temporal-axis repetition, hw = Repeat (* E Eig:() E R'' This approach can be viewed as injecting the message information mostly into the frequency domain, which greatly limits the message hid- ing capabilities. To improve upon this, knowing the input audio length T, we introduce a temporal message condi- tioning mechanism that employs a linear modulation layer WM E RRK''' to obtain the message latent:
Message Decoding with Cruss-Attention. Ideally, for the given embedding table E with 2K entries and a water- marked audio T. = x + (ax, w)), we want reconstruct the original K' embedding vectors .B.,....: Eiw l. that were used to compose the ground-truth message latent h and then feed them as context for the message decoding. To achieve this, we use an attention mechanism that transforms the embedding table into key and value using two different linear projections, and does attention based vector merging with query from the reconstructed latent representation ht.. Specifically, since the nearby two entries in E represent one position with different bit states, merging them into one 2H- dim vector represents the latent of each position. We trans- form this reshaped embedding matrix E' E R Xx2 into K, V with two linear projections WK. Wv E 2 that is K- = E' and V = Wy. Then, we demodu- late the reconstructed waveform latent along the temporal axis h TRT*H, to obtain the first version of raw predic- tion V. for the original K components. To do this, we first use linear projection W Weem E TRT* K for obtaining the query sequence hdcm = WT h2 E &R Rx*H and follow dem
h WM V(w) E RC*H
(3)
This design avoids relying only on the frequency domain for distributing the K-bit message, but also significantly facilitates the learning process for message decoding (see Figure 6).
4.2. Psychoacoussti--llgneed Time-Frequency Masking
Achieving imperceptibilityy is a key requirement of any wa- termarking system. Ideally, a watermark should exploit the perception characteristics of the human auditory sys- tem so that the artifacts remain imperceptible. Among all the psychoacoustic effects, auditory masking (Gelfand, 2017; Holdsworth et al., 1988) offers strong theoretical guid- ance for designing perceptual losses in watermarking (Deng et al., 2023; Qin et al., 2019). While prior work like Au- dioSeal (San Roman et al., 2024) introduced perceptual losses such as TF-Loudness, these often adopt a coarse ap- proach based on loudness differences within fixed TF tiles. Such methods face two main limitations: (i) they may not fully capture sophisticated auditory masking effects, partic- ularly the interactions between masker and maskee signals across different tiles, and (ii) loudness differences alone can provide weak supervisory signals for guiding the water- mark embedding, resulting in a coarse penalty distribution (visualized in App. C.9 in Figure 12). These limitations motivate our development of more psychoacoussti-aligned TF-masking loss.
with a linear query projection Wo E R Then we can utilize the embedding table E to further refine the final component prediction V. with the following cross- crss-attention mechanism:
Q = h
dem
K= = E' WK E RKXH
V E' Wv ERKXH
QKT
Per-Tile Penalty with Asymmetric Temporal-Frequency Decay. Let So t) denote the magnitude of the orig- inal audio's 's mel-spectrogram, and the watermarked mel- spectrogram is Sw (m, t). To identify those strong "masker" tiles M, we apply a magnitude threshold as to different timestamps on each frequency band (as = 0.8):
A = softmax
RRK
VH
V = acc((A)
E R KXl
where H is the latent dimension, K is the number of mes- sage bits, is the temporal dimension, and act(-) can be any activation function. In this study, units (ELU)) (Clevert et al., 2016). With the predicted V,, we further employ a linear projection layer Wàee E IR x1 with sigmoid activation function for the construction of the message decoding head. The final logit for message predic- tion bit is then obtained as uे (V,, Wace), where (-)
exponential linear
Mmasker = (m,))) So t) > as max So
Each index pair (m te) E Mmasker acts as a masker, and we model its masking influence over neighboring tiles with a linear energy decay in the decibel domain. Specifically,
5

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
given mel-scale frequency radii ri", rg. and time-axis ra- dius r/.. ryy. the local region of masker is defined as:
Per-Tilc-Weighted &g Loss Computation. Psychoacoustic masking effectively applies only when the masker's mask- ing energy surpasses the tile's energy. We therefore apply the masking threshold on watermarked mel-spectrogram at 20 1o Sw (m, 1), filtering the set of masked tiles as:
R(me, te)= = {(m, 1) - S Am S h
r S A S rry
where Am = m - me, and At = t- tc. Aligned with the empirical studies (Necciari et al., 2016) that the post- masking region (forward masking) usually longer than the pre-masking region (backward masking) in the temporal axis, we set asymmetric radii r and ry, with a forward masking region of 200ms and a backward masking region of 20ms. For the frequency-bin radius, we first compute the empirical critical bandwidth for each mel-hin using the formula from Zwicker & Fastl (2013), W (ma) = 25 +
Maaskee = {(m,,) Muss (m, t) > 20 log10 Sw (m, t)}.
To encourage the model to embed the watermark in those maskee regions, we design per-tile penalty terms which weight the & difference (Chen et al., 2023) between the watermarked and original audio in mel-spectrogram space:
Mmn 1) = + 1 (m,t))MMaate Fm (m,,1/200
(5)
2) 0.69
||Sw (m, t) - So (m. t)
where F(me) is the approx- imated center frequency obtained by converting each Mel bin value m. back Hz using F(m,) = 700 (10 -1) 1). Then the radii for each mel-bin are set as r|" rx rg" .%, F(mo)l is an adaptive seal- ing term, V [m.] denote the i-th mel-bin value, rg" is a base radius that is set to be 3. With the dynamic radius scaled by the critical bandwidth- - where higher-frequency compo- nents have wider masking ranges (Holdsworth et al., 1988) we observe a better performance than using constant ?-
75
1 11+1. 1.4
[[(m])) 1000
LIF = Σ
(6)
A(m,t)
(m..)
where 10 (m.,t)))) is the masking energy, and the weight- ing term X(m, 1) is larger for tiles in the masking region Mmsskee. effectively allowing more watermark signal to be embedded in those locations. For non-masked tiles, A(m, t) = 1 enforces standard & loss.
where y = W ((me)/ (m=))[[(mm))
Other Auxiliary Perceptual Loss. Following San Roman et al. (2024), we use & constraint on the watermark sig- nal 6w to ensure waveform-domain smoothness, and we incorporate multi-scale Mel spectrogram loss Lmsspec (e.g.. as in Défossez et al. (2023)) to manage frequency-domain fidelity. Furthermore, we also adopt adversarial losses Ladv on multi-scale STFT spectrograms for perceptual improve- ment, but with lower weight A = (while AudioSeal uses Audv = 4). The reason behind this is that aggressively pushing the watermarked audio distribution to be close to the clean audio distribution in an adversarial sense may penalize desirable watermark characteristies (e.g., certain musical "remixes" can produce satisfactory imperceptible watermarks, but still yield high adversarial loss). We ob- serve that lowering the adversarial weight allows the model to focus more on the psychoacoustic aspects to develop a human-centric, imperceptible watermark.
The physical structure of the cochlea determines that higher frequencies (located at the base of the cochlea) cause broader and stronger excitation patterns compared to the lower frequencies (located at the apex). To inject this bias of slower decay in the upward spread of masking (Zwicker & Fastl, 2013), we set the decaying term for the upward direction to be smaller than the one in the downward di- rection, i.e., a S cr.. For the temporal decay, following Necciari et al. (2016), since the forward decay is slower, we set the forward decaying slope less steep than the backward one, i.e., 3,+t < 3,- Given a masker (Me, fc), the threshold energy Emas for the maskee tile (m, 1) is computed as:
Emask (m, t; (m., tc)) = 20 1o (m tc) - -AEE,, where AE = a max(0) Am) + a, min Am) +3, m0 At) + 3, min At),
5. Experiments
(4)
5.1. Experimental Setup
In psychoacoustic modeling, particularly in audio compres- sion (Bosi et al., 1997), the global masking threshold is determined by considering only the most dominant masker at each frequency component, Following this, the final masking threshold Fima (m, t) is computed as:
Following prior works (San Roman et al., 2024; Chen et al., 2023), we use a sampling rate of 16 kHz and one-second mono samples for training (T 16000) under 16 diverse audio editing transformations. We train the models on a mixed audio dataset of 4100 hours containing speech (3016 hours VoxPopuli (Wang et al., 2021) and 100 hours Lib- riSpeech (Panayotov et al., 2015)), music (9 hours Music- Caps (Agostinelli et al., 2023) and 880 hours Free Music Archive (Defferrard et al., 2016)), and sound effects (98 hours AudioSet (Gemmeke et al., 2017)). For evaluation,
Diuus (m,
max (mexee) E
Emask (m, t; (merte))
This ensures that each maskee tile receives masking energy from its most perceptually dominant masker, while weaker maskers have negligible impact (Gelfand, 2017).
6

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Table 1. The accuracy of detection and attribution across different editing operations. For detection, we also report the True Positive Rate
(TPR) and False Positive Rate (FPR) where the threshold is selected by Youden's 's Index (Youden, 1950) on balanced validation set.
AudiowMark
WayMark
TimbreWM
AudioSeal
XATTNMARK
Edit
Det. (TPR/PRI)
Att.
Det. (IPR/ERR)
Att.
Det. (TFR/FR))
Att.
Det. (TPRAPR)
Att.
Det. (TPR/PRR)
Att.
Identity
1.00 (1.00..0)
1.00
1.00
(10000.00
1.00
0.995
(19980))
0.93
1.00
(s.900.0e)
0.69
0.995
(0.990.0))
1.00
Bandpass
1.00 (1.00.00)
1.00
1.00
(10000.00)
1.00
0.985
0.93
1.00
(0.90.00)
0.31
0.995
(1999000)
0.99
Boost
1.00 (L000.00)
1.00
1.00
(10000.00)
1.00
0.98
(0.60.0))
0.91
1.00 (0.9w0.0)
0.50
0.995
(0.900.0)
1.00
Duck
1,00 (1 (0000))))
1.00
0.995
(1990000
1.00
0.97
(194000)
0.89
1.00 (coww0.r)
0.56
0.995
(0:90000)
1.00
Echo
1.00 (1000000)
1.00
1.00
(1.001000)
1.00
0.945
(188910))
0.91
1.00
(0oww0o))
0.38
0.995
(0990000)
0.99
Highpass
1.00 (1000000)
1.00
0.95
(0.9001.00
1.00
0,985
0.93
1.00
(0.900.0)
0.31
0.995
(0.90000)
1.00
I Lowpass
1.00 (1 .000:00)
1.00
1.00
(10000000)
1.00
0.985
(193800)
0.90
1.00
(0.900.0)
0.56
0.995
(0:900000)
1.00
MP3
0.94 (o.s00)0)))
1.00
0.805
(06580000)
1.00
0.95
(090000)
0.86
1.00
(0.900.0))
0.38
0.995
(0:900000
1.00
Pink Noise
1.00 (1000..0)
1.00
1.00
(10000.00)
1.00
0.995
(0.9m.0))
0.92
1.00
(0.900.00)
0.75
0.995
(0.990.00)
1,00
White Noise
1.00 (1 (1.00.00)
1.00
1.00
(10000.00)
1.00
0.975
(0.50.0c)
0.90
1.00
(0.900.00)
0.56
0.995
(..900.00
1,00
Smooth
1.00 (1 (1000.00)
1.00
0.96
(0.928.00)
0.88
0.98
(a (0)
0.80
1.00
(0.900.00)
0.19
0.995
(0:990.00)
1.00
Speed
0.50 (0.000.00)
0.00
0.50
(0.0000.00
0.00
0,515
0.18
0.61
(0.300.15)
0.00
0.995
(0.990.00)
0.03
Resample
1.00 (1.000.00)
1.00
1.00
(Lo0N).00)
1.00
0.98
(o (0.960.0)))
0.92
1.00
(0.900.00)
0.56
0.995
(0.990.00)
1,00
AAC
1.00 (1 (1:00000))
1.00
1.00
(1 (1890100)
1.00
0,975
(195810)))
0.90
1,00
(ooww.o))
0.12
0.995
(1390000)
0.88
EnCodec (6)=16)
0.50 (0.00000))
0.00
0.805
(0600000))
0.00
0,625
(1500225)
0.07
1,00
(0.90000)
0.31
0.965
(1.530000)
0.99
Crop
0.965 (0:90000))
1.00
0.995
(0:900.000
1,00
0.96
(o (192000)
0.85
1.00
(0/900.0)
0.12
0.975
(0.980.0))
1.00
Average
0.929 (0.89901000)
0.88
0.918
(000000000))
0.87
0.925
(186000019)
0.80
0.971
01950011100
0.39
0,9919
(1995690/0119)
0.93
we use held-out test set from MusicCaps of size 100. For each audio file, we embed 100 distinet messages, resulting in 10k unique watermarked audio samples. Each of these samples is then subjected to 16 different audio transforma- tions, leading to a total of 160k evaluated instances. The audio duration is set as 5s by default in evaluation. The loss weights are set as: ÀTF = 1, Aadv 1, de, 0.1, Amessage 10. We use the Adam optimizer (Kingma& & Ba, 2015) with learning rate 1e-5, 31 = 0.4, 32- = 0.9, and Exponential Moving Average (EMA) (Tarvainen & Valpola, 2017) with decay factor of 0.99 updated at every step. We train for 73k steps with batch size 16 and latent size H = 32. To boost the sampling of the transformation efficiency, we update the sampling probability of each transformation every 1000 steps on the validation set, adjusting it based on the validation accuracy of each transformation.
5.2. Detection and Attribution Effectiveness
Robustness to Standard Edits. We present the detection and attribution performance across a comprehensive suite of standard audio editing operations in Table 1. For detec- tion, we report both the overall accuracy and the true/false positive rates (IPR/FPR). For attribution, we report the aver- aged performance across different user numbers (see Figure for the decomposed result). Our method achieves new state-of-the-an performance both tasks, maintaining high detection accuracy (99.19% average) and attribution accu- racy (93%) on average across transformations. Notably, while AudioSeal achieves strong detection performance (97. 1% average). it struggles to perform effective attribution (39% average). The traditional approach, AudiowMark. ex- hibits more balanced cnrccciinnaatriuttion trade-offs (around 88% attribution) but lower overall detection performance (92.9% average). Under the speed-change transformation, our method achieves 99.5% detection accuracy while all other methods degrade to near random-gues: levels (50- 1)). While attribution remains challenging under speed change, we show that the attribution performance can be partially restored via a speed-reversion layer (see App. C.3.1 for details). Figure further validates that XATTNMARK consistently maintains high attribution accuracy even with increasing message pool sizes. Furthermore, we conduct statistical tests to validate the statistical significance of the improvement in attribution across different transformations (App. 14 and App. C.3 for more details). In summary, we demonstrate that XATTNMARK is more robust under standard edits for both detection and attribution.
Amsspec- = 2, Aeetcct
For the attribution experiment, we follow the simulation protocol in San Roman et al. (2024) that defines a message pool of size N (E {100, 1000, 100003), where each mes- sage is uniquely associated with a different user. As N increases, the message length (in bits) also grows, making attribution more challenging due to the increased complex- ity of distinguishing individual messages. During decoding, the message is retrieved from the pool by selecting the one with the closest Hamming distance. We define attribution accuracy as the fraction of correctly attributed messages among all audio inputs that were positively detected as wa- termarked. We compare to four state-of-the-ar baselines: AudiowMark (Westerfeld, 2020), WavMark (Chen et al., 2023), TimbreWM (Liu et al., 2023a), AudioSeal (San Ro- man et al., 2024). Please refer to App. A for more details.
Robustness to Generative Edits. Beyond the standard audio transformations that are seen during training, audio generative editing is one particularly challenging transfor-
7

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
White box
Semi black box
Gaussian noise
Table 2. The detection performance (Accuracy (TPR/FRR) under
1.0
two generative edits applied with different editing strengths t.
Detection Error Rate
System
Audiow Mark
WarMark
TimbreWM
AndioSral
XATTNMARK
0.8
Stuble
70
0.50 possoo))
(
0.50 (100)))
0.50 ( (wwo)))
0.47 (oo0s))
0.59 (1rs0 s)
0.99 (s)in
0.91 (1sssin)
Andio
110
0.50
0.50 ( (o003))
0.53 (1)0.9
0.99
29
0.91 (ssasas)
0.6
LDM2
Audio
70
10
0.50 (ussust)
0.50 pussust))
0.50 (t (0wor))
0.50 (wwor))
swont)
0.50
0.53 (u
0.50 ())
0.59 (1942299
0.59 (1ssa)))
m)
0.94 (1s44ti)))
0.94 (1548100)
0.4
0.2
Table 3. Detection performance and audio quality metrics after
0.0
HSJA-based (black-box) adversarial attacks on the waveform and spectrogram domains on AudioMarkBench dataset (Liu et al.,
3
2
PESQ ( (+ stronger attacks)
2024a).
Difl. Setup
AndioScal
XATINMARK
Figure 3. Watermark-removal attacks under three different knowl- edge settings. PESQ is measured between attacked audios and genuine ones (PESQ < 4 indicates strong quality degradation).
Acc.
PESO
SISNR
VISQOL
Acc.
PESQ
SISNK
VISQOL
Wavefoom
0.15
1.14
8.97
2.61
0.68
2.80
12.79
334
Spectrogram
0.15
1.05
-17.82
2.45
0.26
1.56
-24.88
2,37
#Q=100
0.15
1.14
8.97
2.61
0.68
2.80
17.79
3.34
#Q-2000
0.15
1.33
2.64
0.57
2.30
6.93
291
4Q=500
0.15
1.13
8.72
2,63
0.47
1,97
1.66
2.55
also significantly degrade audio quality in the process. In the waveform domain, increasing the query budget from Q = 100 Q 500 further reduces detection accuracy from 0.68 to 0.47. However, across all experimental configu- rations, XATTNMARK consistently outperforms AudioSeal in terms of both detection robustness and perceptual qual- ity preservation. Aligning with San Roman et al. (2024), we further evaluate XATTNMAR ARK against white-box, semi- black-box (using a re-trained surrogate model with the same architecture but different initialization), and Gaussian noise attacks, with results shown in Figure 3. The results reveal that model vulnerability increases with attacker knowledge, highlighting the importance of keeping detector models pro- prietary to mitigate watermark removal threats.
mation that watermarking systems might have to endure at deployment (Liu et al., 2024b). To simulate this, we use two state-of-the-art audio generative models, AudioLDM2 (Liu et al., 2024b) and Stable Audio (Evans et al., 2024), with a text-guided DDIM inversion method proposed in ZETA (Manor& & Michaeli, 2024). We test various editing strengths L {10, 70, 110), which represent the diffusion forward step when using DDIM inversion. As shown in Table 2, AudiowMark and WavMark degrade to a random- guess level performance, while TiMbreWM and AudioSeal show inferior performance across different editing strengths, around 50-60%. In contrast, our method maintains consis- tently high detection accuracy, averaging 91-94% across all editing strengths, and is consistent across both generative models. While XAttMMark demonstrates strong detection robustness (e.g., >9% accuracy) against generative editing attacks in a zero-shot manner, achieving similarly robust attribution performance under such strong edits remains a challenge and an important direction for future work. Nev- ertheless, our detection performance marks significant advancement. This demonstrates that ARR gener- alizes better unseen generative edits compared to existing methods. Please refer to the App. C.4 for more details.
Extending to Watermark Localization. We further ex- tend XATTNMARK to perform watermark localization fol- lowing the Brute Force Detection (BFD) method used in WavMark (Chen al., 2023). This is achieved by employ- ing sliding window detection mechanism. Given that our model operates on 1-second audio segments and incorpo- rates transformations robust to time shifts, we can distribute the per-segment detection probability to a finer, per-frame level by utilizing multiple overlapping detection windows. As shown in Figure 4, XATTNMARK achieves localization performance comparable to AudioSeal and significantly out- performs WavMark, in terms of both sample-level F1 score and Intersection Over Union (IoU) metrics. Furthermore, we also demonstrate the effectiveness in the case where the duration of the watermark segment is fixed at 1 second, and the total audio length varies from 2 seconds to 10 seconds. Please refer to the App. C.10 for more results.
Robustness to Adversarial Watermark Removal. Recent work by Liu et al. (2024a) demonstrates that adversarial per- turbations can be crafted to make watermarks undetectable, even with only black-box access to the detection model. To evaluate robustness against such attacks, we employ the blo HopSkipJumpAAttack (HSJA) (Chen et al., 2020), which iteratively crafts minimal perturbations in either the waveform or spectrogram domain to fool the detector. As shown in Table 3, we first evaluate HSJA attacks in both domains with a query budget of Q = 100. While the at- tacks successfully reduce detection accuracy, the attacks
5.3. Quality and Stealthiness Assessment
We evaluate the following objective quality metrics of the watermarked audio: Scale Invariant Signal to Noise
8

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
1.0
5.4. Ablation Study
Sample-level
We first conduct ablation on the cross-attention module and temporal modulation layer to showcase their effective- for boosting learning in message decoding, As shown in Figure 6, our approach demonstrates superior learning efficiency, reaching approximately 98% accuracy. When removing the modulation component, performance drops significantly to around 60%. More dramatically, ablating the cross-attention mechanism causes accuracy to drop to random-guess levels (50%), highlighting its crucial role in the architecture. Our ablation studies highlight the contribu- tions of key architectural components. Notably, the partial parameter sharing enabled by the cross-attention mechanism is particularly impactful for improving message decoding efficiency and accuracy. While the temporal conditioning module also contributes, the ability of the detector to directly attend to shared message embeddings via cross-attention appears be more dominant factor in achieving robust at- tribution compared to ablations without it. Furthermore, we ablate the proposed TF loss LTF and the adaptive bandwidth with constant weight y = 1. As shown in Table 4, both of them contribute to perceptual quality.
0.5
Ours
AudioSeal
Wavmark
0.0
1.0F
0.5
3
seconds from 10s audio are watermarked
Figure 4. Watermark localization results across different dura- tions of watermarked windows in terms of sample-level FI score Intersection Over Union (IoU) metrics (t is better).
Table 4. Audio Quality Metries. Comparison of objective percep- tual quality metrics across watermarking methods.
Methods
SI-SNR
PESQ
STOL 1
ViSQOL
L(6..)
AndiowMark
27.88
4,59
0.988
4.72
44.96
WavMark
36.35
4.43
0.985
4.62
-33.01
Altribution Accurac Srruney (99)
Messsaaa art Accurccy (N6)
Timbre WM
26.45
4.29
0.974
4.63
43.69
AudioSeal
25.32
4.51
XATINMARK
29.00
4.43
1.000
4.56
-54.63
w/o Eyp
19.64
4,25
0.990
4.26
-52.63
70-
w/o Adaptive y
21.89
4.22
0.991
4.24
-54.30
22
80
100
1020
10000
20
40
60
100
Number of Users (N)
Training Steps (k)
Ratio (SI-SNR), as well as PESQ (Rix et al., 2001),
Figure 5. Atrriuution accuracy with different #Users.
Figure Ablation study on the
ViSQOL (Hines al., 2012) and STOI (Taal al., 2010). Furthermore, also report the loudness of watermark residual L(8w) according to ITU-R 17.0770-4 (Interna- tional Telecommunication Union, 201 1), which is an impor- tant metric for watermarking stealthiness, especially under residual- based detection (Yang et al., 2024). As shown in Table 4, XATTNMARK achieves competitive performance across all perceptual quality metrics. While WavMark achieves the highest SI-SNR (36.35 dB), XTTNNMARK excels in perceptual quality measures, achieving the best STOI score (1.000) and the lowest watermark residual loud- ness (-54.63 LUFS). For the PESQ and ViSQOL scores, XATTNMARK achieves competitive performance, with a PESQ score 4.43 and a ViSQOL score of 4.56. We also conducted subjective listening test using the MUSHRA protocol (International Telecommunication Union, 2014) with 12 participants, and the results suggest that XATTN- MARK achieves comparable quality with AudioSeal scor- ing around 91 (while ground truth scores around 95, more details in the App. C.1 and App. C.9). In summary, XAT- TNMARK achieves superior utility while maintaining com- petitive perceptual quality.
proposed architecture.
6. Conclusion
In this paper, we propose a neural audio watermarking framework that achieves both reliable watermark detec- tion and accurate message attribution under challenging transformations, including generative editing. By integrat- ing partial parameter sharing, a cross-attention-basee de- tector, and a temporal conditioning module, our approach improves robustness and decoding accuracy compared to existing neural watermarking methods. Additionally, the psychoacoustic-aligne TF masking loss enables quality- preserving watermarking. To knowledge, this is the first method to demonstrate non-trivial watermark detection under generative editing, marking a significant step toward reliable content attribution in generative Al settings. Future work includes developing stronger watermarking for robust attribution under speeding and various generative editing, and leveraging spatial psychoacoustics improve the per- ceptual quality and support spatial audio watermarking.

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Acknowledgments
Tagliasacchi, M., et al. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023.
This work was supported in part by Dolby Laboratories Inc. and Lehigh University. In this work, Yixin Liu and Lichao Sun were partially supported by the National Sci- ence Foundation Grants CRII-2246067, HTTD-2427915, NSF POSE-2346158, and Lehigh Grant FRGS0001 1497. This work is also supported by the Delta/DeltaAl systems at NCSA and the Bridges-2 system at PSC through allocation CIS240308 from the Advanced Cyberiffrastructure Coordi- nation Ecosystem: Services Support (ACCESS) program, supported by National Science Foundation grants 2138259, 2138286, 2138307, 2137603, and 2138296. We also thank Leslie Famularo from Dolby Laboratories Inc. for helpful discussions and the anonymous reviewers for their construc- tive feedback.
Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F. M., and Weber, Common voice: A muasively-multiinguual speech corpus. In LREC, 2020.
Barnett, J., Garcia, H. F., and Pardo, B. Exploring musical roots: Applying audio embeddings to empower influence attribution for a generative music model. arXiv preprint 2401.14542, 2024. URL http:httpa//xiie../e// wwg.aaiwiii.r....oo.ooooooo pdf// 2401.14542v1.
Bilika, D., Michopoulou, N., Alepis, E., and Patsakis, C. Hello me, meet the real me: Audio deepfake attacks on voice assistants. arXiv preprint 23302.0328, 2023. URL /2302. 10328v1.
Impact Statement
Bosi, M., Brandenburg, K., Quackenbush, S., Fielder, L., Akagiri, K., Fuchs, H., and Dietz, M. Iso/iec mpeg-2 advanced audio coding. Journal of the Audio engineering society, 45(10):789 814, 1997.
This research addresses pressing concerns regarding media authenticity and intellectual property protection stemming from the rise of generative AI. Our work introduces an im- proved audio watermarking system. The system's 's value lies in its ability to reliably detect embedded identifying marks (watermarks) and accurately trace their origin, even when audio has been substantially altered by sophisticated AI tools. This capability makes our technology valuable asset in combating deepfake audio and the spread of misin- formation. Furthermore, it offers enhanced protection for the rights of audio content creators.
Brigham, N. G., Wei, M., Kohno, T., and Redmiles, E. M. "violation of my body:" perceptions of ai- generated non-consensual (intimate) imagery. arXiv preprini 2406.05520, 2024. URL http://arxis org/pdf/2406 05520v2.
Buo, S. A. The emerging threats of deepfake attacks and countermeassrees. arXiv preprint 2012.07989, 2020. URL 2011////////////.////ooo 2012. 07989v1
Like many technologies, advanced watermarking holds the potential for dual use. Although designed to protect and authenticate, such systems could potentially be misused by malicious actors or institutions for purposes such as unautho- rized tracking or censorship if implemented irresponsibly. Recognizing these risks, we strongly advocate for trans- parency and informed user consent in the deployment of such systems. The development of clear ethical guidelines and robust policies crucial to prevent abuse.
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P., and Sun, L. A survey of ai-generated content (aige). ACM Computing Surveys, 57(5):1338, 2025.
Chen, G., Wu, Y., Liu, S., Liu, T., Du, X., and Wei, F. Wavmark: Watermarking for audio generation. arXiv preprint arXiv:2308. 12770, 2023.
Chen, J., Jordan, M. I., and Wainwright, M. J. Hop- skipjumpattack: A query-efficient decision-based attack. In S&P, 2020.
References
Abbott, R. and Rothman, E. Disrupting creativity: Copy- right law in the age of generative artificial intelligence. Fla. L. Rev, 75: 141, 2023.
Cho, W. Artists score major win in copyright case against ai art generators. The Hollywood Reporter, August 2024.
Clevert, D.-A., Unterthiner, T., and Hochreiter, S. Fast and accurate deep network learning by exponential lin- ear units (elus). In Proceedings of the 4th International Conference on Learning Representations, 2016.
Agnew, W., Barnett, J., Chu, A., Hong, R., Feffer, M., Netzorg. R., Jiang, H. H., Awumey, E., and Das, S. Sound check: Auditing audio datasets. arXiv preprint 2410.13114, 2024. URL http: htap:://aaxiia /ttpxxxxivvv////////eee/eaaaa.la wogg/pf// 9410.13114v4
Copet, J., Kreuk, F., Gat, I., Remez, T., Kant, D., Synnaeve, G., Adi, Y., and Défossez, A. Simple and controllable music generation. Advances in Neural Information Pro- cessing Systems, 36, 2024.
Agostinelli, A., Denk, I., Borsos, Z., Engel, J., Verzetti, M., Caillon, A., Huang, Q., Jansen A., Roberts, A.,
10

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Cui, Y., Ren, J., Xu, H., He, P., Liu, H., Sun, L., Xing, Y., and Tang, J. Diffusionshield: A watermark for copy- right protection against generative diffusion models, 2024. URL https:://// / arxiv. org/ abs/2306. 04 042.
Linguistics. doi: 10.8655ww/222.fmiingsseemm// 541. URL http:///aacaaancooooo.o.o.o eeriinggs-en1p...0
Hines, A., Skoglund, J., Kokaram, A., and Harte, N. Visqol:
Defferrard, M., Benzi, K., Vandergheynst, P., and Bresson, X. Fma: A dataset for music analysis. arXiv preprint 1r1iv.16/2.018444 2016.
The virtual speech quality objective listener. In /WAENC 2012; International Workshop Acoustic Signal En- hancement, pp. 1-4. VDE, 2012.
Deng, J., Teng, F., Chen, Y., Chen, X., Wang. Z., and Xu, W. V-cloak: Intelligibilityy. naturalness- & timbre-preserving real-time voice anonymization. In Proceedings of the 32nd USENIX Security Symposium (USENIX Security 23), pp. 5181-5198, Anaheim, CA, USA, August 2023. USENIX Association. ISBN 978- URL
Holdsworth, J., Nimmo-Smith, I., Patterson, R., and Rice, P. Implementing a gammatone filter bank. Annex C of the SVOS Final Report: Part A: The Auditory Filterbank, 1: 1-5, 1988.
Hu, Y., Ma, M., Lu, W. Xiong, N. N., and Wei, J. Se- lection of the optimal embedding positions of digital au- dio watermarking in wavelet domain. arXiv preprint 2010.11461, 2020. URL hitp: /awxiviivvv////////i///..////.aa wwwoor/pplloo 2010. 114 v1vvv.
1-9391333-7-33.
Desai, D. and Riedl, M. Between copyright and com- puter science: The law and ethics of generative ai. arXiv preprint 2403.14653. 2024. URL http://arxiv. org/pdf/ 2403.14633v2.
International Telecommunication Union. Algorithms to measure audio programme loudness and true-peak audio level. Technical Report 1S.1770-4, ITU-R, 2011.
International Telecommunication Union Method for the subjective assessment of intermediate quality level of audio systems (MUSHRA). Technical Report BS.1534-3, ITU-R, 2014.
Du, L., Zhou, X., Chen, M., Zhang, C., Su, Z., Cheng, P.. Chen, J., and Zhang, Z. Sok: Dataset copyright auditing in machine learning systems. arXiv preprint 2410.16618, 2024. URL http: http///xxivvoroe wwt..orggggoooooo 2410. 16618v1.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015.
Défossez, A., Copet, J., Synnaeve, G., and Adi, High fidelity neural audio compression. Transactions on Ma- chine Learning Research, 2023, 2023. ISSN 2835-8856. Publisher Copyright: ② 2023, Transactions Machine Learning Research. All rights reserved.
Li, C., Wang, R., Liu, L., Du, J., Sun, Y., Guo, Z., Zhang, Z., and Jiang, Y. Quality-aware masked diffusion trans- former for enhanced music generation. arXiv preprint arXiv:2405. 15863, 2024.
Evans, Z., Parker, J. D., Carr, C. J., Zukowski, Z., Tay- lor, J., and Pons, J. Stable audio open. arXiv preprint arXiv:2407. 143588.,444 2024.
Liu, C., Zhang, J., Zhang, T., Yang, X., Zhang, W., and Yu, N. Detecting voice cloning attacks via timbre wa- termarking. arXiv preprint 2312.03410, 2023a. URL http: wwwpxxxivv..oggpgoo paf 2312 03410v1.
Gelfand, S. A. Hearing: An introduction to psychological and physiological acoustics. CRC Press, 2017.
Liu, H., Guo, M., Jiang, Z., Wang, L., and Gong, N. Z. Au- diomarkbench: Benchmarking robustness of audio water- marking. In NeurIPS, 2024a. Datasets and Benchmarks Track.
Gemmeke, J. F. Ellis, D. P., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., and Ritter, M. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 776-780. IEEE, 2017.
Liu, H., Yuan, Y., Liu, X., Mei, X., Kong, Q. Tian, Q., Wang, Y., Wang, W., Wang, Y., and Plumbley, M. D. Audioldm 2: Learning holistic audio generation with self supervsde pretraining. IEEF/ACM Transactions on Audio, Speech, and Language Processing. 2024b.
Guan, B., Wan, Y., Bi, Z.. Wang, Z., Zhang, H., Zhou, P., and Sun, L. Codeip: A grammar-guide multi- bit watermark for large language models of code. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Findings of the Association for Computational Linguis- tics: EMNLP 2024, pp. 9243-9258, Miami, Florida, USA, November 2024. Association for Computational
Liu, X., Wang, X., Sahidullah, M., Patino, J., Delgado, H., Kinnunen, T., Todisco, M., Yamagishi, J., Evans, N., Nautsch, A., et al. Asvspoof 2021: Towards spoofed and deepfake speech detection the wild. IEEF/ACM

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Trunsactions on Audio, Speech, and Language Process- ing, 2023b.
Qin, J., Lyu, S., Deng, J., Liang, X., Xiang, S., and Chen, H. A lattice-based embedding method for reversible audio watermarking. IEEE Trans. Dependable Secure Comput., 21(4):261-2633) 2024. doi: 10.1109/TDSC. 2023.3314612.
Liu, Y., Fan, C.. Dai, Y., Chen, X., Zhou, P., and Sun, L. Metacloak: Preventing unauthorized subject-driven text- to-image ciffusion-basee synthesis via meta-learning, In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 24219-24228, 2024c.
Qin, Y., Carlini, N., Cottrell, G., Goodfellow, I., and Raffel, C. Imperceptible, robust, and targeted adversarial exam- ples for automatic speech recognition. In Proceedings of the 36th International Conference on Machine Learn- ing. volume 97 of Proceedings of Machine Learning Re- search, pp. 5231-5240 Long Beach, CA, USA, 9-15 Jun 2019, PMLR. URL ht https: /ttpp/eeeocceggeseeeeeeeeeeess.e .mlr. press/v97 http////////////// 19tmml1.......
Liu, Y., Hu, H., Chen, X., Zhang. X., and Sun, L. Water- marking text data on large language models for dataset abs/
copyright, 2024d. URL https: 2305. 13257.
Liu, Y., Xu, K., Chen, X., and Sun, I.. Stable unlearn- able example: Enhancing the robustness of unlearnable examples via stable mroor-mnn noise. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38. pp. 3783-3791, 2024c.
Qiwei, L., Zhang, S., Kasper, A. T., Ashkinaze, J., Eaton, A. A., Schoenebeck, S., and Gilbert, E. Reporting non-consensual intimate media: An audit study of deep- fakes. arXiv preprint 2409. /2/38, 2024. URL http: / faaxiiv.org wwt///////////////////ee/eeee/t/ 2t0.///2.9..1.8.....
Manor, H. and Michaeli, T. Zero-shot unsupervised and text- based audio editing using ddpm inversion. arXiv preprint 10009, 2024. URL http: /ttp//aaiivv//..////ii///////aaa org/ / 2402. 100.200000 10009v4.
Rafi, Z., Liutkus, A., Stöter, F.-R., Mimilakis, I., and Bittner, R. The MUSDBI8 corpus for music separa- tion, December 2017. URL attps://dol. org/10. 5281 1281/2enodo. 1117372.
Meerza, I. A., Sun, L., and Liu, J. Harmonycloak: Making music unlearnable for generative ai. In Proceedings of the IEEE Symposium on Security and Privucy, pp. 430-448 IEEE, 2025.
Ren, J., Xu, H., He, P., Cui, Y., Zeng. S., Zhang, J., Wen, H., Ding, J., Huang, P., Lyu, L., Liu, H., Chang, Y., and Tang, J. Copyright protection in generative ai: A technical perspective. arXiv preprint 2402.02333, 2024. URL http://arxiv.. org/ http//df/222// 02333v2.
Necciari, T., Laback, B., Savel, S., Ystad, S., Balazs, P., Meunier, S., and Coonland-Martinee R. Auditory time- frequeneyy masking for spectrally and temporally maximally-compe stimuli. PloS one, 11 1e11))6016693, nov 2016.
Rix, A. W., Beerends, G., Hollier, M. P., and Hekstra, A. P. Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs. In 2001 IEEE International Confer- ence on Acoustics, Speech, and Signal Processing. Pro- ceedings (Cat. No. 01CH37221), volume 2, pp. 749-759 IEEE, 2001.
Office, U. C. Copyright and artificial intelligence: Part digital replicas report, 2023. Accessed: 2025-01-23.
OpenAI. Sora: Creating video from text. https://
openai www.cooo//oooo/. sor, 2024.
Pan, Y., Pan, I.., Chen, W., Nakov, P., Kan, M.-Y., and Wang. W. Y. On the risk of misinformation pollution with large language models. arXiv preprint 1366, 2023. URL http:/ / arxii. org/pdf/////oooo...o///pppp./p// 2305 13661v2.
Robinson, K. Will umg's lawsuit against believe usher in new era of regulation for diy distribution? Bill- board, November 2024. URL https: / //tinurrl...r com/2b fon/2b7v7yrr.
Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. Librispeech: an asr corpus based on public domain audio books. In ICASSP, 2015.
San Roman, R., Fernandez, P., Elsahar, H., Défossez, A., Furon. T., and Tran, T. Proactive detection of voice cloning with localized watermarking. In International Conference on Machine Learning, volume 235, 2024.
Park, P. S., Goldstein, S.. O'Gara, A., Chen, M., and Hendrycks, D. Ai deception: A survey of examples, risks, and potential solutions. arXiv preprint 2308. 14752, 2023. URL http: http/rxxiiiie//ee///eee/eeeaeelt
/tlL:tttp///axxv..o/g//// wttp://w...pppppppeeeee 2308 14752v1.
Shoaib, M. R., Wang, Z., Ahvanooey, M. T., and Zhao, J. Deepfakes, misinformation, and disinformation in the era of frontier ai, generative ai, and large ai mod- els. arXiv preprint 2311.17394, 2023. URL http: / //arxiv.os www.ii.org/pd////ee /
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
12

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Singer, N. Teen girls confront epidemic of deepfake nudes in schools. The New York Times, April 2024. nytimes. deepfake-ai -ideaii--d--weeneee west weresffeeldee wew.tffiel--iiih-shhhhee html. Retrieved April 8, 2024.
Yang. P., Ci, H., Song, Y., and Shou, M. Z. Can simple av- eraging defeat modern watermarks? In The Thirty- Annual Conference Neural Information Processing Systems, 2024.
URL https://www
Youden, W. Index for rating diagnostic tests. Cancer, (1):32-35, 1950.
Taal, C. H., Hendriks, R. C., Heusdens, R., and Jensen, J. A short-time objective intelligibillit measure for time- frequency weighted noisy speech. In 2010 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing, pp. 4214 4217. IEEE, 2010.
Zhang. H. A time-frequency perspective on audio wa- termarking. arXiv preprint 2002.03/56, 2020. URL hhtgg/dff/// 2002. 03156v1.
Zhang, J., Peng, X., and Zhang, S. Robust audio water- marking algorithm based on moving average and DCT. Proc. Ini. Conf. Artificial Intelligence and Security (ICAIS), volume 11635 of Lecture Notes in Computer Science, pp. 419-429. Springer, 2019.
Tarvainen, A. and Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems, volume 30, pp. 1195-1204, 2017.
Zhou, C., Li, Q. Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q. He, L.,, Peng, H., Li, J., Wu, J., Liu, Z.. Xie. P., Xiong. C.,, Pei, J., Yu, S., and Sun. L. A comprehensive survey on pretrained foundation models: history from BERT to ChatGPT. International Journal of Machine Learning and Cybernetics, 2024. doi: 10. 1007/513344-0044-2444666 E-pub ahead of print (24 Nov 2024).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Atten- tion is all you need. In Advances in Neural Information Processing Systems, volume 30. pp. 5998-59999999 0008,60080200 2017.
Verma, P., Tenjarla, R., and Sand, B. Ai is spawning a flood of fake trump and harris voices. here's 's how to tell what's real. The Washington Post, October 2024. Retrieved 6:05 a.m.
Zwicker, E. and Fastl, H. Psychoacousties: Facts and Mod- els. Springer Science & Business Media, mar 2013.
Vermillio. Wme announces first major ai partnership with authenticated ai company vermillio to protect artists and create new revenue opportunities. Vermillio Blog, January 2024.
Wang, C., Riviere, M., Lee, A., Wu, A., Talnikar, C.,, Haziza, D. Williamson, M., Pino, J., and Dupoux, E. Voxpopuli: A large-scale multilingual speech corpus for representa- tion learning, semi-supervisee learning and interpretation. In Proceedings of the 59th Annual Meeting of the Associ- ation Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Processing, pp. 993-1003, 2021.
Wenger, E., Bronckers, M., Cianfarani, C., Cryan, J., Sha, A., Zheng. H., and Zhao, B. "hello, it's me": Deep learning-based speech synthesis attacks in the real world. arXiv preprint 2109.09598, 2021. URL
Westerfeld, S. audiowmark: Audio watermarking. https: //github.. com/ wwwsterfelddeeeeeeeee///eee ownar,, 2020. Accessed: 2025-01-02.
Wikipedia contributors. Content authenticity initia- tive Wikipedia, The Free Encyclopedia, 2025. URL https: :/en wikipedia org/wiki/
Content iutheniiittyInniiaaiieee. Initiative. Ac- cessed June 2025.
13

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
A. Implementation Details
Model Architecture. Following AudioSeal (San Roman et al., 2024), we leverage pair of convolutional encoder-decoder models that operate on the waveform domain to generate and detect watermarks. The watermark generator consists of a waveform encoder and decoder, both utilizing components from EnCodec (Défossez al., 2023). The encoder employs a ID convolution containing 32 channels and a kemee size of 7, then uses four convolutional blocks. Each of these blocks contains residual unit and a down-sampling layer, utilizing convolution with stride S and kernel size K = 2S. The residual unit contains two kernel-3 convolutions with a sipp-oonecttio, with channels doubling during down-sampling. The encoder ends with a two-layer LSTM and final ID convolution having a kernel size of 7 and 128 channels. The stride S values used are (2, 4, 5, 8), and residual units utilize the Exponential Linear Unit (ELU) as a nonlinear activation. The decoder reflects the encoder's s structure but employs transposed convolutions, with strides in the opposite order.
The detector contains an encoder, transposed convolution, and linear layer for the detection head. The encoder utilizes the same structure as the one from the generator, but with different weights. The transposed convolution includes H output channels and upscales the activation map to match the original audio dimension, yielding an activation map shaped (T, H). This frame-level latent state serves both the detection head and the message decoding head. For the detection head, the linear layer serves to reduce the dimensionality of H to two, followed by a softmax function that produces per-sample detection probability scores. To obtain the sample-level detection probability from frame-level logits, we apply temporal averaging (mean pooling) across the time dimension. Additional details on the message decoding head are provided in the next section.
Cress-Allention Message Decoding Head. The message decoding head aims decode the embedded message bits by leveraging the shared embedding table from the generator. The key idea is to have "message vocabulary" to look up when decoding the message bits. To this end, we employ a single-head cross- -ots-nttention layer with the message latent state as a query and the units in the embedding table as key-value pairs. Note that the reconstructed waveform latent is first passed through a fully connected layer to decompose the time dimension and obtain a K-ddimensional message latent state as a query. For the embedding table part, we reshape the embedding table to a 2.ll-immesioonl vector by combining the two vicinal units that correspond to the same message bit into one 2H -dimensionnl vector. All the linear projections Q, K, V are designed to map the latent dimension H dimensions. After the cross-attention operation, we apply an ELU activation function followed by final linear projection W E #
1 to obtain the final per-bit message logits for the logistic regression.
Training and Inference Details. During the training stage, we employ a curriculum learning method to first disable the gradient-basee based quality balancing term, guiding the model to focus more on learning watermark decoding at the beginning of training. After 59000 training steps, we enable the quality balancing term to help the model improve the watermark auditory transparency with more dynamic scaling across different perceptual loss terms. To boost the sampling of the transformation efficiency, we update the sampling probability of each transformation every 1000 steps on the validation set, adjusting based on the validation accuracy of each transformation. In detail, the sampling ratio Py for transformation g is given by
2000 + e, where accg is the accuracy of the model on transformation g and e is a small constant for keeping the Pg 1-28,
sampling ratio non-zero. We observe that it boosts the learning efficiency in attribution with at least 2x improvement under the main setup. The training checkpoint at 73000 is selected based on its balanced performance between quality and utility scores on the validation set. During the inference stage, for given audio with arbitrary duration, we first pad and split the input audio into multiple Is audio segments and then apply the watermarking model on each chunk. Audio chunks are then concatenated to obtain the final watermarked audio.
Perceptual Loss. Following AudioSeal (San Roman et al., 2024), we employ pool of perceptual loss terms to guide the model to learn the watermarking task. The perceptual loss terms include: i) multi-scale mel-spectrogram loss Emaspec that computes both L1 loss on linear-scale mel-sppectrograms and MSE loss log-scale mel-spectrograms across multiple FFT window sizes (from 2 to 2)1,,, with each scale weighted by 2 - I for scale i; iii feature matching loss Lieat that minimizes L1 distance between intermediate feature maps extracted from the adversarial discriminator, averaged across all layers. iii) waveform 1.1 loss Lwaveform that minimizes 1.1 distance between the original waveform and the watermarked waveform; iv) our proposed psychoacoustic-aligee alliggg TF masking loss Lausk, which captures the weighted lz loss in the mel-spectrograms domain, using a per-tile weight obtained through simulation of the masker energy decaying in the time-frequency domain.
For the adversarial discriminator, we adopt a multi-scale STFT discriminator architecture that operates on the complex STFT representations of the audio at different scales. Specifically, it consists of multiple sub-discriminators, each processing the STFT with different FFT sizes (512, 1024, 2048), hop lengths (128, 256, 512), and window sizes (512, 1024, 2048). Each -ub-discriminttor first computes the STFT, concatenates the real and imaginary components along the channel dimension,
14

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
and processes them through series of 2D convolutions with increasing dilation rates (1,2.4). The feature maps from these convolutional layers are used for the feature matching loss. These losses jointly optimize for perceptual quality by matching both time-frequency characteristics and learned audio representations.
Evaluation Setup. For the attribution experiment, following San Roman et al. (2024), we set up a pool of N potential users, where each user's sequential bit message is derived from their ordinal number, i.e., the i-th user's message is given by W;- = binary(i). This creates a space of all potential messages Ew. The attribution is achieved by retrieving the message in Ew with the smallest Hamming distance. Compared wit- message accuracy, this metric provides a more practical measure of message retrieval performance, aligning better with real-world watermarking applications and attribution use cases. The message length (in bits) determines the maximum number of uniquely identifiable users. Specifically, for message of K bits, the system can support up to 2 unique users. In our experiments, we use 16-bit message, enabling attribution across 216 = 65,536 users at maximum.
A.1. Robustness Augmentations
Following AudioSeal (San Roman et al., 2024), we apply these audio editing augmentations during training and evaluation:
Bandpass Filter: Simulates frequency- selecctive audio equipment by allowing only mid-range frequencies. Allows
frequencies between 300H2-8000Hz to pass through.
Highpass Filter: Removes bass frequencies to simulate poor bass response. Cuts frequencies below 500Hz.
Lowpass Filter: Removes high frequencies simulate muffled audio. Cuts frequencies above 5000Hz.
Speed: Simulates playback speed variations. Changes speed by random factor 0.8-1.2.
Resample: Robustness to sample rate conversion. Upsamples to 32kHz, then downsamples back to the original rate.
Boost Audio: Simulates volume increase. Amplifies by factor 1.2.
Duck Audio: Simulates volume decrease. Reduces volume by a factor of 0.8.
Echo: Simulates room acoustics and reverberation. Adds delayed copy with 0.1-0.5s delay and 0.1-0.5 volume.
Pink Noise: Adds realistic environmental noise. Adds pink noise with std 0.01.
White Noise: Adds Gaussian noise with standard deviation 0.001.
Smooth: Simulates low-quality audio processing. Moving average filter with window size 2-10.
AAC: Robustness to common lossy compression. AAC encoding at 128kbps.
MP3: Robustness to common lossy compression. MP3 encoding at 128kbps.
EnCodec: Tests neural audio codec compression. Resamples to 24kHz, encodes with 16 streams (ng = 16), resamples to 16kHz.
Crop: Robustness to audio truncation, padding, and in-batch audio mixing. While San Roman et al. (2024) uses this to obtain localization mask, we implement crop as one of the edits to gain cropping- based robustness. Specifically, we first randomly select & starting points, then modify T/2k consecutive samples in one of four ways:
Revert to original audio (40% probability)
Replace with zeros (20% probability)
Substitute with different audio from the same batch (20% probability)
Leave unmodified (20% probability)
-
A.2. Dataset Details
We utilize a mixed dataset of 4100 hours in total for training, which contains 100.59-hour LibriSpeech (Panayotov et al., 2015), 98.53-hour AudioSet (Gemmeke et al., 2017), 879.29-hour Free Music Archive (Defferrard et al., 2016), 9-hour MusicCaps (Agostinelli et al., 2023), and 3,016.43-hour VoxPopuli (Wang et al., 2021). Each dataset is described as follows:
LibriSpeech. LibriSpeech (Panayotov al., 2015) an English speech dataset derived from audiobooks in the Libri project. The dataset contains approximately 1000 hours of read English speech sampled 16 kHz, with careful segmentation and alignment. The audio is paired with transcribed text, making it suitable for speech recognition tasks. We used 100.59-hour subset of the full 1000-hour dataset for training.
AudioSet. AudioSet (Gemmeke et al., 2017) is a large-scale dataset containing 2,084,320 human-labeled 10-second sound clips drawn from YouTube videos. It consists of 632 audio event classes organized in a hierarchical ontology, covering a wide range of sounds, including human and animal sounds, musical instruments, genres, and common environmental sounds. The dataset contains approximately 5,790 hours of annotated audio. For our experiments, we used 98.53-hour randomly sampled subset of the full dataset for training.
15

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Free Music Archive. FMA (Defferrard al., 2016) is large-scale, open-source dataset of music tracks with clear licensing. We used the "large" subset containing 879.29 hours of audio data with 106,574 30-second tracks sampled at 44.1 kHz. The dataset spans multiple genres, including Rock, Electronic, Experimental, Hip-Hop, Folk, and Instrumental, making it suitable for training robust watermarking models across diverse musical styles.
MusicCaps. MusicCaps (Agostinelli et al., 2023) is a dataset of 5.5k music-text pairs (9.47 hours total), with high-quality numtnewritten captions describing musical attributes like genre, mood, instruments, and tempo. The dataset was created by having music experts write detailed captions for a subset of 5.5k music clips from AudioSet, with each caption carefully describing the musical content, instrumentation, style, and other sonic characteristics. We used subset of hours for training.
VoxPopuli. VoxPopuli (Wang et al., 2021) is large-scale multilingual speech corpus collected from European Parliament event recordings between 2009-2020. It contains approximately 400K hours of unlabeled speech data across 23 languages, 1.8K hours of transcribed speech for 16 languages, and 17.3K hours of speech-to-speeds interpretation data. For English specifically, it provides 24. IK hours of unlabeled speech data and 543 hours of transcribed speech from 1,313 speakers. The dataset has been widely used for representation learning, semi-supervised learning, and interpretation tasks, We used a 4.-16,43 subset that covers balanced speech from 23 languages for training.
B. Discussion on Limitations
While our method demonstrates strong performance across various standard audio transformations, several limitations and opportunities for improvement remain. First, although we achieve robust performance against most common audio edits, our attribution accuracy degrades significantly for challenging operations like speed changes and generative editing. Specifically, as shown in Table 7, XATTNMARK achieves only 4% average attribution accuracy for speed changes across different user setups, though this still represents improvement over existing baselines, which fail completely. To enhance robustness against speed variations, one promising direction is to incorporate temporal adjustment layer similar to Westerfeld (2020), which performs bbx optimization to identify and reverse speed changes. We present preliminary work in this direction in App. 1. Additionally, the current solution's 's attribution robustness against Al-based editing remains an area requiring further improvement.
C. More Results
C.1. Subjective Evaluation with MUSHRA
The MUSHRA protocol is a crowdsourced test in which participants rate the quality of various samples on scale of 0 to 100. The ground truth is provided for reference. We utilized eight randomly sampled speech and music samples from different sources, each lasting 5 seconds. As part of the study, we included a low anchor, which is a low-pass-filee pass-filtered version at 3.5kHz. Each sample was evaluated by 16 participants, with a post-hoe filtering process to exclude participants who failed to correctly identify more than two hidden reference samples in their scoring, resulting in 12 valid participants at the end. For comparison, the ground truth samples received an average score of around 95, while the low anchor's average score was around 25. As shown in Figure our method achieved a score of approximately 91, closely matching AudioSeal. Among all the methods, AudiowMark is the one that achieves the highest score at around 94. WavMark received a slightly lower score of 88, while TimbreWM scored the lowest, around 80. These results demonstrate that our watermarking approach maintains perceptual quality that is competitive with state-of-the methods.
C.2. Analysis on the Training Dynamics of Models with Different Architectures
We analyze the training dynamics of different architectures under a controlled experimental setup to better understand their inherent learning capabilities. To isolate architectural effects, we remove most auxiliary losses, retaining only the adversarial loss (weight=1) for all methods. Additionally, we include a waveform-domain & loss (weight=1) for AudioSeal and our method, as we observed improves the learning efficiency. Figure 8 illustrates the validation accuracy trajectories and quality for both watermark detection and message-bit decoding during training. The results reveal distinct learning characteristics that highlight the trade-offs between different architectural choices.
WavMark's sulle-shared architecture exhibits remarkably rapid initial learning, converging high accuracy within just a few hundred steps. However, this early success is followed by concerning degradation in detection accuracy as training
16

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Fabanih
IedrootM
2:66
15
Figure 7. MUSHRA subjective listening test results comparing perceptual quality across different watermarking methods. Higher scores indicate better audio quality as rated by human listeners. Our method achieves quality scores comparable to AudioSeal.
progresses. This pattern suggests that while full parameter sharing enables quick initial learning, it may lead destructive interference between the detection and the decoding tasks during extended training.
AudioSeal's 's fully-disjoint architecture in general shows slow convergence in both detection and message-bit decoding. After training for 30k steps, it quickly converges to 99% detection accuracy but the learning of message-bit decoding progresses slowly, achieving around 70% message bit accuracy at the end of 50k training steps. Moreover, the perceptual quality of AudioSeal is also degraded more than XATTNMARK and WavMark.
Compared these, our blended architecture demonstrates steady and comprehensive learning both detection and decoding accuracies improve gradually, ultimately achieving near-perfect performance only with about 19k training steps. Notably, with only 2k training steps, XATTNMARK can converge to 99% detection accuracy. This great boost in learning efficiency suggests that our partial parameter-sharing strategy and cross-attention mechanism enable more efficient learning of both tasks.
WavMark
Ours
AudioSeal
Audia Quality
3.0
1.0
4.5
0.9
0.5
4.0
SudiSSes
Accuracy
0.0
Detectio Message
Accuracy
Derection Message
Accuracy
Detectioo Nessage
3.0
3.5
2.5
2.8
0.6
0.5
1.5
1.0
10003
20000
03000 50000
0
38000 20003
10030
40303
50n00
0
10300
20005
30000
40000
50003
10030 20303
30000 503co sacon
Training Steps
Training Steps
Training Steps
Training Steps
Figure 8. Validation accuracy and quality curve of different methods over training steps. Using blended architecture, XATTNMARK able to achieve a hetter balance in terms of learning efficiency in detection, and message-bit decoding and watermark imperceptibilityy Compared to our method, WavMark, the sulle-shared architecture, suffers from rohustness degradation in detection the training progresses; the fully-disjoint architecture, AudioSeal, suffers from training efficiency issues in learning message- bit decoding.
C.3. Robustness Standard Edits
We present the full results summarizing detection and attribution robustness against standard edits in Tables and 7. For detection, method achieves consistently high performance across all transformations, with average accuracy of 99.19% and AUC of 99.93%, significantly outperforming all baselines. Even for challenging transformations like speed changes and neural codec compression (EnCodec) where other methods mostly fail (accuracy around 50-60%), our approach maintains exceptional detection performance with accuracies of 99.5% and 96.5%, respectively.
For attribution, the results in Table 7 show that our method maintains high attribution accuracy (92-94% average) across different user pool numbers (and associated messages), demonstrating strong scalability. The performance is particularly
17

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
robust for most standard audio transformations like bandpass filtering, boost/duck, echo, and noise addition, achieving near 100% attribution accuracy. While speed changes remain challenging, method still achieves better performance than baselines which completely fail. This indicates that while our method handles most common audio edits exceptionally well, speed changes continue to be challenging transformation for reliable attribution. We also explore potential simple extension of speed reversion to further boost the performance App. C.3.1.
Our approach shows particularly strong robustness against modern audio codecs maintaining near-perfect attribution accuracy for both MP3 and AAC compression (98-100%) while baselines like AudioSeal struggle significantly with these transformations (showing accuracies below 60%). This demonstrates our method's effectiveness against slate-of-the-a audio compression techniques that are increasingly common in real-world applications. Additionally, our method shows outstanding robustness to cropping operations, maintaining 100% attribution accuracy, while AudioSeal struggles in this task (12% accuracy).
To further clarify our attribution evaluation, particularly regarding the metrics used, we also report the False Attribution Rate (FAR) and compare different definitions of attribution accuracy, aligning with methodologies in AudioSeal (San Roman et al., 2024). In our work, attribution accuracy is defined as the fraction of correctly attributed samples among those successfully detected as watermarked. This metric equivalent adopt this definition because it allows for decoupled analysis of pure attribution performance from the initial detection performance. In AudioSeal, the attribution accuracy is defined as the fraction of correctly altributed samples among all samples, which couples the performance of both detection and attribution. We present the results on both the MusicCaps and PoxPopuli datasets in Table 5, illustrating performance under these specific metrics. The results suggest that in the speech domain, when the user pool size is relatively small, AudioSeal is slightly better than XATTNMARK, and in all the other setups, XATTNMARK consistently outperforms AudioSeal and WavMark.
FAR (False Attribution Rate, reported in AudioSeal). We
Table Attribution results edits MusicCaps (our main setup) and VoxPopuli (align with AudioSeal's setup in San Roman al. (2024)). We report the false attribution rate (FAR) and two attribution accuracy metrics (Att. and Att.+Att.). The key difference between our definition of attribution accuracy and AudioScal's is that isolate the attribution performance from the detection performance. The best score is in bold, and the runner-up is underlined.
Dataset
Metric
Method
N=10
N=100
N=1000
N=10000
WavMark
14.67
14.92
15.18
16.07
FAR (%)
AudioSeal
18.77
25.76
36.63
30.53
XAttmMark
6.40
6.13
6.44
6.56
WavMark
85.33
85.08
84.82
83.93
MusicCaps
Acc. (AIL) (%)
AudioSeal
81.23
74.24
63.37
69.47
XAttnMark
93.60
93.87
93.56
93.44
WavMark
79.69
80.56
79.94
79.06
Acc. (Det.+Att) (8)
AudioSeal
80.81
73.56
62.69
68.75
XAtmMark
90.50
87.19
90.75
91.81
WavMark
13.11
13,21
13.96
15.00
FAR (%) &
AudioSeal
3.71
12.86
18.91
16.31
XAttMark
5.03
6.01
6.25
7.70
WavMark
86.89
86.79
86.04
85.00
VoxPopuli
Acc. (AIL) (
AudioSeal
96.29
87.14
81.09
83.69
XAttnMark
94.97
93.99
93.75
92.30
WavMark
78.69
78.34
78.06
76.88
Acc. (Det.Att.) (%)
AudioSeal
89.50
86.60
79.83
83.45
XAttMark
92.45
92.69
92.49
91.08
C. SIMPLE SPEED REVERSION LAYER WITH BLACK- BOX GRID SEARCH
We explore a host-hoc simple speed reversion layer with a black-box grid search to further boost the robustness against speed changes. The speed reversion layer is added on top of the watermarking model. The main idea reverse the speed
18

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Detection Accuracy
Attribution Accuracy
Gaussian Noise
Pink Noise
MP3 Compression
1.0
Accuracy
Accuracy
0.6
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
10
10
10
10
10
10-1
10
12
24:32
128
256
Standard Deviation
Standard Deviation
Bitrate (kbps)
AAC Compression
Up-Down Resampling
Highpass Filter
1.0
1.0
1.0
0.8
0.8
0.8
Accuracy
Accuracy
0.6
0.6
0.6
0.4
0.4
0.4
0.2
24
32
128
0.4
1000
2000
3000
4000
Bitrate (kbps)
Intermediate Frequency (Heles
Cutoff Frequency (Hz)
Lowpass Filter
Speed Change
Volume Boast
1.0
Accuracy
Accuracy
Accuracy
0.2
0.0
0.0
500
1000
1500
2000
2500
0.5
1.0
1.5
2.0
20
60
60
Cutoff Frequency (Hz)
Speed Factor
Boost Amount (%)
Volume Reduction
EnCodec
Echo
1.0
1.0
1.0
0.8
0.8
0.8
Accuracy
Accuracy
Accuracy
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
Reduction Amount (%)
Codebook Size
Duration (s)
Figure 9, Detection and Attribution Accuracy of our method augmented samples across different augmentation strengths.
19

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Table The full detection results of XATTNMARK across different standard audio edits on the MusicCaps dataset. Ace. (TPR/FPR) the accuracy (and TPR/FPR) obtained for the optimal threshold from Youden's 's Index (Youden, 1950) on a balanced validation set.
MudiowMark
WayMark
TM
AudioSeal
XATTNMARK
Edit
Acc. (TPR/EER)
AUC
Acc. (TPRAPPR)
AUC
Acc.
(TPR/PPR)
AUC
Acc.
(TPR/PPR)
AUC
Acc. (TPR/FR))
AUC
Identity
1.00 (1nom))
1.00
1.00
(10001))
1.00
0.995 (sussnac)
1.00
1.00
0.995
Bandpass
1,00 (100000)
1.00
1.00
(1oon0c)
1.00
0.985
(10t100))
1,00
1.00
(c.00.00)
1.00
0.995
(0:90000)
1.00
Brost
1.00 (1m00)))
1.00
1.00 (1ooo)
1.00
0.98
(1owar))
0,9998
1.00
(e)
1.00
0.995
(19wnor)
1.00
Duck
1.00 (umomn)
1.00
0.995 (c000)n0)
0,995
0.97
(194n10)
0,9999
1.00
0.995
1.00
Echo
1,00 (1m00))
1.00
1.00
(1:00000)
1.00
0.945
(1usonod)
0,9965
1.00
(c:00)
1.00
0.995
1.00
Highpass
1.00 (1000)))
1.00
0.95
(1.90806)
0.95
0,985
1.00
1.00
(0.000.00)
1,00
0.995
(199000)
1.00
Lowpass
1,00 (1000.00)
1.00
1,00
(1.000000)
1.00
0.985
0,9986
1,00
(6.000.00 o0)
1.00
0.995
(1.59000)
1,00
MP3
0.94 (0.s00m))
0.94
0
0.805
0.95
(191800))
0,9971
1.00
(c:000)0))
1.00
0.995
(10000)))
1.00
Pink Noise
1.00 (1oponn
1.00
1,00
(1.00006))
1.00
0.995
1.00
1.00
(c:900000)
1,00
0.995
(19980))
1,00
White Noise
1.00 (1000m)
1.00
1.00
(1:00000)
1.00
0.975 (105800d)
0,9999
1.00
(c.9000))
1.00
0.995
1,00
Smooth
1.00 (1000)))
1.00
0.96
0.98 (0.60.00)
0.9976
(6:00.000
0.995
(10960)
1.00
Speed
0.50 (1000.00))
0.50
0,50
(1.00000))
0,50
0,515
0,4812
0.61
(6:300
0.61
0.995
(1.9000))
1,00
Resample
1.00 (1000m))
1.00
1.00
(1 (1oonoo)
1.00
0.98
(1000000)
1.00
1.00
1.00
0.995
1.00
AAC
1.00 (1spom)
(10000ac
1.00
0.975
0,9999
(c000100)
0.995
(100000)
1.00
EnCidee (6q=16)
0.50 (suzomn)
0.50
0,805
0.805
0.625
(1000023)
0,7092
1.00
1.00
0.965
0.991
Crop
0.965 (e.son.co)
0.965
0.995
0.995
0.96
(0/22000)
0.9918
1.00
(1.900.0))
1.00
0.975
0.997
Average
0.9294 (cassssntoe)
0.9294
0,9181 (1xseomrco))
0.9181
0.925 (180999t110))
0,9482
0.9706 (co0st01)
0.9757
0.9919
0.9993
Table 7. The full attribution results for all methods, and associated overall average performance, across all transformations the CasicCaps datasct.
Transformation
MudiowMark
WavMark
TimbreWM
AudioScal
XATTNMARK
N=100
1000
10000
Avg
100
1000
10000
Avg
100
1000
10000
Avg
100
1000
10000
Avg
100
1000
10000
Avg
Identity
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.96
0.93
0.85
0.93
0.75
0.50
0.50
0.59
1.00
1.00
1.00
1.00
EnCodee
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.03
0.00
0.00
0.07
0.50
0.00
0.00
0.31
1.00
0.99
0.99
0.99
AAC
1.00
1.00
1.00
1.00
1,00
1.00
1.00
1,00
0.94
0.90
0.85
0.90
0.50
0.00
0.00
0.12
0.99
0.96
0.69
0.88
Randpass
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.92
0.92
0.89
0.93
0.50
0.25
0.00
0.31
0.99
0.99
0.99
0.99
Boost
1.00
1.00
1,00
1.00
1.00
1.00
1.00
1.00
0.91
0.91
0.87
0,91
0.25
0.50
0.25
0.50
1.00
1.00
1.00
1.00
Duck
1.00
1.00
1.00
1.00
1,00
1.00
1.00
1.00
0.93
0.85
0.84
0.89
0.50
0.50
0.25
0.56
1.00
1.00
1.00
1.00
Eebo
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.92
0.92
0.87
0.91
0.25
0.25
0.25
0.38
1.00
0.98
1.00
0.99
Highpass
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.96
0.92
0.87
0.93
0.50
0.25
0.00
0.31
1.00
1.00
1.00
1.00
Lowpass
1.00
1,00
1.00
1.00
1,00
1.00
0.93
0.86
0.85
0.90
0.50
0.50
0.50
0.56
1.00
1.00
1.00
1.00
MP3
1.00
1.00
1.00
1.00
1,00
1.00
1.00
1,00
0.85
0.85
0.86
0.86
0.50
0.50
0.00
0.38
1.00
1.00
1.00
1.00
Pink Noise
1.00
1.00
1.00
1.00
1.00
0.93
0.91
0.87
0.92
0.75
0.75
1.00
L.00
1.00
1.00
White Noise
1.00
1,00
1.00
1.00
1.00
1.00
1.00
1,00
0.88
0.89
0.87
0.90
0.50
0.75
0.50
0.56
1.00
1.00
1.00
1.00
Smooth
1.00
1.00
1.00
1.00
1.00
1.00
0.75
0.88
0.75
0.86
0.69
0.80
0.00
0.25
0.25
0.19
1.00
1.00
0.99
1.00
Speed
0.00
0.00
0,00
0.00
0.00
0.00
0.00
0.00
0.20
0.04
0.19
0,18
0.00
0.00
0.00
0.00
0.00
0.00
0.08
0.03
Resample
1.00
1.00
1.00
1.00
1,00
1.00
1.00
1.00
0.93
0.92
0.87
0.92
0.75
0.25
0.25
0.56
1.00
1.00
1.00
1.00
Crop
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.93
0.76
0.80
0.85
0.00
0.00
0.00
0.12
1.00
0.99
1.00
1.00
Avg
0.88
0.88
0.88
0.58
0.88
0.88
0.86
0.87
0.81
0.78
0.75
0.80
0.42
0.33
0.22
0.39
0.94
0.93
0.92
0.93
change by searching for the optimal speed factor with the guidance of the watermarking model. We observe that on the unwatermarked audio, changing the speed can not adversarially increase the detection accuracy, while on the watermarked audio, speed changing will significantly affect the watermark message decoding pattern. We observe that the watermarking model predicts more stable message bits for each I-second chunk when the speed remains unchanged. However, after applying a speed change, the model predicts more diverse message bits with lower confidence for each chunk. To leverage this insight, given the original speed change space [7mmn 7maxx and the subsequent search space "Yimax" Yumi we design the score function r(a) as the average of the watermarking model's mean detection score over each chunk p(a:. and the average standard deviation of the predicted message bits {m(a). Formally, this is expressed as r(a) = + Σ (p:(a) + (p.(a)+$m.((a)) (a)), where L is the number of chunks, a is the speed factor, p:(a)) is the detection score for chunk i, and (m. is the average standard deviation of the predicted message bits for chunk i. Then we conduct a two-round linear grid search to find the optimal speed factor a that maximizes the score function r(a). with first-round step size S₁ and second-round step size S2. The speed change parameter 7min and 7max are set to 0.8 and 1.25, respectively. We conduct experiments on both watermarked and unwatermarked audio to investigate whether speed reversion might cause false positives. For the watermarked case, we report the attribution success rate/detection time, and for the unwatermarked case, we report the FPR. As shown by the results in Table 8, with a more fine- grained search space, the speed reversion layer can further improve the robustness against speed changes while maintaining the overall detection efficiency. Meanwhile, for unwatermarked audio, applying the speed reversion layer does not falsely increase the probability of being classified as watermarked by the detection model.
20

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Table Comparison of speed reversion layer performance. We report both the attrihution success and the average detection time (in seconds) for different user pool sizes N. The speed reversion layer significantly improves robustness against speed changes while maintaining efficient detection time.
w/o Search
w/ Search (51 = 0.1, S2 = 0.01)
w/ Search (S) = 0.01, S2 = 0.001)
#User
Success Rate
Det. Time (s)
Success Rate
Det. Time (s)
FPR
Success Rate
Det. Time (s)
FPR
N
0.391
0.005
0.537
0.220
0.01
1.000
0.653
0.00
N = 10
0.069
0.006
0.237
0.229
0.00
1.000
0,583
0.01
N 100
0.222
0.006
0.612
0.230
0.00
0.990
0.545
0.00
N = 1000
0.000
0.007
0.163
0.230
0.00
1.000
0.497
0.00
C.4. Robustness to Generative Edits
We use the DDIM-inversion-bae
editing approach of ZETA (Text-Based Audio Editing) introduced in Manor & Michaeli (2024) to manipulate the audio with text prompts. We briefly summarize the ZETA editing process in the next section. For more details on this editing process, please refer to Manor & Michaeli (2024).
ZETA. Given the original signal To (waveform or latent space), this approach first conduct guided forward process Jt = Vā + vI- Ō,Et; t = 1,..., ,, where ãt is the cumulative variance schedule, and é, is the independent Gaussian noise. Then a sequence of noise vectors that capture the main characteristics of the source audio extracted using 2, (zt--1 14r (a+)) /o,, - T. ....... 1, where (4(()) is denoising neural network trained predict the mean component of the noise added at time t. Note that souree prompt Pare also be used at the conditioning layer of the denoising neural network to guide the forward diffusion process. During the editing process, given target prompt Ptgt and a noisy state IT, a guided diffusion backward process is conducted via ø;-1 Ae (æx) + oz t = T..... 1. where the noise vector sequence z, are the ones extracted from the previous forward process. The target prompt Pig is embedded and applicd on the cross-attention part that affects the editing trajectory.
We present the full results of robustness against generative edits in Table 9, where we evaluate our method against two state- of-the-art audio generation models: Stable Audio (Evans al., 2024) and Audiol.DME (Liu et al., 2024b). Different forward diffuuion steps T' are used to create varying strengths of audio manipulation, ranging from 10 to 110. The source prompt Pare is set to "A recording of music", and the target prompt Ptg is set to "A recording of EDM music with strong rhythm". We follow the default settings in the original paper for other hyperparameters.
For Stable Audio, our method achieves consistently high accuracy (0.81-0.94) and AUC scores (0.89-0.99) across different T values. In contrast, AudioSeal shows more variable performance: while achieves decent accuracy (0.69-0.78) for lower T values (10-30), its performance degrades significantly T increases, with accuracy dropping to around 0.59 and AUC falling to around 0.50-0.53 for higher T values. For AudioLDM2, approach maintains strong performance with an accuracy of around 0.94 and an AUC above 0.93 across all diffusion steps. AudioSeal again struggles with higher T values, showing consistent accuracy around 0.59 but poor AUC scores (0.33-0.53).
Furthermore, we also compare against Timbre Watermark (Liu et al., 2023a), a frequency-based watermarking method that has demonstrated state-of-ther the-art robustness against voice cloning training pipelines. While this approach could potentially handle generative edits, which also modify audio significantly, our experiments show that it performs poorly on this task. As shown Table 9, Timbre Watermark achieves only random-guess level performance (accuracy around 0.50) for both Stable Audio and AudioLDM2 across different T values, significantly underperforming both AudioSeal and our method.
C.5. Generalization Across Standard Editing with Different Strengths
We present the generalization of XATTNMARK against transformations of different strengths in Figure 9, Our method demonstrates remarkable robustness across wide spectrum of audio transformations with varying intensities. For lossy compression operations, EnCodec shows detection accuracy of 69.5-95.099 5.0% and attribution rates of 70.0-100% depending on bitrate. AAC maintains strong detection (97.5%) across all bitrates with attribution ranging from 53.0-98.055 MP3 compression achieves consistent detection (97.5%) and perfect attribution (100%) across all bitrates.
For frequency-domain operations, our watermark remains highly effective. When filtering out low frequencies (highpass filtering,, we maintain excellent performance with 99.0-99.55 99.5% detection accuracy and 99.0- 100% attribution accuracy, even
21

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Table We demonstrate that our watermarking approach can generalize better to the unseen, challenging generative edits setting. Detection Results for two generative edits applied with different forward t steps.
AudioSeal
Timbre Watermark
XATTNMARK
Editing Model
T
Acc. (TPREPR)
AUC
Acc. (TPR/ERR)
AUC
Acc. (TPR/FPR)
AUC
10
0.6875 (0,6250025)
0.7227
0.4688 (0.812500875)
0.4219
0.9375 (0:9375000055)
0.9883
30
0.7813 87500.1225)
0.8125
0.4063
(0 (0.562500775)
0.3789
0,8125 (0.7500.225)
0.8906
Stable Audio
0.5938 (0.875KL75)
0.3398
0.4688
(0.062540.125)
0.4863
0,9063
(0.8750.0025)
0,9688
70
0.5938 (0.75805625)
0,5313
0.5313 (0.187500125)
0.5547
0,9063 (0.87500.025)
0.9648
90
0.5938
61877)06675)
0,5469
0,5000
(0500000.5000)
0.5996
0,9063
(0.87590.0625)
0,9688
110
0.5938 (0.9575K175)
0,5039
0.5313
(0.562500.5000)
0,5488
0,9063
(0.8750.0625)
0.9570
10
0.5938 (0.957500775)
0.5273
0,5313 (0.812500755
0.4961
0.9375
(0.957500.0625)
0,9492
0.5938
(0.937590775)
0.3945
0.4375 (0.00.125)
0.4570
0.9375
(0.93750000255)
0,9414
AMdioL.MM2-Music
50
0.56 (0.88A075)
0.34
0,5313
(0.562500.000)
0,5645
0,9063
(0.8750000255)
0.9336
70
0.5938
(0.95550077)
0.3477
0,5000
(0.1250.125)
0.5703
0.9375
(0.93750.0625)
0.9453
0.5938 (0.95750775)
0,3945
0,5000
(0.50009.50000
0.5723
0.9375
(0.937500.0625))
0,9414
110
0.5938 (0.95750775)
0.3359
0,5000 (0.00.0)
0.5137
0.9375
(0.9375000025))
0.9414
as we vary the cutoff frequency from 500Hz to 4000Hz. Similarly, when filtering out high frequencies (lowpass filtering with cutoffs between 500-2500Hz), we achieve 97.5% detection accuracy while maintaining 100% attribution accuracy. For volume changes, our method handles both increases and reductions well maintaining 97.5% detection and 100% attribution for volume boosts up 90%, and only showing slight degradation (85.0% detection) when reducing volume to 10% of original while still maintaining 100% attribution.
For additive noise, both Gaussian and pink noise maintain 97.5% detection accuracy up to 0.1 standard deviations, though attribution degrades from 100% at low noise levels (50.01) to 0% at high intensities (20.5). Speed changes achieve 94.5-97.5% detection but, as expected, only maintain attribution at normal speed (1.0x). For resampling operations across frequencies from 16kHz to 1024kHz, detection remains strong (97.5%) with perfect attribution (100%). Echo effects across different delay times (0.1-0.5) consistently achieve 97.5% detection and 100% attribution, demonstrating the watermark's comprehensive resilience to common audio transformations.
C.6. Generalization Across Different Datasets
To study how XATTNMARK generalizes across different domains and datasets, we conduct in-distribution and out-of- distribution tests on multiple audio datasets. We randomly sample 100 audio clips with a 5s duration from each dataset. The user size N is set to 1000.
For in-distribution evaluation (Table 10), we test on FMA-Large (music), LibriSpeech (speech), VoxPopuli-10K (speech), and AudioSet (general audio). Our method achieves consistently high detection accuracy (96-99%) and attribution accuracy (87-93%) across all datasets and editing operations. The only exception is speed change, where attribution fails as expected since it fundamentally alters the temporal structure that watermarks rely on. We also observe slightly lower performance on EnCodec compression for FMA-Large and AudioSet, where detection accuracy drops to 72-81% 81% and attribution accuracy 70-97%, reflecting the challenge of handling highly compressed audio. Notably, the average detection accuracy across all datasets remains strong at 96-99%, with attribution accuracy ranging from 87-93%
generalization (Table 11), we evaluate on four unseen datasets: BudioMarkBench (Liu et al., 2024a) (which is the speech subset from Common Voice (Ardila et al., 2020)), ASVspoof (speech) (Liu et al., 2023b), MusicGen (AI- generated music) (Copet et al., 2024), and MUSDB 18 (music) (Rañi et al., 2017). Our method maintains robust performance on these challenging OOD datasets, with detection accuracy ranging from 93-99% and attribution accuracy from 86-94%. The speech datasets (AndioMarkBBecch and ASVspoof) show slightly lower but still strong detection rates of 93-97%, while music datasets (MusicGen and MUSDBI3 18) achieve near-perfect detection at 98-99%. Attribution performance remains consistently high across all OOD datasets, except for speed changes, which consistently fail as expected. Notably, the average attribution accuracy for OOD datasets ranges from 86-94%, demonstrating the robustness of our watermarking approach across diverse audio domains.
For
These results highlight the strong generalization capabilities of our method, both in-distribution and out-of-distribution. The
22

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Attribution
vs
Attribution vs Strength
vs Strength
0.593
Attributton nccuracy
00.922
Accuracy
0.6
Detection Accuracy
0.30
0.9
0.6
0.88
0.86
0.4
0.44
0.2
0.6
0.986
0.0
Audio Duration (seconds)
Audio Duration (seconds)
Watermarking
Watermarking Strength
Figure 10. The detection and attribution accuracy of XATTNMARK across different audio duration and watermark strength on MusicCaps. The performance is averaged over all the standard audio editing. The attribution pool size set {100, 1000, 10000}.
consistent performance across datasets and editing operations, with only expected failures (e.g., speed changes and extreme compression), underscores the practical applicability of our watermarking system in real-world scenarios where diverse audio content and editing operations are encountered.
C.7. Performance across the different audio duration and watermark strength a
To investigate the performance of our method across different audio durations and watermark strength o, we conduct experiments with audio duration ranging from Is to 10s and watermark strength a ranging from 0.0 to 1.5. As shown in Figure 10, we analyze the impact of these parameters on detection and attribution accuracy.
For duration analysis, we observe that the detection accuracy remains robust (98.6 (98.6-99.3%) across all tested durations from 1-10s, demonstrating our method's effecctiveness even with short audio clips. Attribution accuracy shows greater sensitivity, improving from 81.2% at Is peak of 93.0% 5s, then stabilizing 91.5-92.6%% for longer durations. This suggests 5s segments provide the optimal trade-off between practical usability and attribution performance, though reliable detection (>98.5%) is achievable with clips as short as ls.
For watermark strength analysis, we observe three distinct operational regimes: i) weak watermarks (a < 0.3), ii) moderate watermarks (0.4 < a I 0.7), and iii) strong watermarks (a 2 0.8); achieve asymptotic performance with detection 998.3% and attribution 92.3%, showing diminishing returns beyond a 1.0.
Our default a= 1.0 configuration achieves 98.5% detection and 92.7% attribution accuracy. While higher a values (up to 1.5) marginally improve detection to 98.9%, attribution plateaus 93.1-93.2%, suggesting upper bound for useful watermark strength. Notably, the complete absence of watermarking (a = 0) yields chance-level detection (55.7%) and zero attribution capability, confirming the system's 's dependence on the embedded signal.
C.8. Watermark Residual Visualization
To better understand the characteristics of different watermarking methods, we visualize the watermark residuals in Figure 11. The visualization includes waveform, spectrogram, and mel-spectrogram representations for the original audio and four watermarking approaches (XATTNMARK, AudioSeal, WavMark, and AudiowMark). Our analysis reveals several key insights: (1) Our method introduces watermarks with the lowest energy, making them less perceptible in both time and frequency domains. (2) The spectral patterns of our watermarks appear more stealthy and natural compared to other methods. (3) Unlike WavMark, which shows discrete interruptions, our watermarking pattern exhibits better continuity, contributing to enhanced robustness against audio transformations while preserving imperceptibilitt.
C.9. Visualization of TF-wighhted Penalty Distribution.
Comparison with TF-loudness loss in AudioSeal. AudioSeal adopts a coarse approach implement auditory masking, using naive loudness difference between the noise and the original audio. The main idea is that high loudness regions should mask lower loudness regions in the watermark residual. A positive penalty is assigned when the residual loudness is higher than that of the original audio for each tile.
Specifically, the waveform is directly divided into x W regions, where each region shares the same weighting term. Here, B represents the number of frequency bands for splitting, and W denotes the number of time tiles. The loudness difference between the watermark residual and the original waveform is then computed as 1 = Loudness( Loudness
23

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Table 10. The performance of detection and attribution of our method on different in-distribution datasets over different editing operations.
Dataset
-MgeLarge
I LibriSpeech
VoxPopuli-
AudioSet
Edit
Det. (TPR/PPR)
Att.
Det. (TPR/EPR)
Att.
Det. (TPR/FPR)
Att.
Det. (TPR/FFR)
Att.
Identity
0.995 (0.990000000)
1.000
0.995
(0.990/0.000)
1.000
0.997
(0.994/0.000)
1.000
0.990
(0.98000.000)
0.970
Bandpass
0.995 (0.990/0.000)
0,980
0.995 (0.9900.0000
1.000
0.997 (0.9940.000)
1.000
0.995
(0.990/0.000)
0,990
Boost
0.995 (0.99000.000)
0.990
0,995
(0.9900.0000
1.000
0.997 (0.994/0.000)
1.000
0.985
(0.980N0.010)
0.970
Duck
0,995 (0 9900000000
1.000
0.995
(0.990000000)
1.000
0,997 (0.99400000))
1.000
0.995
(0.99000.000)
0,980
Echo
0.990 (0.98000.000)
1.000
0,995
(0:990000000)
1.000
0.997 (0.994/0.000)
1.000
0.985
(0.970X0.000)
0,960
Highpass
0.995 (0.990/0.000)
0,990
0.995
(0.990/00000)
1.000
0.997
(0.994/0.000)
1.000
0.995
(0 (0.99000.000)
1.000
Lowpass
0.965 (0195001020)
1 1,000
0.995
(0 (00))0)))00)
1.00
0.994
(0.99400000)
1,000
0.955
(1950NL004)
0.969
MP3
0.975 (0.970X0.020)
0,990
0.995
(0.990000000)
1.000
0,994
(0.994/0.006)
1.000
0.955
(0 (0.960000050)
0,947
Pink Noise
0.995 (0.990/0.000)
0,990
0.995
(0.9900.000)
0.920
0.997
(0.99400.000)
0.918
0.995
(0000000000)
0,940
White Noise
0.995 (0.990X0.000)
1.000
0,995
(0.99000000)
1.00
0.997
(0. (0.994/0.000)
1.000
0.995
(O (0.990X0000)
0.980
Smooth
0,990 (0.98000.000)
1. 1.000
0.995
(0.990000000)
1.000
0,997
(0.99400000)
1.000
0.985
(0.99000022)
0.927
Speed
0.905 (0.83000.020)
0,000
0,995
(0.99000000)
0.000
0.994
(0.994/0.006)
0,000
0,935
(0.920XL000)
0,000
Resample
0,970 (0.96000.020)
1.000
0.995
(0.990000000)
1.000
0.994
(0.987/0.000)
1.000
0.960
(0 (0.94000022)
0,949
AAC
0.990 (01980010000)
0.960
0.995
(055900)000)
1.000
0.997
(0.99400L00))
1,000
0.980
(1998N0022)
0.950
EnCodec (nq=16)
0.720 (0.16600.020))
0,970
0.810
(0.730/0.110)
0.000
0,956
(0.913/0.031)
1.000
0.735
(0.620N0.150)
0.706
Crop
0.995 (0.990X1.000)
0.947
0.995
(0:9900.000)
1.000
0.997
(0.994/0.000)
0.992
0.995
(0.990X0.000)
0.964
Average
0.967 (0.939X08006)
0.926
0.983
(0:9740.007)
0.870
0.994
(0.99000.003)
0.932
0.965
(0.952X0.223)
0.888
Table 11. The performance of detection and attribution of our method different out-of-distribution datasets (unseen in training) over different editing operations.
Dataset
BudioMarkBench
ASVspoof
MusicGen
MUSDB18
Edit
Det. (TPR/FPR)
Att.
Det. (TPR/FPR)
Att.
Det. (TPR/FPR)
Att.
Det. (TPR/FP)
Att.
Identity
0.938 (755/1000)
1.000
0.967
(0.933/0.000)
1.000
0.995
(0.99000000)
0.917
0.996
(0.99101000)
1.000
Bandpass
0,938 (0.87510.000)
1.000
0.967
(0.933/0.000)
1.000
0.995 (0.99090.000
1,000
0.996 (0.991/0.000)
1 1.000
Boost
0.938 (0.875X0.000)
1.000
0.967
(0.933/0.000)
1.000
0.990 (0.979/0.000)
0.969
0.996
(0.99L/0.000)
1.000
Duck
0.938 (0.87501000)
1.000
0.967
(0953000000
1.000
0.990
(0.9790.000)
0.969
0.996
(0.91XX1000)
1 1.000
Echo
0.938 (0.875X0.000)
1.000
0.967
(0.933/0.000)
1.000
0,984
(0.979/0.010)
0.938
0.996
(0.991/0.000)
1.000
Highpass
0.938 (0.875X.800)
1.000
0.967
(0.9330.000)
1.000
0.995
(0.99000.000
1.000
0.996
(0.991X1000)
1.000
Lowpass
0.938 (0.875X00000)
1.000
0.967
(0.9330.000)
0.933
0.969
(0.979H00042)
0,926
0.991
(0.991
1 1.000
MP3
0.938 (0. 875/0.800)
1.000
0.933
(0 (077
0.867
0,969
(0. (0.9690.031)
0.926
0.991
(0.983X1000)
1.000
Pink Noise
0.938 (0.875X0.000)
1.000
0.967
(0.9330.0000)
1.000
0.990
(0.979/0.000)
0.979
0,996
(0.991/0.000)
1.000
White Noise
0.938 (0.87500000)
1.000
0.967
(0.953)000)
1.000
0.990
(0.97900000)
0.896
0.996
(0.91101000)
1 1.000
Smooth
0.938 (788000000
1.000
0.967
(095500000)
0.867
0.974
(0.9790.051)
0.895
0.996
(0991X11000)
1 1000
Speed
0,938 (0.875X0.000)
0.000
0.967
(0.933/0.000)
0.000
0,911
(0.896/0.073)
0.000
0.987
(0.983100009)
0.000
Resample
0.938 (0.875X18000)
1.000
0.967
(0.9330.000)
0.867
0.974
(0.9790.031)
0.860
0.991
(0.991X1.09)
1.000
AAC
0.938 (0.875X0.000)
1.000
0.967
(0.93300.000)
1.000
0,979
(0.9900.031)
0.781
0.996
(0.991/0.000)
1.000
EnCodec (nq=16)
0.875 (0.75000,000)
1.000
0.867
(0.867/0.133)
1.000
0.885
(0875/0.104)
0.833
0.957
(0.957700443)
1.000
Crop
0.938 (0.875X0.000)
0.000
0.967
(0.933/0.000)
1.000
0.891
(0.875/0.094)
0.871
0.905
(0.888/0.078)
0.988
Average
0.934 (0.86 (0.867X1.000)
0.875
0.958
(0992500008)
0.908
0.967
(0.96300002))
0.860
0.986
(0.98221.01)
0.937
24

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
2.41
7006
4290
125
3340
330
-375
Rediiti Whoetorm
Our- Recdall Stectropram
Speidton Hel-specroggnm
500e
0.62
2006
3.5
5006
2008
4.5
AudioSeat- SedCUN Wooeeorr
Austirfeal Recdual Spectrogram
Ausic i- Spridiun He lepectrooo
xxce
sone
-3
2006
3.00
55.9
312
1.5
45
7.
52
Swnctefeldd Reniiual Wevetemm
Sewnderfedd Recidan! Spectrogrw
Sweeterfwis Resifial mepeeeccrogmmmm
63
3.0
2330
02
tone
Time
Tima
Time
206-
3.5
3.5
4004
-110
Time
Figure 11. Visualization of original audio and watermark residuals across different methods for randomly selected sample from the MusicCaps dataset, showing waveform (left), spectrogram (middle), and mel-spectrogram (right) representations.
where is the window index and b is the frequency band index. The weighting of the penalty term for each tile is exp((") computed in ben-band soft-mat max manner, wi exp(1r)) The final loudness loss is the weighted sum over all the tiles,
L Ew - Σ t. Cy.
However, we identify two limitations of this TF-based approach: (1) it lacks consideration of sophisticated auditory masking effects, where the masker and the maskee are not in the same tile; (2) using the loudness difference as the discrepancy measure only provides weak supervision for I'Fguiieed watermarking the spectrogram domain. To resolve these two issues, we propose to use a more sophisticated weightttted & loss to guide the watermarking process. First, we simulate in the TF domain the 2D energy asymmetric decay. This helps us identifying the potential masker-masker pairs using filtering rules based on psychoacoustic principles, Secondly, instead of using loudness difference as a discrepancy measure, we compute the -eaa-s error between the watermarked audio and the original one in the mel-spectrogram domain. This approach provides denser and finer-grained guidance signal for the imperceptible watermark.
Visualization of the TF-weighted penalty distribution. We further visualize the penalty weight distribution of different TF-based approaches of AudioSeal and XATTNMARK with the same spectrogram partitioning configuration. We use a randomly sampled watermarked audio from a model that is at the beginning of training, where the watermark still did not exploit the potential of the TF masking effect for better imperceptibility. As we can see from Figure 12, our method provides
25

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
(a) AudioSeal
(b) Ours
Ezoquracy Band
Time Bih
0.1
0.
0.6
0.8
Weight Magnitade
Weight Magnittude
Figure 12.
Visualization of per-tile masking penalty distributions the mel-spectrogram. Compared AudioScal, our method demon- strates more fine-grained penalty allocation. Blue colors denote regions with lower penalties, preferred for watermark embedding. The audio sample from MusicCaps (Agostinellii et al., 2023), with the watermark generated by early-stage model.
a more sieeggained and dense penalty distribution, while AudioSeal's 's approach just assigns the penalty weight to one tile (the red tile in the figure). Since we use multiple maskers in the vicinity to match their masked regions, our method is able to provide broader and more dense penalty guidance. This indicates that our method is more effective in exploiting the TF masking effect for better imperceptibility.
C.10. More Results on Watermark Localization
Similar to the Brute Force Detection (BFD) in WavMark (Chen et al., 2023), we further extend our model to have the localization ability. Specifically, since our model includes transformations robust to time shifts and operates on Is segments, can distribute the per-segment detection probability the per-frame level with multiple overlapping detection windows.
Mathematically, let the audio signal be represented approach utilizes multiple overlapping detection windows, where each window Wi corresponds to 1s audio segment. For each window We. our model computes watermark detection probability P(M (W), where AM denotes the presence of the watermark. given frame fi may be part of several such overlapping windows; let this set be denoted as W(fi) = (Wx ft E W. The per-frame detection probability P(M then determined by aggregating the probabilities from all windows covering that frame. We take the average of P()) for all W W(f)) as the per-frame detection probability P(M)ff,, ie., P(M))t) [WJJ Ew,www. PMM)... This process effectively distributes the segment-level detection scores finer, per-frame resolution, enabling precise localization of the watermarked regions.
sequence of frames F = f2, ... JNJ. ,N
Our localization
The results shown in Figure 13.
The experiments involve two setups: (1) fixing the audio length to 10s and varying the watermark segment length (2-10s) within it; (2) fixing the watermark segment length to Is and varying the total audio length (2-10s). For both setups, per-frame level detection is performed. Results show that XTTTNMARK can achieve comparable localization performance to AudioSeal and significantly outperforms WavMark.
C.11. Sensitivity Test in Model Architecture
We investigate the sensitivity of XATTNMARK to key architectural choices, specifically the hidden dimension (H) of the message embedding table and the architecture of the temporal conditioning module. The results of this analysis, focusing on convergence speed for message decoding (message-bit validation accuracy), are presented in Figure 14.
For the message embedding table, we explored various hidden dimensions H relative to the secret message bit-length b (fixed 16 in our experiments). We performed grid search over E {b/2, 6, 2b, 4b, 86). As illustrated in Figure 14a, the choice of H affects the convergence speed of the model, particularly for message decoding. We found that H values ranging from b/2 (i.e., 8) 4b (i.e., 64) yield relatively fast convergence. However, using much larger dimension, such as H = 8b (i.e., 128). can slow down the convergence process (the H 128 case did not converge within 50k training steps).
Regarding the temporal conditioning module, we conducted an ablation study by comparing different MLP architectures: a linear projection (as proposed in XATTNMARK), a 2-layer MLP, and 3-layer MLP. The results, shown in Figure 14b, indicate that the linear projection is the most effective, being the only one among the tested options that converges
26

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
1.0
1.0
Sample-ievel F1
0.8
Sample-level F1
0.8
0.6
0.6
0.4
0.4
Ours
Ours
AudioSeal
0.2
AudioSeal
Wavmark
Wavmark
0.0
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
10
seconds watermarked signal second audio
second watermarked signal second audio
(a) Fixed audio length (10s), varied watermark segment length (2-10s).
(b) Fixed watermark segment length (Is), varied audio length (2-
10s).
Figure 13. Watermark localization performance comparison. XAttnMark achieves localization comparable to AudioSeal and significantly outperforms WavMark.
successfully within the observed training steps. This suggests that the model's convergence sensitive to the architectural complexity of the temporal conditioning module.
C.12. Inference Efficiency and Model Size
We additionally report the model size, computational complexity, training speed, and inference speed comparison with AudioSeal in Table 12. Our model uses fewer parameters and has smaller size for both the generator and the detector. Although our generator has higher FLOPs and slightly increased inference time per segment (~-0.3 ms/segment), our detector significantly reduces FLOPs while maintaining similar inference efficiency overall. In training speed, our model achieves similar second- per-iteration rate (around 1.15 s/iter) AudioSeal, with faster convergence speed in learning message decoding shown in the App. C.2.
Table 12. Comparison of model statistics and inference speed between our method and AudioScal. Training speed is computed with batch size of over the first 100 steps.
Method
Component
#Params +
Model Size $
FLOPs 4
Inference Time (ms) 4
Train Speed (s/iter) J
Generator
10.14M
38.67 MB
1.18 x 10t1
0.411 + 0.108
Ours
Detector
7.59M
28.95 MB
1.96 10t
0.981 = 1.57
Total
17.73M
67.62 MB
3.14 x 10"I
1.39 + 1.58
1.15 + 0.51
Generator
14.68M
56.00 MB
1.35 x 10
0.707 + 0.118
AudioSeal
Detector
8.65M
32.99 MB
6.78 x 10t1
0.355 + 0.478
Total
23.33M
88.99 MB
6.92 10
1.06 0.493
1.05 + 0.27
C.13. Comparison of Perceptual Quality under Different Watermark Strengths
To compare the perceptual quality of XATTNMARK and AudioSeal under different watermark strengths, we conducted experiments adjusting the watermark intensity. For both XATTNMARK and the AudioSeal baseline, we targeted Signal- to-Noise Ratios (SNRs) at levels from 10 dB, with an interval of dB. The experimental results highlight two
27

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Mm
# 32
#
120
1.0
Message Bit Accuracy
Message Bit Accuracy
1.0
0.9
0.8
0.8
Linear (Ours): b-
2-Layer MLP: b
0.7
3-Layer MLP: b 21/2 01/4 -
0.6
0.5
10000
20000
30000
40000
50000
10000
20000
30000
40000
50000
Training Steps
Training Steps
(a) Message-bie bit validation accuracy with different embedding di- mensions (
(b) Message- bit validation accuracy with different temporal condi- tioning architectures.
Figure 14, Sensitivity analysis two model architecture components within 50k training steps. The message-bit validation accuracy evaluated the validation set.
key advantages of our approach. First, as illustrated Figure 15a, the average attribution accuracy (across edits) of XATTNMARK is consistently higher than AudioSeal when compared under different PESQ scores. Second, across different SNR levels, XATTNMARK consistently better or comparable perceptual quality than AudioSeal in terms of PESQ scores. In summary, under different watermark strengths, XATTNMARK achieves better/comparable performance in both attribution accuracy and perceptual quality, compared to AudioSeal.
Ours
AudioSeal
Ours
AudioSeal
1.0
Avg. Attribution Accuracy
4.5
0.9
4.0
0.8
PESQ
3.5
0.7
0.6
3.0
4.50
4.25
4.00
3.75
3.50
3.25
3.00
2.75
10
20
30
40
50
PESQ Score (Adjust with Watermark Strength)
SNR (dB)
(a) Average attribution accuracy different PESQ constraints.
(b) PESQ under different SNR constraints.
Figure 15. Performance comparison under different perceptual quality constraints with targeted SNR (10-50 dB, 5 dB interval). Figure 15a shows our method achieves better attribution accuracy across quality levels. Figure 15b shows PESQ under SNR constraints.
C.14. Statistical Significance Analysis Across Transformations
To rigorously evaluate the performance differences in robustness between XATTNMARK and AudioScal across various audio transformations, we conduct statistical significance tests. We employ MeNemar's 's test compare the proportions of correct attributions (a binary outcome for paired samples) and the Wilcoxon signed-rank test to compare the distributions of bit- bit-wise message accuracy scores (a continuous measure for paired samples). In Table 13, we report the results of these tests for each transformation applied to 10k audio samples per transformation. For the vast majority of transformations, XATTNMARK demonstrates statistically significant improvements over AudioSeal. Specifically, the p-values for both McNemar's 's test (on attribution correctness) and the Wilcoxon signed-rank test (on bit-wise accuracy) are typically far below the standard significance threshold of 0.05 (often 0.001, indicated as 0.000 the table). This indicates that the superior performance
28

---

XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
of XATTNMARK result of random variation. For instance, in the comprehensive 'Mixed of All' scenario (consisting of all transformations), XATTNMARK achieves attribution accuracy of 0.88 compared AudioSeal's 0.58, and bit-wise accuracy of 0.99 versus 0.93 for AudioSeal. In summary, XATTNMARK achieves statistically significant improvements over AudioSeal across various audio transformations.
Table 13. Statistical comparison watermarking robustness between XAttnMark and AudioSeal on our main setup (user size 10* used). For cach clip, generate 100 messages and thus obtain 10k audio samples in total for each transformation. Results show both McNemar's 's test on attribution correctness (discrete values) and the Wilcoxon signed-rank test on bit-wise message accuracy (continuous values). Significance is marked 'Yes' for p-values < 0.05 (i.e., 0.05).
McNemar's Test
Wilcoxon Test
Transformation
Ours
AudioSeal
Significant
Ours
AudioSeal
Significant
EnCodec (nq=16)
0.85
Yes (0,000)
0.99:
0,85
Yes (9.7e-14)
AAC compression
0.831.0.34
g..02
Yes (0.000)
1.0.3
0.62 1,010
Yes (2.8e-18)
Bandpass filter
0.99
0.76-2042
Yes (0.000)
1.00.10.00
0.98
Yes (7.1c-05)
Boost audio
1.00.
0.76
(0.000)
1.00.10.00
0.98
Yes (1.8e-05)
Duck audio
1.00::0:02
0.78=0042
Yes (0.000)
1.00=+0:00
0.98
Yes (1.2e-05)
Echo
0.98
0.50
Yes (0.000)
1.00
0.93
Yes (6.8e-10)
Highpass filter
0.99
0.77 2..42
Yes (0.000)
1.00
0.98
Yes (4.4e-05)
Identity
1.00=#0.01
0.78
Yes (0.000)
1.00=
0.98
Yes (5.9c-05)
Lowpass filter
1.00=+0:02
0.78
Yes (0.000)
1.00
0.98
Yes (5.9c-05)
MP3 compression
1.00::0.03
0.69
Yes (0.000)
1.00
0.98
Yes (1.4e-05)
Pink noise
1.908
0.77 10.42
(0.000)
1.00
0.98 1..04
Yes (1.7e-05)
White noise
1.00+:0:1
0.76=20.3
Yes (0.000)
1.00+10:00
0.98
Yes (3.6c-05)
Smooth
1.00
0.55
Yes (0.000)
1.00
0.93
Yes (2.9e-10)
Speed
0.03:t0.17
0.00=0005
Yes (7.8e-48)
0.43
0.53
Yes (9.5e-08)
Up/down resample
1.00.10.02
0.78 20.42
Yes (0.000)
1.00.10.00
0.98 10.04
Yes (5.9e-05)
Crop
0.76
0.37=20.48
Yes (0.000)
0.98
0.96
Yes (0.001)
Mixed of AlL.
0.88
0.58
Yes (0.000)
0.99
0.93
Yes (0.000)
29


Reviews:
[{"review_id": "qMFBp8bjCI", "reviewer": "Reviewer_fJ8F", "summary": "This paper proposes XATTNMARK, a novel neural audio watermarking system designed to achieve both robust detection and accurate message attribution, two goals that are difficult to achieve simultaneously in prior work. The authors blend the architectural benefits of WavMark and AudioSeal by introducing partial parameter sharing between the generator and detector, enabled via a cross-attention decoding mechanism and a shared embedding table. Additionally, a temporal message conditioning module and a psychoacoustic-aligned time-frequency masking loss are proposed to enhance imperceptibility and robustness. Experiments demonstrate that XATTNMARK achieves SOTA performance across a wide range of audio transformations, including generative edits, and adversarial attacks, while maintaining high perceptual quality.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their valuable feedback. We have addressed the reviewer's concerns as follows:\n\n> W1. The model still struggles with extreme transformations like speed changes (acknowledged in the text).\n\nIn the paper we show that the model is able to effectively perform the detection task under speed changes transformations. We also acknowledge that the model struggles against speed changes (and other challenging transformations like generative edits) for the attribution task. However, in the Appendix C.3.1, we show that, for challenging speeding operation, we could build up a simple speed reversion layer, that greatly improve the attribution performance without significant overhead (as shown in Table 7 in appendix). \n\n> W2. Attribution performance under generative edits was not deeply analyzed; only detection was reported.\n\nAs mentioned earlier, we acknowledge that our model is still limited in attribution robustness against generative edits. However, we believe that this still marks a significant step forward in watermarking against generative audio edits. To the best of our knowledge, we are the first to report non-trivial detection robustness (90%+) against generative editing in a zero-shot manner. With additional specialized training on those transformations, the attribution performance might be further improved. We leave this as future work. \n\n> W3. The paper does not evaluate robustness under white-box adversarial attacks.\n\nWe additionally report the robustness against white-box adversarial attacks in [Figure C](https://tinyurl.com/2xmc3km3).\n\n\n> C1. The paper would benefit from clarifying the decoding pipeline under attribution evaluation with large user pools (e.g., scalability of Hamming decoding).\n\nPlease refer to the 'Evaluation Setup' part of Appendix A and Table 6, where we have provided a detailed discussion on the Hamming decoding process used in the attribution evaluation and also the scalability aspect.\n\n> C2. Consider releasing code/models to improve reproducibility and adoption.\n\nThank you for the suggestion. We will consider releasing the code and models upon the paper's publication.\n\n> Q1: Can the attribution method be extended to a continuous space (e.g., using embeddings) to improve robustness under generative edits?\n\nThis is an interesting idea. Currently, the existing watermarking methods are mostly designed for embedding discrete bit-strings (e.g., 0s and 1s). However, our experiments show that, under the challenging generative editing, previous methods fail to succeed at both detection and attribution. One potential reason is that we treat the source as discrete informationless bit-strings, without leveraging the semantic information that could help the attribution task (e.g., style attribution [1]). For example, in the audio domain, a copyrighted timbre might have countless reference audio files, which could be leveraged as attribution anchors for more robust attribution. This is an orthogonal direction to our research, which we leave as future work. \n\n[1] Wang, Sheng-Yu, et al. \"Evaluating data attribution for text-to-image models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n> Q2: How sensitive is the performance to the architecture of the embedding table and temporal conditioning module?\n\nWe additionally provide the sensitivity study on the embedding hidden dimension and temporal conditioning architecture in [Figure D](https://tinyurl.com/49xfer89). For the embedding table, we found that the embedding dimension $H$ will affect the convergence speed of the watermark model in both detection and message decoding (more sensitive to $H$ in the message decoding part). With grid search over $[\\frac{b}{2}, b, 2b, 4b, 8b]$, where $b$ is the bit-length of the secret message ($b=16$), we found that $H$ values ranging from $H=\\frac{b}{2}$ to $H=4b$ both yield fast convergence except for $H=8b$. For the temporal conditioning module, we additionally provide an ablation study with different numbers of MLP layers (linear, 2-layer MLP, and 3-layer MLP). Results show that the linear projection proposed in XAttnMark is the only one that converges, indicating that the convergence is sensitive to the architecture choice of the temporal conditioning module.\n\n> Q3: The paper does not evaluate robustness under white-box adversarial attacks. How would the proposed method perform under white-box adversarial attacks compared with existing methods?\n\nWe additionally report the robustness against white-box, semi-black-box, and Gaussian noise attacks in [Figure C](https://tinyurl.com/2xmc3km3). The results show that XAttnMark is more robust than AudioSeal in Gaussian noise attacks. In the white-box and semi-black-box attack scenario, we observe that XAttnMark is slightly more vulnerable to white-box attacks compared to AudioSeal, which might be due to the smaller model size of the detector module (XAttnMark is 7.59M, while AudioSeal is 8.65M).", "claims_and_evidence": "The central claim is that XATTNMARK simultaneously achieves robust detection and accurate attribution across diverse audio transformations, outperforming existing SOTA methods. The empirical evidence, especially Table 1 and Table 2, supports this claim convincingly. The robustness against generative editing and adversarial removal is particularly noteworthy, as prior methods degrade significantly under such settings. Ablation studies (Figure 4) and quality assessments (Table 4) further strengthen the evidence for each architectural component’s contribution.", "methods_and_evaluation_criteria": "The methodology is technically sound. The partial parameter sharing via a shared embedding table and cross-attention decoding is well-motivated and novel. The experimental protocol is thorough, using 16 types of transformations, two generative editing models (AudioLDM2, Stable Audio), and adversarial perturbations (HSJA). Baselines are strong (AudiowMark, WavMark, TimbreWM, AudioSeal), and evaluation metrics include detection/attribution accuracy, perceptual audio quality, and robustness under various threats.", "theoretical_claims": "The paper is mostly empirical, but the formulation of the psychoacoustic-aligned temporal-frequency masking loss is theoretically grounded in auditory perception literature. The architecture and cross-attention design are sound from a deep learning perspective.", "experimental_designs_or_analyses": "The paper extensively benchmarks detection and attribution across a wide range of realistic scenarios, including speed edits, generative model edits, and adversarial attacks. The performance gains are statistically significant, and trade-offs are clearly analyzed. Ablations are especially useful in isolating the impact of core contributions.", "relation_to_broader_scientific_literature": "The paper properly contextualizes its contribution within prior watermarking methods (AudiowMark, WavMark, AudioSeal), as well as broader work on dataset attribution and copyright auditing. References are recent and well-curated.", "essential_references_not_discussed": "None glaring.", "other_strengths_and_weaknesses": "Strengths:\n\n- Strong empirical gains across detection and attribution.\n\n- Well-structured methodology with insightful architectural design.\n\n- Broad experimental coverage (standard, generative, adversarial).\n\n- High practical relevance in the age of generative audio content.\n\nWeaknesses:\n\n- The model still struggles with extreme transformations like speed changes (acknowledged in the text).\n\n- Attribution performance under generative edits was not deeply analyzed; only detection is reported.\n\n- The paper does not evaluate robustness under white-box adversarial attacks.", "other_comments_or_suggestions": "- The paper would benefit from clarifying the decoding pipeline under attribution evaluation with large user pools (e.g., scalability of Hamming decoding).\n\n- Consider releasing code/models to improve reproducibility and adoption.", "questions_for_authors": "- Can the attribution method be extended to a continuous space (e.g., using embeddings) to improve robustness under generative edits?\n\n- How sensitive is the performance to the architecture of the embedding table and temporal conditioning module?\n\n- The paper does not evaluate robustness under white-box adversarial attacks. How would the proposed method perform under white-box adversarial attacks compared with existing methods?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IsU4A2ANi2", "reviewer": "Reviewer_otVN", "summary": "The paper introduces a novel neural audio watermarking framework called XATTNMARK. The key contributions include:\nA cross-attention mechanism that enables efficient message retrieval by sharing an embedding table between the generator and detector.\nA temporal conditioning module that distributes the message temporally, improving learning efficiency.\nA psychoacoustic-aligned temporal-frequency masking loss that enhances watermark imperceptibility by leveraging human auditory masking effects.\nThe main findings show that XATTNMARK achieves state-of-the-art performance in both detection and attribution, demonstrating superior robustness against a wide range of audio transformations, including challenging generative editing.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for taking the time to review our manuscript and providing valuable feedback. We have carefully considered each point raised and provide our detailed responses below:\n\n> Q1. The subjective listening test involves a relatively small number of participants. \n\n**Response:** We launch our subjective listening test with 18 participants initially following the ITU-R BS1534-1 [1] standard and practice in related audio publications [2,3]. Specifically, ITU-R BS1534-1 suggests that **when the conditions of a listening test are tightly controlled on both the technical and behavioral side, experience has shown that data from no more than 20 subjects are often sufficient for drawing appropriate conclusions from the test**. Our internal test with unified software/process and participants from a similar background profile satisfies this control requirement. \n\nDuring the post-screening process, we further filtered out 6 participants who missed the reference audio to ensure the result's validity, resulting in 12 valid evaluators. Similarly, SilentCipher[3] also performs post-processing on the test group results with 12 valid evaluators in total. While these references support our setup, we acknowledge that the population size for our MUSHRA test is relatively limited. If needed, we will expand the test population and update our results in the final version of the paper.\n\n[1] Method for the subjective assessment of intermediate quality level of coding systems (Recommendation ITU-R BS.1534-1), International Telecommunication Union. (2003). \n\n[2] Davidson, G., Vinton, M., Ekstrand, P., Zhou, C., Villemoes, L., & Lu, L. (2023). High Quality Audio Coding with MDCTNet. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).\n\n[3] Singh, M. K., Takahashi, N., Liao, W., & Mitsufuji, Y. (2024). SilentCipher: Deep Audio Watermarking. In Proc. Interspeech 2024 (pp. 2235-2239).\n\n---\n\n> Q2. The ablation study on the adaptive bandwidth (constant weight $\\gamma$ = 1) is limited to a single configuration.\n\n**Response:** Our ablation study on the adaptive bandwidth mainly focuses on the two most representative cases, the constant weighting and the adaptive weighting for per-mel masking radii, which we propose in our work. In our design, the per-mel-bin masking radii $r^m$ are adjusted based on the frequency instead of being set as a constant across all frequencies. Adjusting different $\\gamma$ values can be viewed as a hyperparameter tuning process on the base radius $r^m_b$, which still assigns the same radii across all the mel-bins and conceptually belongs to the same class of constant weighting. Due to the time constraint, we leave the exploration on this hyperparameter searching for constant weighting as a future work.", "claims_and_evidence": "The claims made in the paper are well-supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The evaluation on diverse audio transformations and benchmark datasets provides a rigorous assessment of the method's robustness and practical applicability.", "theoretical_claims": "The theoretical claims in the paper are supported by empirical evidence and are grounded in well-established principles.\nThe paper mentions adversarial attacks but does not provide detailed theoretical analysis on the robustness of XATTNMARK against such attacks.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper are generally sound and provide strong evidence to support the claims. \nThe subjective listening test involves a relatively small number of participants.\nThe ablation study on the adaptive bandwidth (constant weight γ = 1) is limited to a single configuration.", "relation_to_broader_scientific_literature": "The key contributions of the paper are well-grounded in the broader scientific literature on audio watermarking and generative audio technologies. XATTNMARK builds upon previous work by introducing innovative mechanisms for message retrieval, temporal conditioning, and psychoacoustic alignment.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "No", "questions_for_authors": "See above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TJ8L3DrYan", "reviewer": "Reviewer_f1fs", "summary": "This paper focuses on robust audio watermark detection and source attribution, which is more a technical report than a top-tier conference paper. Specifically, it adopts blended architecture of disjointed generator-detector and fully shared-parameter architecture. Besides, temporal conditioning mechanism and per-tile temporal-frequency masking loss are utilized to improve watermarking performance. In general, the writing is good from the technical aspect. It emphasizes the details of each contribution, lacking more thorough and deep analysis. The experimental results show its effectiveness of proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's insightful attention and precise feedback. We have addressed the reviewer's concerns as follows:\n\n> Q1: How about the comparison of model's parameters, training speed and inference speed?\n\n**Response:** We additionally report the model size, training speed, and inference speed comparison with AudioSeal in [Table D](https://tinyurl.com/2p9p5cpd). Our model uses fewer parameters and has a smaller size for both the generator and the detector. Although our generator has higher FLOPs and a slightly increased inference time per segment (~0.3 ms/segment), our detector significantly reduces FLOPs while maintaining similar inference efficiency overall. In training speed, our model achieves a similar second-per-iteration rate (around 1.15s/iter) as AudioSeal, with faster convergence speed in learning message decoding. Specifically, as shown in Appendix C.2, our model takes ~4k steps to reach perfect detection accuracy, and ~10k steps to reach perfect attribution accuracy, while AudioSeal takes ~32k steps to reach perfect detection accuracy, and 50k steps to reach around 70% attribution accuracy. This demonstrates that XAttnMark achieves 5 to 8 times better training efficiency than AudioSeal.\n\n---\n\n> Q2: As discussed above, the paper focuses on illustrating technical details while lacking more deeper insights for robust audio watermarking.\n\n**Response:** Thank you for this valuable point. We will refine our presentation to include more design insights. Due to the space limit, we put a significant part of the details on the design motivation in the appendix sections. In the appendix, we have provided more analysis and discussion on the proposed modules, including the cross-attention architecture and the proposed temporal-frequency perceptual loss. Specifically, \n- In Appendix C.2. (Analysis of the Training Dynamics of Models with Different Architectures), we analyze the training dynamics of different architectures under a controlled experimental setup to better understand their inherent learning capabilities. \n- In Appendix C.8, we provide a comprehensive comparison with the TF-loudness loss of AudioSeal.\n\nDuring the revision of the paper, we will add these deeper technical insights to the main text for better readability.", "claims_and_evidence": "The experimental results, including comparison with SOTA and ablation studies show its effectiveness of proposed method.", "methods_and_evaluation_criteria": "As discussed above, the paper focus on illustrating technical details while lacking more deeper insights for robust audio watermarking.", "theoretical_claims": "There are no proofs for theoretical claims.", "experimental_designs_or_analyses": "The experimental results, including comparison with SOTA and ablation studies show its effectiveness of proposed method.", "relation_to_broader_scientific_literature": "It achieves both better performance on audio watermark detection and source attribution compared with SOTA methods.", "essential_references_not_discussed": "The references seem adequate.", "other_strengths_and_weaknesses": "As discussed above, the motivation behind the key contributions is not clear, which merely introduce problems and propose detailed technique and verify it with experiments. Although most papers are similar style with it, this paper is much more to be a technical report than other papers to be reviewed.", "other_comments_or_suggestions": "The writing is good in general. However, in my opinion, it is more like a technical report.", "questions_for_authors": "How about the comparison of model's parameters, training speed and inference speed?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WvIutOxXMo", "reviewer": "Reviewer_7QuH", "summary": "This paper presents a robust watermarking scheme XAttnMark for audio content, where the embedding and detection of the watermark is performed using neural networks. A key aim of the work is to improve robust attribution (the ability to recover a binary code hidden in the content) while retaining robust detection (the ability to determine whether the content is watermarked or not). The proposed approach builds on AudioSeal, contributing several architectural modifications and a new loss function to improve imperceptibility of the watermark to the human ear. The architectural changes include: (1) sharing of the message conditioning module between the watermark embedding and detection networks, which involves using cross-attention in the detection network; (2) using a learned) linear function for the message conditioning module in place of mean-pooling with temporal-axis repetition. Empirical evaluations indicate that XAttnMark achieves significantly higher attribution accuracy than AudioSeal, with comparable detection accuracy and perceptual quality. XAttnMark is also shown to be the only watermark that achieves reasonably robustness (accuracy > 90%) under editing using audio diffusion models.\n\n## Update after rebuttal\n\nPrior to the rebuttal my main concerns were around:\n1. The significance of the empirical results due to small sample sizes\n2. Confusion around the cause for the improved performance\n3. Unclear motivation for introducing a new perceptual loss\n\nThe authors addressed all of these concerns:\n1. They explained that the sample size is much larger than 100 as they produce 100 watermarked samples _per_ original audio sample. \n2. They corrected my misinterpretation of the ablation study results, reassuring me that parameter sharing is in fact the leading cause for the improved performance.\n3. They summarized limitations of AudioSeal's perceptual loss in their response, and noted that this is discussed in an appendix. I encourage to the authors to include a summary of this discussion in the body of the paper.\n\nThe authors also provided several new experimental results during the rebuttal period that further enhance the comprehensiveness of the empirical evaluation (evidence of statistical significance, investigation of localized watermarks, more comprehensive attack results, inclusion of false attribute rate for comparison with AudioSeal).\n\nI am now convinced that the paper is sound and will make a strong contribution to the audio watermarking literature. I have therefore increased my score to recommend acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their meticulous and constructive feedback. We will revise the manuscript by improving the introduction and mentioning the CAI initiative and the C2PA standard in the body of the paper. Our responses are as follows:\n\n\n> Q1. On the statistical significance of the results, and concerns on the #validation set.\n\nWe want to clarify that, while our test set is indeed composed of 100 audio files, we follow AudioSeal's protocol by hiding 100 messages per file, corresponding to a total of 10k watermarked audio samples. We then apply 16 audio transformations to each one of them. So the total number of samples contributing to our scores is *160k*. To further validate the statistical significance of our results, we perform McNemar's test and Wilcoxon signed-rank tests across edits with 1e4 users. The results are reported in [Tables A and B](https://tinyurl.com/utkjb4yr), which shows that the results on our setup are statistically significant in both attribution performance and perceptual quality.\n\n> Q2. On the definition of attribution accuracy, report of false attribution rate, and the results discrepancy from the two baseline papers.\n\nIn our work we define the attribution accuracy in a different way, as the fraction of correct attributions among the *detected* audio inputs, which equals to $1 - FAR$ (False Attribution Rate reported in AudioSeal). This definition decouples the attribution performance from the detection performance. To show a direct comparison between the different metrics, we report the results on both MusicCaps and VoxPopuli setups in [Table C](https://tinyurl.com/3kjjr95f).\n    \n> Q3&W.1 On the motivation of the proposed perceptual loss, the comparison with the TF-Loudness loss in AudioSeal, and the justification of additional complexity with score gain.\n\nIn Appendix C.8, we discussed the advantage of our proposed masking loss compared to the TF-loudness loss. The TF-loudness loss employs a coarse approach based on loudness differences within each tile, neglecting sophisticated auditory masking effects, such as the interactions between masker and maskee across tiles. Additionally, we found that using loudness difference as a discrepancy measure provides only weak supervision. In contrast, we have designed a more sophisticated TF-weighted MSE loss, which simulates a two-dimensional energy decay in the temporal-frequency domain, effectively identifying masker-maskee pairs, leveraging psychoacoustic principles. Furthermore, we utilize mean-square error in the mel-spectrogram domain as our discrepancy measure, providing more fine-grained guidance (see our qualitative comparison in Figure 10 in the appendix). To quantitatively justify the effectiveness of our proposed loss, we evaluate the attribution accuracy under different watermark strengths (controlled by PESQ ranges) in [Figure A](https://tinyurl.com/y38skm78). These results clearly indicate that our method consistently achieves significantly higher attribution accuracy at each imperceptibility level.\n    \n    \n> Q4. On the lack of results comparing computational efficiency.\n\nWe additionally report the results on computational efficiency in [Table D](https://tinyurl.com/2p9p5cpd). \n    \n> W2. On the localization capabilities of XAttnMark.\n\nThanks for this insightful point. Although we have not explicitly discussed the localization capabilities of XAttnMark, our model can be easily extended to have this ability with sliding window detection. Specifically, since our model includes shifting-robust transformations and operates on 1s segments, we can distribute the per-segment detection probability to the per-frame level with multiple overlapping detection windows as the BFD in WavMark does. We have implemented this and report the results in [Figure B](https://tinyurl.com/yxtft2mb). Results show that XAttnMark can achieve comparable localization performance to AudioSeal and significantly outperforms WavMark.\n    \n    \n> Q5. On the comparison with stronger gradient-based attacks in the semi-black box and white box settings and fairness on the HSJA.\n\nWe additionally report the robustness against the white-box, semi-black-box attacks in [Figure C](https://tinyurl.com/2xmc3km3). Results show that XAttnMark is slightly more vulnerable to these two attacks compared to AudioSeal (might be attributed to our smaller detector). On the fairness concern on HSJA, we clarify that we use the same attack budget for the two methods, and the higher perceptibility score in ours is because HSJA fails to successfully find more adversarial samples within the given budget compared to AudioSeal's case.    \n    \n> Q6. On the contribution of the cross-attention and the conditioning module.\n\nRegarding the interpretation of Fig. 4, we clarify that our claim is that, when considering a single module in isolation, the cross-attention module (acc. of 62%) is more effective than the temporal conditioning module (acc. of 50%) in enhancing efficiency.", "claims_and_evidence": "- The claims of improved robustness and attribution accuracy are based on validation sets of size 100, whereas the AudioSeal paper uses a validation set of size 10,000. For a validation set of size 100, the standard error of accuracy/FPR/TPR could be as large as 5 percentage points, which may call into question the statistical significance of the claims. \n\n- The paper claims that “the fully disjointed architecture of AudioSeal ($\\Theta_\\mathcal{G} \\neq \\Theta_\\mathcal{D}$) often converges fast for watermark detection learning but struggles to learn the message decoding part efficiently and accurately” (p. 4). However, the ablation study in Fig. 4 suggests that the main limitation may not be the lack of parameter sharing, but rather the choice of message conditioning module. Swapping out the proposed linear message conditioning module with AudioSeal’s results in a message bit accuracy drop from ~98% to ~62%. On the other hand, the use of cross-attention seems to have far less impact on accuracy (between 5-10 percentage points). \n\n- The paper claims that XAttnMask is consistently more robust than AudioSeal against adversarial watermark removal attacks (p. 8). However, the attacks against XAttnMask are generally weaker in terms of their perceptibility than the attacks against AudioSeal, as measured by PESQ, SI-SNR and ViSQOL. Hence the comparison may not be entirely fair. More broadly, the experiments supporting this claim are not as comprehensive as those performed in the AudioSeal paper, which includes stronger gradient-based attacks in the semi-black box and white box settings.", "methods_and_evaluation_criteria": "Yes, the empirical evaluation largely follows norms established in prior work – in terms of datasets, evaluation metrics, and the kinds of benign/adversarial transformations considered for watermark removal.  \n\n \n\nIt’s great to see an ablation study (Table 4 and Figure 4) to assess the impact of the proposed architectural changes in isolation.  \n\n \n\nMy concerns around the evaluation are: \n\n- The use of much smaller validation sets than prior work.  \n\n- The definition of attribution accuracy is unclear. In prior work (San Roman et al., 2024), the attribution accuracy is the fraction of examples for which the detection is positive _and_ the attribution is correct. However, in the paper, it appears to be defined as the fraction of detected examples for which the attribution is correct. \n\n- The paper does not report false attribution rate alongside attribution accuracy (see San Roman et al., 2024). This is important as there is a trade-off between false-positives and false-negatives. \n\n- There are no results comparing computational efficiency.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "- Post-hoc neural-network based watermarking. The proposed architecture and training procedure builds on AudioSeal (San Roman et al., 2024) as explained in Appendix A. Similar approaches have been proposed in the image domain – e.g., StegaStamp by Tancik et al. (2020) which is not cited. The claim that AudioSeal “pioneered the disjointed generator-detector paradigm for neural watermarking” (p. 3) is incorrect. StegaStamp is the earliest example I’m aware of, but there may be others. \n\n- New loss for imperceptible watermarks. The proposed loss is inspired by psychoacoustic masking principles (Gelfand, 2017; Holdsworth et al., 1998), recognizing that human listeners struggle to detect small changes in the temporal/frequency proximity of loud sounds. San Roman et al. (2024) also proposed a perceptual loss for audio called TF-Loudness. The paper does not clearly articulate why a new loss is needed, nor how the two losses differ. \n\n \n**References**\n\n- Tancik, Matthew, Ben Mildenhall, and Ren Ng. \"StegaStamp: Invisible hyperlinks in physical photographs.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "essential_references_not_discussed": "I think it’s important to mention the Content Authenticity Initiative (CAI), as an alternative solution to watermarking. It enables tracking of content provenance (including source attribution) via the addition of C2PA metadata, secured by cryptographic means.", "other_strengths_and_weaknesses": "S1. The empirical evaluation is generally well-executed, apart from my criticism about statistical significance due to small sample sizes. It’s great to see the inclusion of multiple baselines, a range of attacks/benign transformations (including a new diffusion-based attack), and multiple validation datasets. Incidentally, the results for other datasets in Appendix C.6 should be referenced in the body of the paper. \n\n \n\nS2. The writing is generally clear. However, I feel the introduction could be made more accessible for readers who are unfamiliar with watermarking and audio. \n\n \n\nW1. The new perceptual loss introduced in Section 4.2 is not adequately compared with prior work. I would like to see a qualitative and quantitative comparison with the TF-Loudness loss introduced in AudioSeal. The new loss seems more complicated than TF-Loudness in its construction, so it’s important to provide evidence that the additional complexity has some benefit (e.g., increased detection accuracy for a given level of imperceptibility). \n\n \n\nW2. The proposed watermarking scheme does not seem to include _localization_ as a design criterion. In contrast, both WavMark (Chen et al., 2023) and AudioSeal (San Roman et al., 2024) seek to embed _localized_ watermarks in audio, to enable detection of small segments of watermarked audio (e.g., AI-generated speech or copyrighted music) within longer audio clips. By abandoning localization as a constraint, XAttnMark may have an unfair advantage in its ability to achieve high detection/attribution accuracy. I'd like to see some discussion of this in the paper. \n\n \n\nW3. A key focus of the paper is on improving source attribution of audio watermarking. However, there is limited discussion explaining why source attribution is important and explaining whether the proposed watermarking scheme addresses the problem. For example, if users of a service are regarded as “sources”, then is a message pool of 10,000 large enough in practice?", "other_comments_or_suggestions": "- Table 3: The names of the quality metrics are introduced in Sec 5.3, after the table is introduced. The columns are missing arrows, indicating whether higher/lower values are better.", "questions_for_authors": "1. Could the authors comment on the statistical significance of the empirical results. \n\n2. How is attribution accuracy defined? Why are the results for WavMark and AudioSeal different to those reported in the AudioSeal paper?  \n\n3. What is the motivation for introducing a new perceptual loss? Is there a problem with the perceptual loss used in AudioSeal?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents XATTNMARK, a neural audio watermarking framework that aims to deliver both robust watermark detection and precise message attribution. The proposed approach combines elements from prior work (i.e., WavMark and AudioSeal), incorporating partial parameter sharing between the generator and detector through a cross-attention decoding mechanism and a shared embedding table.

Reviewers found the method compelling, with strong performance and clear presentation. However, they also pointed out a lack of detail regarding the method’s motivation and noted some missing evaluations. The authors addressed most of these concerns with additional results and methodological clarifications. I encourage the authors to include these additional experiments as part of their manuscript, this will greatly improve their submission.
Decision: Accept (poster)