MOLM: Mixture of LoRA Markers
Authors: Samar Fares
Nurbek Tastan
Noor Hazim Hussein
Karthik Nandakumar
Venue: ICLR 2026 Poster (2026)
Keywords: Watermarking
Diffusion models
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Generative models can generate photorealistic images at scale. This raises serious concerns about the ability to detect synthetically generated images and attribute these images to specific sources. While watermarking has emerged as a possible solution, existing methods remain fragile to realistic distortions, susceptible to adaptive removal, and expensive to update when the underlying watermarking key changes. We propose a general watermarking framework that formulates the encoding problem as key-dependent perturbation of the parameters of a generative model. Within this framework, we introduce  Mixture of LoRA Markers (MOLM), a routing-based instantiation in which binary keys activate lightweight low-rank adapters (LoRA) inside residual and attention blocks. This design avoids key-specific re-training and achieves the desired properties such as imperceptibility, fidelity, verifiability, and robustness. Experiments on Stable Diffusion and FLUX show that MOLM preserves image quality while achieving robust key recovery against distortions, compression and regeneration, averaging attacks, and black-box adversarial attacks on the extractor. Code is available at [https://github.com/Samar-Fares/MOLM-Watermark](https://github.com/Samar-Fares/MOLM-Watermark)

Published as a conference paper at ICLR 2026
MOLM: MIXTURE OF LORA MARKERS
Samar Fares', Nurbek Tastan', Noor Hussein', Karthik Nandakumar112 'Mohamed bin Zayed University of Artificial Intelligence (MBZUAD, UAE MMichigan State University (MSU), USA samar. fares @mbzuai.........mm.aa ac ae, nandakume msu edu
ABSTRACT
Generative models can generate photorealistic images at scale. This raises seri- ous concerns about the ability to detect synthetically generated images and at- tribute these images specific sources. While watenmarking has emerged possible solution, existing methods remain fragile to realistic distortions, suscep- tible to adaptive removal, and expensive to update when the underlying water- marking key changes. We propose a general watermarking framework that for- mulates the encoding problem as key-dependent perturbation of the parameters of generative model. Within this framework, we introduce Mixture of LoRA Markers (MOLM), a routing-band based instantiation which binary keys activate lightweight low-rank adapters (LoRA) inside residual and attention blocks. This design avoids key-specific re-training and achieves the desired properties such imperceptibility. fidelity, verifiability, and robustness. Experiments on Stable Diffusion FLUX show that MOLM preserves image quality while achieving robust key recovery against distortions, compression and regeneration, averaging attacks, and black-bor box adversarial attacks on the extractor. Code is available at httpss//ggtub..com/mmaaaaaaaaa Pawte//////aaaaaarrr/////O. Watermark
INTRODUCTION
Recent advances in diffusion models have enabled unprecedented progress in text-to-image gener- ation, with models such as Stable Diffusion (Rombach al.) 2022) and FLUX (Labs) 2024) pro- ducing high-quality, photorealistic outputs at scale. While these models provide powerful creative and practical tools, their ability to synthesize realistic content raises concerns regarding authenticity, misuse, and attribution. To address these challenges, watermarking has emerged as a core strategy for enabling model owners to verify whether a given image originated from their model. An effec- tive watermark must satisfy four criteria: imperceptibility (the watermark does not degrade image quality), fidelity preservation (outputs remain close to the real image distribution), robustness (the watermark resists removal and forgery), and detection/attribation (the watermark can be reliably extracted and linked to the model origin).
Existing watermarking methods face persistent challenges. First, the WAVES benchmark (An et al, 2024) shows that while some watermarks survive minor distortions, adversarial attacks easily break them. Regeneration attacks (Zhao al.) 2024b) erase watermarks by denoising and reconstructing images, affecting methods like Tree-Ring (Wen et al. 2023) and Stable Signature (Fernandez et al.,
2023 Averaging attacks (Yang et al. 2024a) remove forge content-agnostic watermarks by com- bining generated samples, which has been demonstrated on Tree-Ring (Wen et al. 2023), Gaussian Shading (Yang et al., (2024b), and Stable Signature (Fernandez et al.) 2023). Surrogate decoders (Jiang et al.] 2023) craft perturbations that bypass the target extractor while preserving recovery in shadow model, and purification-baat
defenses (Saberi al.) (024) can erase watermarks with minimal perceptual change. Second, robustness often conflicts with perceptual quality: improving resilience typically introduces visible degradation (Zhao et al. 2024a), and modifying the initial noise reduces image quality. Third, many approaches are computationally expensive and inflexible. Methods embedding watermarks the latent space or training process demand costly retraining, especially when updating changing watermarking keys. Examples include full-model finetuning in backdoor methods (Liu et al.) 2023, Zhao et al.) 2023). weight modulation in WOUAF (Kim)
al. 2024), per-key training in Stable Signature (Fernandez al. 2023) and SleeperMark (Souri

---

Published as a conference paper at ICLR 2026
Training
Deployment
Go
User
A cat sitting
Go (R(()
Limp
Text Prompt
x
Model Owneri
Verifier
Vn
Vn
Detected
Lver
Extract
Altribute
Figure 1: Proposed watermarking framework. During training (left), text prompt is processed by both the frozen generator Go (producing clean image x) and the perturbed generator (4-+4(()) (producing watermarked image x). where (*(n) denotes key-dependent parameter perturbation. The extractor V,, recovers the embedded key R. The parameter perturbation function (A$) and extractor parameters (1) are jointly learned by optimizing the perceptual loss Limp and key loss Lver. During deployment (right), the model owner/verifier extracts the embedded key to verify the watermark and attribute generated images to a specific model.
et al.
2022), per- per-prompt optimization in ROBIN (Huang et al. 2025). and pretraining in AquaLoRA Feng et al. 2024). Such overhead limits their practicality in dynamic deployments.
In this paper, we make two main contributions: First, we introduce a general watermarking frame- work that formulates watermarking as key-dependent perturbations of frozen generative model. Second, building on this framework, we propose Mixture of LoRA Markers (MOLM), a routing- based instantiation that reinterprets low-rank adapters (LoRA) as watermark carriers. binary key deterministically selects adapter activations across generative model building blocks, embedding watermark information without modifying the backbone. MOLM offers three advantages: Ef- ficiency- only lightweight adapters are trained; no pretraining on watermarked data is required, and the model itself produces watermarked samples during generation; (ii) Scalability capacity scales naturally with the number of routing layers and adapter choices, without retraining for new keys; and (iii) Robustness distributed routing makes the watermark harder to remove or forge while preserving image fidelity. We evaluate MOLM Stable Diffusion and FLUX across the MS-COCO, LAION LAION-Aesthetie Aesthetics datasets, testing fidelity, extraction accuracy, capacity scaling, and robustness against wide range of distortions and adversarial attacks. Our results show that MOLM achieves strong key recovery > 0.98 bit accuracy, preserves image quality (FID degradation < 1.5), and maintains robustness under geometric and photometric distortions, compression and diffusion- based removal, averaging attacks, and adaptive adversarial attacks on the extractor. These findings highlight MOLM as a practical and scalable watermarking method for modern diffusion models.
2 BACKGROUND
2.1 DIFFUSION MODELS
Diffusion models are leading class of generative models, delivering state-of-the-art image quality with stable training (Ho et al. 2020). They rely on two processes: forward pass that gradually adds Gaussian noise to data, and reverse pass that learns to denoise and recover samples. Latent Diffusion Models (LDMs) (Rombach et al., 2022) improve efficiency by operating in a compressed latent space rather than pixel space, with encoder-decoder pair mapping between images and latents (2 = E = D()).. A U-Net backbone (Ronneberger et al., 2015) acts as the denoiser, extracting multi-scale features during the reverse process. Guidance mechanisms such as Classifier- Free Guidance (CFG) (Ho & Salimans, 202T) further enhance controllability. Additional details are provided in Appendix A.1
2

---

Published as a conference paper at ICLR 2026
2.2 WATERMARKING OF GENERATIVE MODELS
Watermarking can be categorized into model-specific watermarking, which embeds signals directly via generative models, and data-specific watermarking, which modifies input data (Wang et al, 2024a). Model-specific watermarking methods fall into three main types: Encoder-decoder meth- ods: These introduce an injector (encoder) to embed messages and a decoder to recover them. Hidden (Zhu et al. 2018) first proposed embedding watermarks as adversarial perturbations, where an encoder perturbs images to encode a secret and a decoder extracts it without the original. For GANs, Yuu et al. 202T) embedded artificial fingerprints into training data, while (Fei et al.,, (022)) trained GANs with a fixed decoder and auxiliary watermark loss to enforce ownership. (Zeng al. 2023) replaced the decoder with detector, adversarially training the injector-detector pair. Backdoor-based methods: Inspired by backdoor attacks, these embed trigger that activates wa- termark generation. (Zhao al.]|| (2033) bound token to a watermark image via DreamBooth fine- tuning (Ruiz et al. 2023). (Liu et al., 2023) proposed naiveWM, linking a trigger word directly to watermark, and fixedWM, requiring a fixed prompt position for stealth. SleeperMark (Wang al.) 2024b) injects latent-level watermarks via UNet fine-tuning with triggered prompts. Generation- process methods: These modify the diffusion process itself, embedding watermarks into latent tra- jectories or model components.
Ring (Wen et al. 2023) inserts concentric patterns in Fourier latents, while ROBIN (Huang et al.) 2025) jointly optimizes patterns with text conditioning. Gaus- sian Shading (Yang et al.]]_]024] (224b) maps watermark bits into latents without training, with extraction via DDIM inversion. (Xiong et al. 2023) fused binary matrices into decoder layers, while Rezaei et al. (202)) and [Mnn al. (2024) progressively embedded across latent layers. [Peng et al. (2023) proposed WDP, where a parallel diffusion trajectory yields verifiable watermarked samples. Stable Signature (Fernandez et al. 2023) fine-tuned the LDM decoder with an extractor to enforce multi- bit signatures. WOUAF (Kim el al. 2024) modulated decoder weights via affine transformations of watermark messages with a joint decoder. AquaLoRA (Feng et al. 2024) embedded watermarks into UNet LoRA modules by pretraining a latent watermark with encoder decoder and fine-tuning with a prior-preserving loss.
3 PROPOSED METHOD
Notations: Let Go : T - X' be a text-to-image generation model that starts with a random initialization (latent) q E Q and uses the conditioning text (prompt) t E T to generate an image x E X. Here, 9 denotes the architecture of the generative model, denotes its parameters, and Q, T, and X' represent the latent, prompt, and image spaces, respectively. A watermarking system W consists of a tuple (Uc, V,,) that represents the watermark embedding and extraction algorithms, respectively. The watermark embedding method Uę: X x K. + X' can be considered around the generator G& that embeds a given M-bit key E K. C 10, 1 within the generated image to create a watermarked image X. The watermark extractor Vn X + IC is designed to extract key R I. from given image. Let
wrapper
Go (q. t) be image generated by the model Go for some q E Q and t E T. Let x = Ux(x, k) be the watermarked image for some E K. Let =V, (x) be the extracted key. Ideally, the watermarking system must satisfy the following four properties: (i) Impercepribilitt: The watermarked image X is as close as possible to the original generated image x; (ii) Fidelity: The distribution of x is as close as possible to the distribution of real images; (iii) Verifiability: The extracted key R is as close as possible to the embedded key R; and (iv) Robustness: It should not be possible to remove the key k from x without significantly degrading its visual content (removal attack) or add the key K to an image (could be real or synthetic) not generated using Go) (forgery attack).
Note that the above watermarking system W can be used either for detection or attribution. A watermark detector certifies that given image contains a valid watermark if the extracted key is sufficiently close to a known key K, i.c., if d(x, R) S e, where d is distance (c.g., Hamming) metric and is the detection threshold. On the other hand, an attribution method stores a database of models or users along with their corresponding keys. The attributor searches for the closest match between the extracted key and the stored keys in the database to determine which specific model or user generated the watermarked image.
Problem Setting: In this work, we consider the following setting with four players: (a) Model Owner owns the generative model Go (e.g., LDM) and the watermarking system W = (Mc, V₁)-
3

---

Published as a conference paper at ICLR 2026
b 11o0
1
0
0
0111 -00
0
0
D
Denoising UNet
Attantion
ResNet
ResNat
A cat sitting in thair
X
Figure 2: MOLM generation pipeline. A binary key x is mapped into a routing collection (Ec]ee(A)
that determines the active LoRA adapters (-4(*)) across ResNet and Attention blocks the frozen generator. During the diffusion sampling process, this routing implements the perturbation A (*)),, yielding the watermarked image x (g- (t) for a given prompt t. The backbone weights + remain frozen, ensuring negligible added inference cost.
When presented with a text prompt t E ' from the user, the model owner randomly samples q E Q and R. K. generate watermarked image & (G t),, k) and outputs this watermarked image to the user. (b) User: The user interacts with the generative model through an API, providing prompts and receiving watermarked images as outputs x. The user has knowledge about the generative model or the watermarking system. (c) Verifier: The watermark verifier has access only the watermark extractor Vy, Given an image x E X' and key λ (or a set of keys in attribution mode), the verifier detects if the given image contains a valid watermark (or attributes the image to a specific model or user). While we envision the verifier to a trusted third-party (with whom the model owner shares the required watermarking keys), it is also possible for the model owner to double up as the verifier. (d) Adversary: The goal of the adversary is to circumvent the watermarking system through removal and forgery attacks. While the adversary may have high- level knowledge about the nature the generative model and/or watermarking system, the adversary does not have access any of the model parameters (Φ) 7). In the removal attack, the adversary attempts to modify a valid watermarked image & so that the verifier detects as a non wane-aatmmareed image. These modifications could either be simple image transformations (e.g., cropping. rotation, brightness adjustment, or JPEG compression) or obtained through black-box adversarial attacks on the watermark detector. In the forgery attack, the adversary attempts to modify non- watermarked image (could be real or synthetically generated using some other generative model) so that the verifier detects as a watermarked image. To aid such an attack, the adversary may collect set of valid watermarked images by querying the generative model via the API.
Problem Statement: Given a generative model Gv. our goal is design a watermarking system W (see Figure [ that strongly satisfies the four required properties (imperceptibility. fidelity, verifia- bility, and robustness). Furthermore, we want to dynamically embed high-entropy keys (measured by the key size M) into the watermarked images without requiring any key-specific re-training. Specifically, our watermark encoder Ue is modeled as perturbation of parameters of the generative model, i.e., Uc (Go (q. x), = Ga+a0(e) (q, t). Given set training samples {(tn, ) )== where x,, Go (q. t,,) (here, q randomly sampled), the task is learn suitable mapping from the key to the parameter perturbation space along with the corresponding watermark extractor Vny
3.1 MOLM: MIXTURE OF LORA MARKERS
Low-Rank Adaptation (LoRA) Huu et al. 2022 is a parameter-effficent fine-tuning technique originally proposed to adapt foundation models for specific downstream tasks. Instead of updating the full set of model parameters, LoRA performs low-rank decomposition of parameter changes. Hence, it freezes the original weight matrices and injects trainable low-rank matrices parallel to the existing layers. More details about LoRA are available in Appendix A.2 In work,
4

---

Published as a conference paper at ICLR 2026
do not LoRA for traditional model adaptation, but reinterpret it mechanism to achieve key-deppndent perturbation ($()) of the generative model parameters. Furthermore, our work is inspired by the idea of learning a Mixture of LoRA Experts (MoLE) (Wu et al. 2024) to adapt large models for diverse tasks. In MoLF, multiple experts/adapters are added in parallel to the layers of the given model and the input is dynamically routed through subset of these experts based on a data-dependent gating mechanism. While MoLE focuses on data-driven expert composition, we leverage this idea to implement key-dependent expert composition.
Watermark Encoder. The given generative model Go can be considered a sequence of T. blocks F2, FL, i.e., Go = (Fr o F (.). (-), where denotes function composition. Let h denote the input to block Fr and by denote its output. We deterministically preselect subset of blocks from the set f2. ... Fr. and add P low-rank adapters
{A AA,)) ..... (
to each selected block & E [T]. Thus, (Lxx) P) low-rank adapters are added to the model architecture and learned. However, during image generation, we activate only one adapter A!aa)) for each chosen block, where 8y E (I...., PJ is the selected adapter index. Thus, the operations involved in selected block can be expressed as:
h (1; a Alt (he-))
where a is fixed scaling factor. Note that for the unselected blocks h; = Fe Let why denote the parameters of the adapter A (the thh adapter in the hh block) and Vp = {. where p E [P] and l E [L], denote the set of all additional parameters added to the generative model.
The critical aspect the proposed method is how the adapters are dynamically selected during image generation based on the given watermark key Firstly, the M-bit binary key is broken down into L non-overlapping chunks K1, K2, ... KL,, where each chunk K2 contains log, P bits. In our implementation, the value of P is always limited a power of 2 and M is set (L- loga P) bits. The chunk Ke is assigned to block f, where l E [Z], and is converted into the corresponding decimal index se E [P]. The collection {sE]EE[L] defines the key-specific routing path through the mixture of low-rank adapters. Since this routing path directly determines the watermarking signal embedded in the generated image, we refer to the proposed framework
mixture of LoRA markers (MOLM). Thus, watermark encoding is achieved by augmenting the parameters & of the generator with set of LoRA markers (1:) selected based on the watermark key H6 i.e., (1) (R) C V.
Watermark Extractor. The watermark extractor Vo is a deep neural network that takes image x as input and produces M logits u = [u]..... u.]. which are passed through a sigmoid function yield continuous approximation of the extracted key A = (u). Note that the binary extracted key A can be easily obtained by rounding the values in R to either 0 or
Training. Recall that the two main requirements of the watermarking system are imperceptibility (which also indirectly ensures fidelity if we assume that the original generative model already has high fidelity) and verifiability. Therefore, we employ two loss functions to enforce these constraints. First, we apply the perceptual loss Limp, instantiated as a feature- based reconstruction loss between the watermarked image Xn = Gw+w(r)(q t,) and its corresponding non- won-maatrmarked image Xn, Go (q, t.)) generated by the same model using the same prompt t, and latent instantiation q.
N
K
Cimp Ennn
Σ
wk ||ox(Go+vco) tw)) (px (l
(2)
N
n=1
R=1
where (sps) ] are fixed perceptual feature extractors (e.g., LPIPS) and wy are the relative weights assigned them.
The watermark extractor parameters 1) are trained using the binary cross-enropyy loss: Lier:
N
M
Lyer EI-Hl N
Σ m=1
Kim log a (um) - -(1 - Kim) log(1 (1 o (Hm)))
(3)
M
where ~ II denotes image-space augmentations of the watermarked image for robustness, u (x)x))) and Kin and u.m are the components of is and u. respectively.
5

---

Published as a conference paper at ICLR 2026
Thus, the overall training objective combines these two losses as follows:
min
(ver + A Cimp]>
(4)
W.y
where are the parameters of the LoRA markers, 1) are the watermark extractor parameters, and À 0 balances between the two losses.
Deployment. As illustrated in Figure 2 a key x deterministically selects a routing path (se (E(L) through the mixture of LoRA markers, which turn defines (*() C V. For prompt and random latent initialization q. the perturbed generator then produces the watermarked image
(5) where (x) v (x) is realized via the activated LoRA markers. The routing mask remains fixed across the denoising trajectory, ensuring that the same key always induces the same execution path and hence, produces an extractable watermark. Importantly, MOLM does not alter the backbone sampling procedure and introduces negligible cost at inference time. At the time of verification, the given image is passed through the watermark extractor V,, and the extracted key is validated.
Go+ (x))))))e (q. (q.t).
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Modern text-to-image generative models are typically implemented as diffusion models with two key components: U-Net denoising network and decoder network (often based on a variational autoencoder (VAE)). Both are composed of modular ResNet and Attention layers, which we refer to routing layers. Each ResNet block itself contains multiple convolutional sub-layers, while attention blocks implement self- or tross-attention In our main implementation, each ResNet block is treated single routing layer.
4.2 EXPERIMENTAL SETUP
We train on 10k image-text pairs Zhai et al. 2023) sampled from the MS-COCO 2014 dataset (Lin
al. 2014). For text-to-image generation evaluation, we use the PNDM scheduler (Liu al. 2022) with = 50 denoising steps. The CFG scale is set to 7.5 unless otherwise specified. For evalua- tion, we generate images from the MS-COCO test set prompts as well as captions randomly sampled from the LAION-Aesthetics dataset (Schuhmann et al. 2022). We evaluate MOLM on two diffusion models: (i) Stable Diffusion (SD) vI.3 (Rombach et al.J 2022). generating 512 x 512 images, and (ii) FLUX (Labs] 2024), recent large-scale diffusion model generating 1024 1024 images. We additionally report qualitative results on SD v3.5. By default, we perturb the parameters of ResNet blocks in the VAE decoder with key-dependent LoRA adapters, i.c., each block is treated as sin- gle routing layer. Unless otherwise stated, we activate 14 such residual routing layers with P adapters per layer (corresponding 2 bits/layer), resulting in 28-bit keys. We also experimented with perturbing the U-Net parameters, which contains 22 ResNet blocks, 16 cross-attention layers, and 16 self-attention layers. This configuration yields total of 108 bits. However, discussed in the Appendix [.I this led to noticeable degradation in generation quality. We leave perturba- tion of U-Net parameters for future exploration. as larger key sizes must be balanced with fidelity constraints. More details on the experimental setup can be found in the Appendix B.
4.3 EVALUATION METRICS
For fidelity, we compute PSNR and SSIM (Wang et al., 2004) between images generated with and without watermarking. To assess distributional quality, we report the Fréchet Inception Distance (FID) (Heusel et al., 2017) between generated samples and real images from the MS-COCC valida- tion set. For key recovery, report the average bit accuracy, defined as the proportion of correctly decoded key bits across watermarked images:
Bit Accuracy s(K, R)
M 1 1kg = Ky]. j=1
where s(K, R)
Σ
(6)
M

---

Published as a conference paper at ICLR 2026
For watermark detection, similar to (Fernandez et al. 2023), perform hypothesis test based on the number of matching bits sK,, K). The input is declared a valid watermarked image if s(K, R) 2 T. Under Hn (no watermark), achieve a target FPR (e.g., 1%), and report TPR @ %FPR on watermarked images. For our default M = 28 setting, this corresponds
[An 0.5), and FPR(T) P(s 2 Ho). We set T to
20 matching bits.
For the attribution task, we consider database of users, each with an M-bit key k( E 10.1 We match the key & extracted from an image with each registered key by computing the number of matching bits, s; Σ [R, "i ] An image is attributed to user i = arg max; 8; if max; s; 2 T. Otherwise, it is rejected as non- wotewatrmarred. We evaluate attribution performance using three complementary metrics: (i) False positive rate (the proportion of non-watermarked im- ages that are not rejected), (ii) True accept rate (the proportion of legitimate watermarked images that are not rejected) and (iii) Conditional attribution accuracy (the fraction of watermarked images that correctly matched to their originating user, conditional on the fact that they are not rejected).
Table I: Detection and robustness results. We report fidelity on undistorted watermarked images
(FID J. SSIM 1. PSNR f) and detection robustness under common distortions. Bit Accuracy 1 and TPR @1 1 % FPR are used as the evaluation metrics for the bit-recovery (Top) and detection-only methods (Bottom), respectively. Values highlighted in Orange denote the difference with respect to the corresponding non-watermarked images. Within each column, bold indicates the best and the second- best value. Robustness results are averaged over two distortion severity levels (see App. E).
Unioond
Crpp
Ree
Ris
Bright
JPDX
AloNeennory Mestoda Ascarwsi
Statt Stpeatine
28.5 (-0-))
0.85
27.4
(1:51)
0.91
(1.56
0.72
0.315
11:01
MS-CO00
WOLAA
AqualaRAA
27.8
30.5 (-3-4)
01733
0.63
24.0
22.3
12.35
11:08
0.96
11:11
0.85
[1.55
0.73
11.91
0.88
0.72
0.56
0.94
Stakee Stgeatur
(4.01-5.5)
0.80
28.0
0.99
0.97
0.57
0.72
0.90
0.59
LAION
WOLAF
05.8 (10.4)
0.70
24.5
0.98
0.91
0.65
0.58
0.94
1..0
32
FRIIX
MS-C000
Statte Sureature MILMM (3sm))
25.8
25.2 (-0.00
(-144)
0.95
0.85
$2.5
21.4
01.88
0.33
0.95
11:01
0.76
0156
0160
0.67
0.97
01502
1.77
26
(Cam)
Deveciiee Onty Mestedd (YPM PSE
Tree-kin
35.9 (-3.00
0:45
13111
01:89
0.59
0.58
11:99
0.51
0.59
M5-COCO
BUBIN
2533
0.75
1.00
0.99
0.503
SD
MELAI (Ulm)))
27.7 (-3.4)
0.77
23.5
1.00
0.35
0.56
0.98
0.30
03.99
12.0
LAION
Geauoe-Shating
66441 (-7.0)
0.50
0.80
0.599
MIX M (Cle))
685 (-0-1)
0.65
23.5
0.89
0.35
0.88
0.91
0.30
0.99
PLUX
MS-COLU
MULM (Utm))
(-3.0-
0.35
11.1
0.35
01308
0152
0.96
0.37
Poussker
Metiadtr
20BL
11.01
0.38
013
(350)
0.200.5m
(00022200
0.882.8953 0.35
0.102_195)
110)
VINE- Aucanary
20,00 (41.0
0.99
96.1
1,00
01.22
1.00
1.00
4.4 DETECTION AND ROBUSTNESS RESULTS
Table presents comparison of MOLM with representative watermarking baselines on Stable Dif- fusion v1.5 (SD) and FLUX. We group prior methods into two categorics: Bit-Recovery methods, which embed explicit binary keys and are evaluated based on bit accuracy, namely Stable Signa- ture (Fernandez et al. 2023). AquaLoRA (Feng et al.] 2024). and WOUAF (Kim et al.) 2024); and Detection-Only methods, which provide binary watermark presence/absence decisions and are evaluated based on TPR @1 1FPR, namely Tree-Ring (Wen et al.] 2023), ROBIN (Huang et al.) 2025), and Gaussian-Shading (Yang et al. (024b)). MOLM achieves strong detection performance with bit accuracy above 0.98 on undistorted images and robust key recovery across all tested dis- tortions. Details about the distortions considered are in Appendix B All results in Table [ are averaged over two severity levels for each distortion. MOLM retains 0.91 accuracy under cropping and 0.89 under JPEG compression. contrast, Stable Signature suffers significant drops under rota- tion (0.56) and resizing (0.72), while AquaLoRA exhibits poor rotation robustness (0.45). WOUAF performs competitively under certain distortions but at a higher training cost (Table 4.. These re- sults highlight the advantage of MOLM's routing-based embedding in maintaining consistent key recovery across perturbations. MOLM also compares favorably to detection-onle baselines. Tree- Ring and Gaussian- -hassiiinnhhding achieve strong TPR for undistorted or mild distorted images. How- ever, both require full inversion during verification. ROBIN introduces adversarial optimization but struggles under rotation (0.64). By contrast, MOLM achieves high TPR (2 0.95) while simulta-
7

---

Published as a conference paper at ICLR 2026
Table 2: Attribution results. True accept rate the fraction of watermarked images that not rejected. Conditional attribution accuracy is the fraction of watermarked images that are correctly attributed to the true user, conditional on the fact that they are not rejected.
Distortion
True accept rate (%) 1
Conditional attribution accuracy (%)
None (undistorted)
98.92
100.00
JPEG (q 80)
96.87
100.00
JPEG (q 50)
88.76
100.00
Crop (5%)
89.09
99.99
Crop (10%)
39.92
99.97
Resize (0.7x)
93.68
100.00
Resize (0.3x)
0.96
97.92
Rotation (25%)
90.53
100.00
Rotation (90%)
0.03
66.67
Brightness (x 1x5)
85.16
100.00
Brightness (x2.
51.79
99.98
neously enabling explicit key recovery. On FLUX, MOLM maintains consistent detection accuracy (TPR 0.96), confirming that it generalizes beyond Stable Diffusion. Similarly, testing on LAION Aesthetics with models trained on MS-COCC shows that MOLM's watermark remains recoverable under distribution shifts. This suggests that MOLM is not tied single dataset archi- tecture, but instead provides a transferable mechanism for embedding and detecting watermarks in generative models. We also report results for post-hoc watermarking methods designed for arbitrary images, TrustMark (Bui et al.) (202)) and VINE (Lu et al.] (024).. Overall, while these post-hoc methods can be robust to mild photometric distortions, they remain vulnerable to geometric attacks. In Appendix B.2 we highlight computational efficiency. MOLM trains within ~ day on a single 100 with no per-key retraining. At inference time, MOLM introduces negligible overhead beyond the frozen generator, ensuring practical deployability.
4.5 ATTRIBUTION RESULTS
We simulate an attribution scenario with 1,000 registered users. For each user, we generate 10 wa- termarked images using their assigned M 28-bit key, yielding a test set of 10,000 in-database images. We set the acceptance threshold т = 27 (i.e., at most one bit error permitted) to target global false positive rate of 10 Under this operating point, MOLM achieves 0.02% false posi- tive rate when tested on 20,000 non-watere watermarked images. MOLM also achieves 98.9 28.92% true accept rate undistorted watermarked images and conditional attribution accuracy of 100%, indicating that the primary source of error is rejection rather than incorrect attribution. This demonstrates that MOLM reliably distinguishes between watermarked and non-watermarked images while maintain- ing near-perfect user attribution when watermark is detected. We evaluate robustness by applying the same distortion suite used in Section 4.4 Table 2 reports the accept rate and conditional attribution accuracy for each distortion type. Common image edits preserve attribution integrity: JPEG compression (quality 80), mild cropping (5%), resizing (0.7x), moderate rotation (25"), and brightness adjustment (x 1.5, x2.0) all achieve acceptance rates above 85% with near-perfect accu- racy among accepted images. Severe transformations, extreme resizing (0.3 x) and 90" rotation are rejected, with acceptance rates below 1%. Our choice of r 27 (one-bit tolerance) is intention- ally conservative, prioritizing low false positive rates for high-stakes applications. In deployments where false negatives are more costly, the threshold can be relaxed to 26 or lower, increasing acceptance rates at the cost a higher false positive rate.
4.6 IMAGE GENERATION QUALITY
We assess the impact of watermarking the perceptual quality of generated images. Quantitatively, we achieve FID values comparable to or lower than existing watermarking methods (Table [ in- dicating minimal degradation. Importantly, the differences relative to vanilla Stable Diffusion
8

---

Published as a conference paper at ICLR 2026
(a) MS-COCO
(b) LAION Aesthetics MOLM
SD
MOLM
SD
MOLM
SD
SD
MOLM
Figure 3: Image generation quality. Visual comparison between Stable Diffusion (SD) and MOLM on MS-COCO (left four columns) and LAION Aesthetics (right four columns). For each prompt, we show the original Stable Diffusion image and the corresponding watermarked image. MOLM preserves high image quality.
Robustness under compression, diffusion, and adversarial removal attacks. We report Bit Accuracy (1) and FID (4). q is the compression quality (lower stronger), "steps" denotes denoising iterations, and is the MSE bound for PGD attacks.
Table 3:
Attack
Params
No-Aug
Aug- Trained
Bit Acc.
FII
Bit Acc.
FID
BMSHJ2018 Cheng2020
q 1,4,8
0.50/0.77/00.0
2/27.8
0.61/0.95 90.99
80.6/28.6/29
q 1,3,6
0.70/0.82/0. 82/0.33
29.2/27.7/278
0.94/0.95/0 10//0.95/0000 05/0.97
30.1/28.9/25
Diffusion Regen.
steps 30,
0.72/0.69/0.. 69/0.2
27.8/28
0.85//0855000 00.82
30.2/29.9/31.
Adversarial (PGD)
10,1007,100
0.93//0.8 /0.66 /0.60
27.7/27.8/2.44
1.00/0.99/0.90
28.4/28.6/29.
small (S 1.5 FID), with no systematic drop in SSIM or PSNR. Qualitative comparisons are shown
in Figure 3 On both MS-COCO and LAION Aesthetics, MOLM outputs remain visually indistin-
guishable from baseline Stable Diffusion generations. The routing mechanism does not introduce noticeable artifacts. Additional visual examples are provided in the Appendix E
4.7
WATERMARK ROBUSTNESS
4.7.1
COMPRESSION AND DIFUUSIO-BASTR REMOVAL. ATTACKS
Recent work (Zhao al. 2024b) shows that invisible image watermarks be removed via com- pression and diffusion-basee regeneration attacks. We evaluate three families of attacks: two learned compression models (bmshj2018 (Ballé et al. 2018) and cheng2020 (Cheng et al. 2020)) and diffusion/noise attacker, and report both bit accuracy and FID of the attacked images in Table 3 Results are shown for an extractor trained without augmentation ("no-aug")) and with augmentation "'aug-trained"" Augmentation training enhances robustness to compression and diffusion removal attacks.
4.7.2 AVERAGING ATTACKS
Yang al. (Yang al. 2024a) recently proposed an effective attack against several modern wa- termarks. Their key observation is that many watermarking schemes across multiple generations maintain consistent watermark while the image content varies. By averaging k generated im- ages, an adversary can estimate and subtract the watermark for removal, or induce watermark signals into clean images for forgery. We follow their evaluation protocol by generating sets of k {5. 10, 20, 50, 100, 200. 500, 1000, 2000, 5000} images and computing averages in both grey- box and black-box settings. We then test two attack variants: Removal and Forgery. We also consider a "same-message" setting where all averaged images embed the identical key. Figures 4 pendix C.I compare MOLM with WOUAF under averaging attacks in both the same- mese-eessage and heterogeneous message cases. Under forgery attempts, MOLM remains at chance accuracy (= 0.5) even at higher & values, even in the same-messe same-message case. Under removal attempts, MOLM maintains high bit accuracy (2 0.96) even when averaging up 5000 images.
and (Ap-

---

Published as a conference paper at ICLR 2026
4.7.3 ADVERSARIAL ATTACKS ON KEY EXTRACTION (WHITE-BOX)
To evaluate robustness against adaptive adversaries, we evaluate white-box attacks that directly opti- mize the input image against the watermark extractor. The adversary has full access to the extractor parameters and detection threshold т, and may perturb an image subject to a perceptual constraint. In our experiments, this constraint is expressed an upper bound on the mean squared error (MSE) with respect to a reference image; equivalently, it can be stated as a lower bound on PSNR. Details on this attack can be found in the Appendix C.2 As shown in Table 3 adversarial PGD attacks under an MSE constraint e significantly reduce key accuracy when no augmentation is used during training (dropping from 0.93 0.60 increases). With augmentation, however, MOLM remains robust, achieving > 0.96 accuracy even for e values corresponding to PSNR & 10. Constraints looser than this threshold lead to visible distortions, indicating that successful removal requires per- ceptual degradation. We also explore mall-knowledge adversary who retrains the generative model independently: the results in Appendix accuracy of & 0.5 when using our extractor, L..., indistinguishable from random guessing.
C,3 show that such attacker- -eenerrtd images yield a bit
4.8
ABLATION STUDIES
Key Capacity. The capacity of MOLM is determined by the number of blocks L selected from the backbone G3 and the number of available adapter paths P per block. Since choosing among P adapters requires log, P bits, the total key size is M I. logz P. In our default configuration, we perturb the whole = 14 ResNet blocks the VAE decoder with P = 4 adapters each, yielding M 14 2 28 bits per image. Each ResNet block treated as a single routing layer, where one index sy determines the active convolutional adapters within the block. We also investigate several extensions: (i) Independent bits per convolution, which doubles capacity by assigning separate bits to the two convolutional layers inside each ResNet block; (ii) Including attention blocks, where rout- ing cross- and self- attfntttentinn layers increases L and thus the total key size; (iii) Expanding adapter paths, e.g., increasing P from 4 to 8 so that each block encodes log, 8 3 bits. Appendix D.2 re- ports the empirical trade-off's between capacity, bit accuracy, and fidelity under these configurations.
Mapping Effect. To analyze how key bits are distributed across routing adapters, we conduct layer-wise weight randomization (details in the Appendix D.3). Averaging results over 100 random prompts reveals that the mapping is largely distributed: many routing layers influence multiple bits with intermediate probability, and individual bits are affected by several layers. While certain bits sensitive, there is strict one-to-one correspondence between adapters and bits. Instead, the key is redundantly encoded across layers, which enhances robustness, as perturbing any single adapter does not deterministically erase the watermark.
Sampling Configurations. We evaluate watermark robustness under variations in generation set- tings, including scheduler type, number of denoising steps, and CFG scale. Results in Appendix[D.. show consistently high bit accuracy (0.94-0.96) across all configurations, with only modest varia- tion in FID (25.8-28.5). This demonstrates that MOLM does not depend on a fixed sampler for detection.
Adapter Rank. Finally, we investigate the effect of LoRA adapter rank on watermark fidelity and bit accuracy. Table in Appendix D.4.I shows that higher ranks yield stronger bit recovery and fidelity, while very low ranks (8) fail sustain reliable decoding. This highlights trade-off between efficiency and watermark robustness.
5 CONCLUSION
We presented a general watermarking framework that views watermark embedding as key- dependent parameter perturbation of frozen generative model, and instantiated it with MOLM, a routing-based design that uses lightweight LoRA adapters as watermark carriers. By encoding keys through deterministic adapter selection across decoder blocks. Across Stable Diffusion and FLUX, MOLM preserves generation quality and achieves reliable key extraction, remaining robust to common image edits, compression, liffusion-basee regeneration, averaging attacks, and adap- tive attacks on the extractor. Together, these results support MOLM as a practical watermarking mechanism for modern text-to-image systems.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et Waves: Benchmarking the robustness of image watermarks. arXiv preprint arXiv:2401.0833 2024.
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational im- age compression with scale hyperprior. International Conference on Learning Representations (ICLR), 2018.
Tu Bui, Shruti Agarwal, and John Collomosse. Trustmark: Universal watermarking for arbitrary resolution images. arXiv preprint arXiv:2311.18297 2023.
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In IEEE/CVF conference computer vision and pattern recognition (CVPR), pp. 7939-7948, 2020.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANS on image synthesis. Ad- vances neural information processing systems (NeurIPS), 2021.
Jianwei Fei, Zhihua Xia, Benedetta Tondi, and Mauro Barni. Supervised gan watermarking for intellectual property protection. In IEEE International Workshop on Information Forensics and Security (WIFS), 2022.
Weitao Feng, Wenbo Zhou, Jiyan He, Jie Zhang. Tianyi Wei, Guanlin Li, Tianwei Zhang, Weiming Zhang, and Nenghai Yu. Aqualora: Toward white-box protection for customized stable diffusion models via watermark lora. The International Conference Machine Learning (ICML), 2024.
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. International Conference Computer Vision (ICCV), 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANS trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems (NeurIPS), 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. Workshop on Deep Generative Models and Downstream Applications (NeurIPS), 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems (NeurIPS), 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: I Low-rank adaptation of large language models. International Confer- on Learning Representations (ICLR), 2022.
Huayang Huang, Yu Wu, and Qian Wang. ROBIN: Robust and invisible watermarks for diffu- sion models with adversarial optimization. Advances in Neural Information Processing Systems (NeurIPS), 2025.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. In ACM Conference on Computer and Communications Security (CCS), 2023.
Changhoon Kim, Kyle Min, Maitreya Patel. Sheng Cheng, and Yezhou Yang. Wouaf: Weight modulation for user attribution and fingerprinting in text-to-image diffusion models. In IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Black Forest Labs. Flux.
com/ black foret--ooostttee Latresttlabbs/EIe Laabs//Elx 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference Computer Vision (ECCV), 2014.

---

Published as a conference paper at ICLR 2026
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations (ICLR), 2022.
Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, and Yang Zhang. Watermarking diffusion model.
arXiv preprint arXiv:2305. /2502, 2023.
Shilin Lu, Zihan Zhou, Jiayou Lu, Yuanzhi Zhu, and Adams Wii-Kin Kong. Robust watermarking using generative priors against image editing: From benchmarking to advances. International Conference on Learning Representations (ICLR), 2024.
Zheling Meng, Bo Peng, and Jing Dong. Latent watermark: Inject and detect watermarks in latent diffusion space. IEEE Transactions on Multimedia, 2024.
Sen Peng, Yufei Chen, Cong Wang, and Xiaohua Jia. Protecting the intellectual property of diffu- sion models by the watermark diffusion process. International Conference on Web Information Systems Engineering ( (WISE), 2023.
Ahmad Rezaci, Mohammad Akbari, Saeed Ranjbar Alvar, Arezou Fatemi, and Yong Zhang. Lawa: Using latent space for in-generation image watermarking. European Conference on Computer Vision (ECCV), 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In JEEE/CVF conference on computer vision and paitern recognition (CVPR), 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In Medical image computing and computer-assisted intervention (MIC- CAI), 2015.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aber- man. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of Al-image detectors: Fundamental limits and practical attacks. In International Conference on Learning Representations (ICLR), 2024.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- formattion processing systems (NeurIPS), 2022.
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations (ICLR), 2021.
Hossein Souri, Liam Fowl, Rama Chellappa, Micah Goldblum, and Tom Goldstein. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Tao Wang, Yushu Zhang, Shuren Qi, Ruoyu Zhao. Zhihua Xia, and Jian Weng. Security and privacy on generative data in aige: A survey. ACM Computing Surveys., 2024a.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 2004.
Zilan Wang, Junfeng Guo. Jiacheng Zhu, Yiming Li, Heng Huang, Muhao Chen, and Zhengzhong Tu. Sleepermark: Towards robust watermark against fine-tuning text-to-image diffusion models. IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024b.
12

---

Published as a conference paper at ICLR 2026
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-rings watermarks: in- visible fingerprints for diffusion images. In Advances in neural information processing systems (NeurIPS), 2023.
Xun Wu, Shaohan Huang, and Furu Wei. Mixture of loRA experts. In International Conference on Learning Representations (ICLR), 2024.
Cheng Xiong, Chuan Qin, Guorui Feng, and Xinpeng Zhang. Flexible and secure watermarking for latent diffusion model. ACM International Conference on Multimedia (ACM MULTIMEDIA), 2023.
Pei Yang, Hai Ci, Yiren Song, and Mike Zheng Shou. Can simple averaging defeat modern water- marks? Advances in Neural Information Processing Systems (NewrIPS), 2024a
Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, and Nenghai Yu. Gaussian shad- ing: Provable performance-lossles image watermarking for diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024b.
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial fingerprinting for genera- tive models: Rooting deepfake attribution in training data. In /EEE/CVF International conference computer vision (ICCV), 2021.
Yu Zeng, Mo Zhou, Yuan Xue, and Vishal M Patel. Securing deep generative models with universal adversarial signature. arXiv preprint ar0iv:2305. 16310, 2023.
Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. Text-to-image diffusion models can be easily backdoored through multimodal data poisoning. In Proceedings of the s1 ACM International Conference on Multimedia, 2023.
Xuandong Zhao, Sam Gunn, Miranda Christ, Jaiden Fairoze, Andres Fabrega, Nicholas Carlini, Sanjam Garg, Sanghyun Hong, Milad Nasr, Florian Tramer, et al. Sok: Watermarking for ai- generated content. arXiv preprint arXiv:2411.14477 2024a.
Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai. Advances in Neural Information Processing Systems (NeurIPS), 2024b.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. recipe for watermarking diffusion models. arXiv preprint arXiv:23033.0133 2023.
Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. Hidden: Hiding data with deep networks. In European Conference on Computer Vision (ECCV), 2018.
13

---

Contents
Introduction
2
Background
2.1
Diffusion Models
2.2
Watermarking of Generative Models
Proposed Method
MOLM: Mixture of LoRA Markers
Experiments
4.1
Implementation Details
4.2
Experimental Setup
4.3
Evaluation Metrics
4.4
Detection and Robustness Results
4.5
Attribution Results
4.6
Image Generation Quality
4.7
Watermark Robustness
4.7.1
Compression and Diflusion-bassee Removal Attacks
4.7.2
Averaging Attacks
4.7.3
Adversarial Attacks on Key Extraction (White-box)
10
Ablation Studies].
10
Conclusion
10
Techniques
15
A.I Diffusion Models
15
A.2 Low-Rank Adaptation (LoRA)
15
B Sctup
15
B.1 Experimental Setup
15
B.2
Efficiency
16
C
Robustness
16
Averaging Attacks
16
C.2
Adversarial Attacks on Key Extraction (White-box
16
C.3
Full Knowledge Attack Scenario
17
D
Ablations
17
D.I
Perturbing the U-Net
17
D.2
Key Capacity|
17
D.3
Mapping Effect
18
D.4
Sampling Configurations
18
D.4.1
Adapter Rank
19
More Visual Examples
19
14

---

Published as a conference paper at ICLR 2026
A TECHNIQUES
A.1 DIFFUSION MODELS
The framework of diffusion models is based on two complementary processes: forward process that progressively adds Gaussian noise to a data sample, and reverse process that learns to recover the original sample by removing noise. This process can be formulated as a Markov chain, where the forward process is defined as:
(xx//=11))= -N)t v - B, 3,,,
where 3, represents noise schedule controlling amount of noise added each step (Sohl-) Dickstein et al. 2015). The reverse process is parameterized by neural network trained via varia- tional inference, approximating:
pe(x)-1lz)) = - t), Ee(T,, t)).
(8)
The model trained to minimize the variational lower bound on the negative log log-liclihood. opti- mizing series of KLL-divergence terms ensure stable sample reconstruction (Ho et al. 2020).
While diffusion models offer high sample quality, their main limitation lies slow sampling, as generating image requires iterating through many timesteps. To accelerate sampling, Denois- ing Diffusion Implicit Models (DDIMs) introduce non-Markovian reformulation of the reverse process, enabling deterministic sampling with fewer steps, given by:
h = v0 + v1 Qt-1
Here, controls the noise schedule, allowing the model to balance between speed and sample quality (Song et al. 2021).
Recent advancements have further improved diffusion models by introducing guidance mechanisms control sample generation. Classifier Guidance (Dhariwal & Nichol 202T) incorporates an exter-
nal classifier to steer the generation process, while Classifier Guidance (Ho & Salimans (021)) removes dependency on external classifiers by training the model to generate both conditional and unconditional outputs, allowing for controlled sampling.
To improve efficiency, Latent Diffusion Models (LDMs) (Rombach al. 2022 reduce compu- tational cost by applying diffusion in compressed latent space rather than pixel space. LDMs leverage an encoder-decoder architecture, where data is first mapped to a lower-dimensional repre- sentation:
= E(x,, ã= = D()),
(10)
and the diffusion process operates on the latent 2 instead of the full-resolution image. This approach significantly reduces memory and computational demands while preserving high-quality outputs. The U-Net architectures (Ronneberger al. 2015) play a crucial role in diffusion models by provid- ing multi-scale feature representations, balancing fine-grained detail retention with computational efficiency.
A.2 LOW-RANK ADAPTATION (LORA)
For linear or convolutional layer with weight matrix W E Rdaadd, LoRA introduces a residual term WÍORA A. where A E Rx and B Rduxx trainable parameters with min(dm; d. The modified output becomes:
We
We + a BAx,
(11)
where a is a scaling factor. This design enables fine-tuning with significantly fewer parameters while preserving the pre-trained model's behavior.
B SETUP
B. EXPERIMENTAL SETUP
The LoRA adapters and key extractor are trained for 12K steps using the AdamW optimizer with learning rate of 1 x 10-1, batch size 4, and weight decay 0.01. Lver is implemented as binary
15

---

Published as a conference paper at ICLR 2026
cross-entropy between the extractor's predicted logits and the ground -tuth key bits, while Limp is an LPIPS loss (VGG backbone) computed between the generated watermarked image and the generated non-watermarked image. The loss is balanced with À = 1.0.
B.2 EFFICIENCY
Table 4 provides detailed training and inference costs across baselines.
Table 4: Training and inference cost across methods. We report one-time pre-training cost, and per-key training cost.
Method
Pre-train Cost
Per-key Cost
Stable Signature
day
min/key
AquaLoRA
2.5 days
None
WOUAF
3.5 days
None
Tree-Ring
None
N/A
ROBIN
per prompt
N/A
GaussianSShading
None
N/A
MOLM (Ours)
day
None
C ROBUSTNESS
C.1 AVERAGING ATTACKS
Figure 4 provides the comparison between MOLM and WOUAF under averaging attacks in the same-message setting. Figure 5 provides MOLM bit accuracy under averaging attacks in the heterogeneous--messge setting.
Forgery Attack (Averaging)
Removal Attack (Averaging)
0.625
1.0
0.600
0.9
Bit Accuracy
0.575
0.8
0.550
0.7
0.525
0.6
0.500
0.5
5
10
20
50
100
200
500 10002000 5000
5
10
20
50
100
200
500 10002000 5000
Number images averaged (k)
Number images averaged (k)
MOLM Grey-box
MOLM Black-box
WOUAF Grey-box
WOUAF Black-box
Figure 4: Averaging attack evaluation: MOLM vs. WOUAF (same message). (Left) Forgery attack.
MOLM stays the chance level (~ 0.5). (Right) Removal attack. MOLM achieves accuracy
2 0.96, whereas WOUAF degrades to 0.85-0.90.
C.2 ADVERSARIAL. ATTACKS ON KEY EXTRACTION (WHITE-BOX)
We consider an adaptive white-box adversary that perturbs watermarked image to induce fail- ure the verifier while remaining within a perceptual budget. Given a watermarked image x E [0, 1|xxXX and an MSE budget the adversarial example Xadv is constrained to
1 N ||Xadv x1|2
MSE (adv x) S E,
MSE Xadv, x)
(12)
16

---

Published as a conference paper at ICLR 2026
Removal Attack (Averaging)
0.9
0.8
0.7
Grey-box
Black box
0.5
5
10
20
50
100
20C
500
1000
2000 5000
Number of images averaged
Figure 5: Averaging removal attack evaluation: MOLM (heterogeneous message).
where N 3HW. The adversary aims to make V,, uninformative (i.e., outputs close random guessing). We optimize the removal loss
M Σ (pe(Xudv) -)²²
frem (Xadv)
(13)
We solve the problem using (2-projected gradient descent (PGD). An attack is deemed successful the verifier's similarity score falls below the detection threshold.
C.3 FULL KNOWLEDGE ATTACK SCENARIO
We consider a white-box box adversary who knows the entire MOLM pipeline (architecture, losses, and training hyperparameters) and is allowed to retrain the generative model on different subset of the same training data, using different random seeds. The attacker trains their own version and then evaluates the attacker-generated images using watermark extractor. Across our experiments, extractor recovers bits from attacker images with average bit-accuracy = 0.5, i.e., no better than random guessing. We attribute this failure to the fact that the adapters and the extractor trained end-to-end, the extractor learns to decode the specific routing-induced activation patterns produced by that training run. Small changes in seed, data order, subset yield different routing statistics, so an independently trained attacker produces images whose routing signature the defender's extractor cannot decode.
D ABLATIONS
D.1 PERTURBING THE U-NET
In addition to perturbing the VAE decoder, we experimented with perturbing the U-Net. The U- Net in Stable Diffusion v1.5 consists of 22 ResNet blocks, 16 cross-attention layers, and 16 self- attention layers. Treating each of these modules as routing layers, with / = 4 adapters per layer (2 bits each), results in total of 108 key bits per image. The image quality degraded significantly. FID increased by 4.15 points, and human inspection revealed a change in the content compared non-we watermarked images as shown in Figure O especially around fine textures and edges. This suggests that the heavier intervention in the U-Net perturbs denoising dynamics more strongly than in the decoder, amplifying the perceptual cost. These results indicate that while perturbing the U- Net offers higher key capacity, it compromises fidelity. This trade-off highlights the importance of carefully balancing watermark capacity against image quality.
D.2 KEY CAPACITY
Table reports the quantitative trade-offs between key size, bit accuracy, detection performance (TPR@ @ 1&FPR), and image quality (FID) under different architectural configurations. Table shows visual examples.
17

---

Published as a conference paper at ICLR 2026
Stable Diffusion
Decoder MOLM (28-bit)
U-Net MOLM (108-bit)
Figure 6: Perturbing the U-Net. Comparison between Stable Diffusion (left), MOLM with decoder parameters perturbation (28 bits, middle), and MOLM with U-Net perturbation (108 bits, right). While capacity increases, perturbing the U-Net introduces visible artifacts and degrades fidelity.
Table Key capacity scaling in MOLM. We report the effective key size (bits), Bit Accuracy (f),
TPR @ 1FFP (1), and FID ()) for different architectural configurations.
Configuration
Key Size
Bit Accuracy
TPR@I%FPR
FID
Independent bits per convolution
56
0.89
0.99
27.9
Including attention blocks
30
0.92
0.99
28.2
paths per layer (3 bits/block)
42
0.90
0.99
27.3
D.3 MAPPING EFFECT
Starting from a watermarked image &, we iterate over routing blocks. For each block b we identify the active adapter path from the routing mask, temporarily replace its weights with random values of the same shape, regenerate the image using the unchanged routing mask, and run the trained extractor obtain predicted key. The original weights are then restored before moving to the next block. Let E {0, 1 denote the embedded key and ((() the extractor output after perturbing block We define the binary flip matrix
Fb. 1(ke + kz}.
(14)
where Fby = 1 indicates that bit j flipped when block was randomized. Figure 7 shows the flip heatmaps
(a) Key
Key
Figure 7: Bit-flip heatmaps for two different embedded keys, each averaged over 100 random prompts. Rows correspond routing blocks columns key bits; brighter cells indicate higher probability that the bit flips when the corresponding block is perturbed.
D.4 SAMPLING CONFIGURATIONS
Table 7 reports the results. Across all tested configurations, MOLM consistently achieves high bit accuracy (0.94-0.96), demonstrating robustness changes in the sampling procedure.
18

---

Published as a conference paper at ICLR 2026
Table 6: Qualitative examples for key capacity configurations in MOLM. Rows are different
sampled outputs for the same MS-COCO prompt; columns are configurations.
SD (mo WM)
MOLM: Indep. bits
MOLM: +Athn
MOLM: paths
D.4.1 ADAPTER RANK
Table 7 rreports the FID of watermarked images and the bit accuracy of the trained extractor for ranks (64, 32, 16, 8}. Higher ranks yield a stronger watermark recovery (higher bit accuracy), while very low ranks (8) cannot sustain reliable bit decoding.
E
MORE VISUAL EXAMPLES
and provide extended visual examples covering different models, distortion scenarios, comparisons with existing methods, and ablations on sampling configurations and LoRA rank.
Tables 8
Table Sampling and ablation studies. Left: Effect of sampling configurations on MOLM. Bit
accuracy remains consistently high (2 0.94), while FID varies modestly across settings. Right: Ablation on the rank of LoRA adapters. Higher ranks improve bit recovery but slightly increase FID.
Factor
Setting
Bit Accuracy t
FID
DDIM
0.96
27.4
DPM-S
0.96
28.5
Sampler
Rank
FID J
Bit Accuracy
DPM-M
0.95
27.3
Euler
0.94
26.5
64
27.7
0.98
32
28.2
0.96
15
0.95
27.3
16
29.5
0.91
Steps
25
0.95
25.8
34.8
0.75
100
0.95
27.4
5.0
0.96
25.9
CFG
10.0
0.95
27.6
19

---

Published as a conference paper at ICLR 2026
FLUX
MOLM
for the cleate our ti
for the cleate our Seer
Figure 8: Qualitative comparison on FLUX. Side-by-side generations from FLUX (left) and MOLM (right) on MS COCO prompts. MOLM preserves the visual fidelity of FLUX while em- bedding recoverable watermark.
20

---

Published as a conference paper at ICLR 2026
SD 3.5
MOLM
Figure 9: Qualitative comparison on SD 3.5. Side-by-side generations from SD 3.5 (left) and MOLM (right) on MS-COCO prompts. MOLM preserves the visual fidelity of SD 3.5 while em- bedding recoverable watermark.
21

---

Published as a conference paper at ICLR 2026
Original
Crop (0.1)
Crop (0.5)
Rotate (25%)
Rotate (90")
Resize (0.3)
Resize (0.7)
Brightness (x 1.5)
Brighiness (x2.0)
JPEG (q=80)
JPEG (q=50)
Figure 10: Effect of distortions. MOLM generations from MS COCO under variety of distortions
applicd during evaluation. These include geometric transformations such as random cropping (10% and 50%) and rotations (25", 90%), resizing different scales (0.3x, 0.7x), photometric changes such as brightness adjustment (x1.5, x2.0), and lossy compression using JPEG (quality and 50).
SD
Stable Signature
AquaLoRA
WOUAF
Tree-Ring
-hassian-Shading
MOLM
Figure 11: Comparison with existing methods. Visual examples from MS COCO comparing base- line Stable Diffusion (no watermark). Stable Signature, AquaLoRA, WOUAF, Tree-Ring, Gaussian- Shading, and our proposed MOLM. MOLM maintains fidelity while embedding robust and recov- erable watermark.
22

---

Published as a conference paper at ICLR 2026
Classifier-Fres Guidance (CFG)
CFG = 5.0
CFG = 10.0
Number of Sampling Steps
Steps 15
Steps =
Steps 100
Different Samplers DPM-Solver
DDIM
Euler
Figure 12: Sampling configurations. Qualitative examples from MS COCO showing MOLM gen- erations under the sampling settings reported in Table CFG scale (top), number of steps (middle), and sampler type (bottom). Across all settings, watermarks remain consistently recoverable while maintaining fidelity.
Rank = 64
Rank 32
Rank 16
Rank = 8
Figure 13: LoRA rank ablation. Qualitative examples from MS COCO showing MOLM genera-
tions with different LoRA adapter ranks. Higher ranks preserve watermark recovery more reliably.
23


Reviews:
[{"review_id": "7Q3fljGgFA", "reviewer": "Reviewer_yLkc", "summary": "This paper addresses verifying and attributing AI-generated images. The authors propose a watermarking framework inspired by the Mixture of LoRA Experts. They formulate the problem as key-dependent perturbations of frozen generative models, where binary keys are used to route among a mixture of LoRA adapters inserted into the generative model. In addition, the authors consider imperceptibility, fidelity, verifiability, and robustness by incorporating the L_imp​ and L_ver​ losses.", "questions": "The authors mention that the proposed method is inspired by Mixture of LoRA Experts (MoLE), but the motivation behind this choice is unclear. It would be helpful to explain the intuition for why MoLE's structure is beneficial for the watermarking problem, beyond the fact that it can be adapted to it.\n\nDiffusion version 3.5 Large (8B) has already been released on Hugging Face. However, this paper still evaluates on version 1.5. It would be more convincing to include results from a more recent version to demonstrate scalability and relevance.\n\nIn Table 1, the bolded values appear to indicate the best and second-best results within each column, but this should be explicitly explained in the table caption. The meaning of the orange-highlighted values should also be clarified.\n\nIt would strengthen the paper if more baseline methods were reimplemented and evaluated on the FLUX architecture and LAION dataset to provide a fair and comprehensive comparison with the proposed approach.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The underlying ideas are reasonable.\n\nThe evaluation demonstrates imperceptibility, fidelity, verifiability, and robustness.\n\nThe paper includes ablation studies supported by experiments and visualization.\n\nThe experimental details are clearly presented.", "weaknesses": "The watermarking results do not clearly demonstrate state-of-the-art performance; they are only competitive with previous methods.\n\nThe description of the proposed method is generally ok, but some paragraphs are poorly written and fragmented, which affects readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mcBu9DFT3I", "reviewer": "Reviewer_Tahr", "summary": "The paper proposes MOLM, a watermarking framework for text-to-image diffusion models that views watermarking as key-dependent parameter perturbations of a *frozen* generator. Concretely, the method installs LoRA adapters in selected residual/attention blocks and routes activation through one adapter per block according to chunks of a binary key. Thus, a key defines a deterministic execution path (the perturbation), while the backbone weights remain unchanged. A lightweight extractor network recovers the key from generated images. The training objective balances a perceptual loss for imperceptibility with a bitwise BCE for verifiability. Experiments on Stable Diffusion v1.5 (512×512) and FLUX (1024×1024) show high key recovery, small FID deltas (≤ ~1.5), and robustness to a range of common image distortions, learned compression, diffusion-based regeneration, averaging, and white-box PGD attacks. Default configuration routesL=14 decoder ResNet blocks with P=4 adapters per block, yielding M=28-bit keys; larger capacities are discussed via more blocks or choices", "questions": "1. If a key leaks, how quickly can you rotate without retraining? Is there support for *soft revocation* (e.g., attenuating adapters) versus installing fresh adapter banks?\n2. You evaluate averaging removal/forgery with the same messages; what about heterogeneous-key collusion across many users? Can mixed-key averaging partially cancel the watermark?\n3. Have you tried prompt/noise optimization targeting the extractor gradient via a differentiable proxy, or classifier-free guidance tuning to hide bits while maintaining content?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Casting watermarking as key-conditioned parameter routing over a frozen backbone is elegant, modular, and orthogonal to model architectures. The idea neatly ties together LoRA efficiency with watermarking needs.\n2. Keys correspond to routing masks; adapting capacity does not require retraining the backbone. The paper reports ~1 GPU-day one-time training and no additional inference cost beyond activating chosen LoRA paths.", "weaknesses": "1. The paper frames both detection and attribution but largely evaluates bit recovery / TPR against a fixed key/threshold. How MOLM behaves with large key databases (collisions, nearest-neighbor attribution errors, false-match rate under heavy post-processing) is not fully quantified.\n2. The white-box attacks target the extractor in image space. Stronger adversaries could attempt prompt-, noise-, or sampler-level optimization to suppress/flip bits while staying on-distribution (e.g., plug-and-play adversaries).\n3. While MOLM avoids per-key retraining, operational details (e.g., revoking/rotating keys, multi-tenant key assignment, rate-limited leakage scenarios, collusion of users averaging many outputs with heterogeneous keys) are not empirically explored. \n4. The table compares to Stable Signature, AQuaLoRA, WOUAF (bit recovery) and Tree-Ring/ROBIN/Gaussian Shading (detection). Some strong recent defenses/attacks (esp. post-2024 SoK/benchmarks) may warrant a tighter apples-to-apples setup (identical prompts/seeds, extractors retrained with the same aug sets), though the paper cites them. \n5. The robust baselines for watermarking arbitrary images are lacking, such as TrustMark [1], VINE [2], StegaStamp [3]. Incorporating those methods would make the comparison more thorough and robust.\n\n\n[1] TrustMark: Universal Watermarking for Arbitrary Resolution Images\n\n[2] Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances\n\n[3] StegaStamp: Invisible Hyperlinks in Physical Photographs", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TAb1X5bnnK", "reviewer": "Reviewer_D7NA", "summary": "MOLM repurposes multiple LoRA adapters as watermark “markers” and uses key-dependent routing (MoLE-style) to embed a bitstring while keeping the backbone frozen. A learned extractor recovers bits. Experiments on SD-1.5 and FLUX report near-zero inference overhead, small FID changes, and robustness to common distortions and selected white-box attacks.", "questions": "1.Novelty. The core idea of MOLM is to repurpose LoRA adapters as watermark carriers and use key-driven routing to activate specific adapters for embedding. This is essentially a composition of mature techniques: 1) LoRA is a well-established parameter-efficient fine-tuning method; MOLM merely reinterprets it as a watermark carrier. 2) The routing logic follows MoLE, replacing data-gated expert selection with key-gated selection.\n2.Baselines. Include recent watermarks or clearly justify exclusion.\n3.Presentation. Fix typos (e.g., “Deployment”; “~8 days”), align the Table-3 title with its contents ('inference' is not listed, although it is mentioned in the paper), and unify notation/spacing across equations and tables.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.Practicality. Lightweight LoRA adapters, no architecture changes, negligible runtime overhead.\n2.Coverage. Evaluation on SD-1.5/FLUX with ~28-bit default capacity and higher-capacity variants; ablations on placement/configuration.\n3.obustness. High bit accuracy under JPEG/crop/resize and diffusion regeneration; PGD-style white-box results show resilience.", "weaknesses": "1. The method is a straightforward combination of existing techniques (LoRA + MoE routing).\n2. Fails to compare with recent state-of-the-art watermarking methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)