Disentangled Style Domain for Implicit $z$-Watermark Towards Copyright Protection

Authors: junqiang  huang
zhaojun  guo
ge  luo
zhenxing  qian
sheng  li
xinpeng  zhang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 12:22:28

Abstract: Text-to-image models have shown surprising performance in high-quality image generation, while also raising intensified concerns about the unauthorized usage of personal dataset in training and personalized fine-tuning. Recent approaches, embedding watermarks, introducing perturbations, and inserting backdoors into datasets, rely on adding minor information vulnerable to adversarial purification, limiting their ability to detect unauthorized data usage. In this paper, we introduce a novel implicit Zero-Watermarking scheme that first utilizes the disentangled style domain to detect unauthorized dataset usage in text-to-image models. Specifically, our approach generates the watermark from the disentangled style domain, enabling self-generalization and mutual exclusivity within the style domain anchored by protected units. The domain achieves the maximum concealed offset of probability distribution through both the injection of identifier $z$ and dynamic contrastive learning, facilitating the structured delineation of dataset copyright boundaries for multiple sources of styles and contents. Additionally, we introduce the concept of watermark distribution to establish a verification mechanism for copyright ownership of hybrid or partial infringements, addressing deficiencies in the traditional mechanism of dataset copyright ownership for AI mimicry. Notably, our method achieved One-Sample-Verification for copyright ownership in AI mimic generations. The code is available at： [https://github.com/Hlufies/ZWatermarking](https://github.com/Hlufies/ZWatermarking)

Main paper: 
 introduction: 
 Recent advancements in text-to-image generation technologies [1, 2, 3] have revolutionized art creation by enabling users to replicate the unique styles of artists and art images through simple prompts. Simultaneously, text-to-image personalization technologies [4, 5, 6, 7] make it easy to fine-tune generative models with minimal online personal portfolios, which may not be authorized. However, a question arises: are individual styles and contents entitled to copyright protection? Recent studies [8, 9, 10, 11] indicate significant visual and stylistic similarities between AI-generations and unauthorized datasets. For example, the artistic style generated from the prompt, "an image of a vast grassland resembling the style of Van Gogh's 'Starry Night'", can be linked to Van Gogh's artistic domain, despite the absence of direct observation of 'original artwork' generations. Therefore, a new paradigm is needed to emphasize ownership of styles and content for dataset copyright protection.Currently, there are many efforts (e.g., Glaze [12] , DIAGNOSIS [13] and Luo et al. [14] ), aimed at protecting the copyright of personal datasets. Glaze prevents fine-tuning by introducing calculated perturbations into protected datasets, inducing mimic models to learn different image styles. However, Bochuan et al. [15] indicate that these perturbations are susceptible to adversarial purification. Meanwhile, Glaze makes authorized training impossible. Besides, DIAGNOSIS, which constructs backdoors based on diffusion model memorization, is an approach to copyright protection. However, integrating backdoors into the dataset might introduce new harmful security risks [16] . At the same time, Luo et al. employ digital watermarking methods to detect unauthorized data usage, yet their robustness is insufficient, as easily removable at the latent level as indicated in [17] .To address the above problems, we introduce an implicit Zero-Watermarking scheme that focuses on the distinct style and creative essence ingrained within datasets, rather than only the digital carriers (e.g., digital images). Inspired by recent studies in disentangled representation learning [18, 19, 20, 21, 22] and style IP customization [23, 24, 25] , we consider diffusion model embodies a disentanglement learning approach. In other words, image generation is conceptualized as entangled combinations of styles and contents, within the mutually exclusive contraction domains generalized from the anchor of the original samples. Therefore, different from the existing methods of embedding invisible information into protected datasets, we use the domains to delineate the copyright boundaries.In this paper, we aim to generate implicit watermarks from the disentangled style domains of protected units, enabling self-generalization and mutual exclusivity. Specifically, we initially employ the style domain encoder to disentangle each protected unit into its style representation, serving as the center anchor of the contraction domain. Then, we generalize the contraction domain by the dynamic contrastive learning between central samples and boundary samples of the specific protected unit. Finally, the domain achieves the maximum concealed offset of probability distribution through both the injection of identifier z and dynamic contrastive learning, facilitating the structured delineation of dataset copyright boundaries which are mapped to the implicit watermarks. During the verification phase, to address the complex copyright boundaries in image generation with multiple sources of styles and contents, we propose a verification mechanism utilizing the style domain and watermark distribution to tackle hybrid or partial infringements. We highlight our main contributions as follows:1. We propose a novel watermarking method for dataset copyright protection against unauthorized AI mimicry. To the best of our knowledge, this work is the first study that facilitates the structured delineation of dataset copyright boundaries in the disentangled style domain. Notably, experiments demonstrate that ours accomplish the One-Sample-Verification challenge for copyright ownership of hybrid or partial infringements. 2. We utilize strategies for the self-generalization and mutual exclusivity of z-watermarking, breaking away from the traditional methods of embedding invisible information into datasets. 3. To tackle hybrid or partial infringements in image generation with multiple sources of styles and contents, we introduce the concept of watermark distribution to establish a verification mechanism for dataset copyright ownership by the disentangled style domain. 4. Extensive experiments on benchmark datasets demonstrate the effectiveness, robustness, and versatility of our method against various challenges including adversarial fine-tuning methods (e.g., Dreambooth), watermark removal (e.g., Latent attack) and the usage detection of unauthorized data in unlearnable black-box cross-APIs and models (e.g., DALL•E•3).2 Related WorksQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: In this paper, the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
text-to-image generation and diffusion models: 
 Recently, the field of visual synthesis has experienced significant advancements, with various research [26, 27, 28, 29, 30, 1, 31, 32] achieving impressive outcomes. Notably, diffusion models [30, 1, 31, 32] have emerged as pioneers in image generation, surpassing earlier models based on generative adversarial networks [26, 27] and autoregressive methods [28, 29] . Among these, Stable Diffusion [31] stands out for its noteworthy contributions to latent diffusion models. Besides, recent studies, such as Lora [4] , Dreambooth [5] , Textual inversion [6] , and ControlNet [7] , have shifted towards personalized fine-tuning of pre-trained diffusion models. These advancements empower individuals to replicate specific styles and contents with just minimal shared unauthorized samples. The main pipeline of dataset copyright verification with our method. Notably, we use the watermark extractor (with specific K watermark mapping relationships) and identifier z to detect protected datasets usage, instead of the traditional embedding and extraction pairing process.There are several ways [12, 33, 34, 35, 13, 36, 14] to prevent unauthorized data usage. Adversarial example-based methods (i.e., Glaze [12] , AdvDM [33] , and Anti-Dreambooth [34] ) introduce perturbations to induce mimic models to learn different image styles during training and fine-tuning. Nevertheless, the added perturbations are dependent on and constrained by the surrogate model, resulting in weak generalization and transferability. Besides, backdoor-based dataset ownership verification [35, 13, 36] is conducted by defenders triggering whether the suspicious models exhibit specific backdoor behaviors. Yet, the integration of backdoors into datasets might introduce new harmful security risks indicated in [16] . At the same time, Luo et al. [14] propose a watermarking framework for detecting art theft mimicry based on digital watermarking techniques [37, 38, 39, 40] . However, these methods' robustness is insufficient, as easily removable as indicated in [17] .Disentangled representation learning [41] aims to model the factors driving data variations [42] .Early works [43] used labeled data to factorize representations in a supervised manner. Recently, unsupervised method [44] has been largely explored, especially for disentangling style and content from the image [45, 42, 46, 47, 48] . Inspired by recent studies in disentangled representation learning [18, 19, 20, 21, 22] and IP customization [23, 24, 25] , we consider that the act of generating them from scratch requires a deep understanding of the underlying factors and complex generative processes, unlike mere analysis of text or images. In other words, image generation is conceptualized as entangled combinations of styles and contents of the original samples. Taking this viewpoint, we redefine the concept of image beyond digital forms, viewing them as compositions of multiple representations that serve as class-free guidance for diffusion models in self-disentanglement. Additionally, since these disentangled representations are mutually exclusive in high-dimensional space, they naturally demarcate copyright boundaries through mutual exclusivity.3 MethodQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations of the work performed in this paper, and explore valuable research directions related to the topic for the future.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
threat model: 
 Attacker's Goal and Capability. Attackers could train or fine-tune on protected datasets (D) to replicate the styles and contents of personal portfolios, and then exploit this for financial gain or involvement in criminal. The attacker's capabilities are as follows:• Unauthorized access to proprietary datasets, such as personal portfolios and photo albums.• Utilize data attacks like second-stage fine-tuning, mixed-clean dilution, purification-latent attack, prompt attack, and data augmentations to remove potentially hidden information.• Just publicize the APIs and keep the mimicry details hidden, including fine-tuning approaches and training parameters.Defender's Goal and Capability. The defender aims to detect single or minimal instances of mimic images, i.e., publicly available online or offline, to track back copyright ownership. Before sharing data, defenders register the identifier z and implicit watermarks W z for protected units with a third party. The defender's capabilities are as follows:• General Ability: Defenders obtain stylized mimic images from known suspicious models or APIs to verify copyright ownership in black-box setting.• Limited Ability: Defenders occasionally and randomly acquire minimal (even a single) mimic images online or offline without any prior knowledge.The overview of z-watermarking is depicted in Figure 2 . Our pipeline consists of three phases (i.e., Disentangled Style and Contents, Mutual Exclusivity, and Self-Generalization). Firstly, disentangle all units of the protected dataset and place the contraction domain within the style domain, maximizing the offset by identifier z to achieve non-overlapping from a standard distribution. Secondly, selfgeneralization of each contraction occurs through dynamic contrastive learning between the central and boundary samples of the protected unit. Finally, it's implicitly marked as a watermark due to the mutual exclusivity of contraction domains, rather than embedding invisible information into datasets. Here, z serves as the key or special bias of the disentangled style domain S z of protected units, and Q denotes the dynamic historical negative queue.The style domain encoder E z (ResNet) and denoising decoder D z (UNet with 2m + 1 activation layers h i ) are formally defined by a pair of forward and backward Markov chains representing a T -steps transformation from a normal distribution z T ∼ N (0, 1) into the learned distribution z 0 ∼ p θ (z x ).We aim to achieve disentangling of images at the latent level, hence we regularize data x into latent representations z x following a Gaussian distribution N by VAE as follows.EQUATIONEq.1 denotes the probability distribution q ϕ (z x |x) for z x , which is the mean µ(x) and variance σ 2 (x) of z x (i.e., ϕ denotes the parameters of VAE). The style domain encoder is represented as follows:E z (z x |(x, ϕ), z) = s, where s = {v i } m i=1 (i.e., v 1:m is semantically or visually relates to x). Identifier z serves as the key or special bias of the style domain S z . Identifier z can be the spatial embedding vector (e.g., image, text, audio, model, etc.). In this paper, we set the text 'swz' to be converted into text feature embeddings by CLIP (i.e., ϕ z ) as z, embedding it into E. Then, we partition the vector s into m + 1 sections, which is half the number of layers in the decoder Dz. Each v i ∈ s is utilized to modulate the corresponding pair of layers (h i , h 2m-i ), thus fostering specialization among the latent sub-vectors. Moreover, we implement layer-wise guidance dropout by selectively zeroing out portions of s 1:m , thereby diminishing the decoder's dependency on sub-vector correlations. The details and tricks are in the supplemental material, and we derive a pre-trained style encoder trained in MS-COCO.Based on the pre-trained style domain encoder, we design the parameters θ d of the discriminator and the θ w of watermark extractor in z space. Specifically, let D = {x i } K i=1 that denotes the protected dataset, where K is the number of the protection units (i.e., each sample or class with shared attributes). Let D s = {x(n) i } K,Ni=1,n=1 denotes mimic samples that include K subsets of protected units generated by the surrogate model M trained on D, where N is the number of mimic samples for the k-th protected unit. The optimization objective for the parameters θ of E z is as follows:EQUATIONwhere c k ∈ C denotes the class of k-th protected unit, and w k denotes the mapping watermark relationship to k-th contraction domain. F s denotes the cosine similarity function, and L(•) is the loss function (e.g., H 1 is cross entropy loss, H 2 is mse loss). Specifically, identifier z denotes the representation s is shifted to the marginal distribution. Moreover, ψ in Eq.2 represents the domain regularization term, aimed at achieving dynamic self-generalization and mutual exclusivity of the contraction domain according to the following constraints as Eq.3 and Eq.4.1EQUATIONLet x k ∈ D and {x(n) k } = D k s ∈ D s , where D k s denotes the similar mimic set of the k-th protected unit x k . Let D k s = D k + s + D k - s , where x + i ∈ D k + sis the central sample of the contraction domain of x k , and x - i ∈ D k - s is the boundary sample of the contraction domain. c denotes the boundary value of the contraction domain and I(•) denotes the distance function. We aim for the contraction domain to ensure self-generalization in Eq.3, while evolving mutual exclusivity in Eq.4. Let x ¬k ∈ D ¬k s denote the complement of D k s , serving as the negative samples, whereEQUATIONwhere β is a positive hyper-parameter. To achieve the above constraint, letψ = λ 1 ψ 1 + λ 2 ψ 2 ,where λ 1 , λ 2 are two hyper-parameters. ψ 1 aim to achieve self-generalization and ψ 2 ensures mutual exclusivity and maximum offset described in §3.4 and §3.5.As previously mentioned, we aim to explore the style boundaries of the contraction around the anchor sample x k to establish the range we want to protect. ψ 1 aims to achieve self-generalization in Eq.5.EQUATIONwheres k ∼ D, s i ∼ D k s and s + i ∼ D k + s .τ is the temperature parameter. We designate a subset of the k-th protected unit as positive samples and the rest as negative (normal). Notably, s k is disentangled by E z , while s i and s + k are disentangled by E ′ z . Representing the parameters of E z as θ and those of E ′ z as θ ′ , we update θ ′ by momentum update:θ ′ ← mθ ′ + (1 -m)θ, where m is a momentum coefficient. Only θ are updated by back-propagation.Let s k ∼ D, s ∈ D s and s i ∈ D k s . The contraction domains of protected units are mapped from E z to the pre-defined implicit watermarks. Formally, the regularization term ψ 2 is defined in Eq.6.EQUATIONwhere Q denotes the dynamic historical negative queue. Q is initially populated with anomalous samples and anomaly identifiers, consistently serving as negative instances for contrastive learning.Here, we also employ momentum updates of E ′ z to encode negative examples.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: The code for reproducing main experiments is available at: https://github. com/Hlufies/ZWatermarking. Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
dataset copyright verification via z-watermarking: 
 One-Sample Verification. We aim to verify copyright ownership by single or minimal mimic images. Therefore, we propose the disentangled style domain facilitates the structured delineation of dataset copyright boundaries. Experiments show that our method offers robust copyright verification, with a single-sample success rate far exceeding the baseline, in Table 2 . The probability of the watermark guided by z in the contraction domain of the protected unit is denoted as Eq.7.EQUATIONwhere lim x→D P z (x|ϕ) = η indicates x (mimic sample) originating from protected D is an extremely unlikely probability event (i.e., η denotes infinitesimal). The occurrence of extremely low-probability events ensures the credible mathematical basis for the ownership of sample copyrights. Meanwhile, in generative scenarios, the difficulty in identifying the target carrier (relying only on manual similarity judgments) renders traditional one-to-one watermark verification mechanisms limited.Extensive Statistical Verification. To further validate our method's effectiveness from multiple perspectives, we propose the concept of watermark distribution. Assuming the multi-styles and multi-contents of T datasets are present in D m , T defenders utilize {E zt , z t } T t=1 to disentangle the image generations. Let E zt (x) = (c i , w i ), where c i ∈ {0, k t } and k t ∈ K t (i.e., K t is the number of protected units of t-th dataset). Watermark distribution is defined as Eq.8:EQUATION) Let t@k@100%wd = t@wd[arg max kt {{ x∼Dm c i }|F b (•) = 100%, c i = k t }]that denotes the best distribution (i.e., within D m , the number of samples of the k t type of t-th datasets is the largest) in the most accurate distribution (i.e., with bits accuracy reaching 100%). When assessing data copyright for single-party verification, two criteria must be fulfilled: Avg acc > α and t@k@100%wd > γ (where Avg acc represents Average Watermark Accuracy). In this paper, the threshold for α is set to 0.99, and the threshold for γ is set to 0.80. For multi-party hybrid or partial infringements, copyright ownership is determined by comparing the maximum value of t@k@100%wd tested across T different style domain encoders in D m , attributing it to the k-th unit of the t-th protected dataset. 
experimental setting: 
 Datasets and Models. In this paper, we pre-train the style domain encoder [49] and decoder [50] on MS COCO [51] . We conduct experiments on three open-source benchmark datasets (i.e., CelebA [52] , Pokenmon [53] , Dreambooth dataset [5] ), 17 Artists (e.g, Van Gogh and Monet) and 10 AI' artworks (e.g., GhostMix and CatLora). The surrogate model is Stable diffusion v1.5 [32] fine-tuned (i.e., Lora [4] and Dreambooth [5] ) on the benchmark datasets. Moreover, the attackers include Stable Diffusion v1.5&v2.0, and the APIs of DALL•E•3 [3] , Imagen2 [1] , PG-v2.5 [54] , PixArt-α [55] .We use CLIP to extract the z emb of text-z in our setting. Specifically, we randomly select a subset containing K protected units from each dataset for training (i.e., N mimic images per unit generated).Baseline and Metrics. We compare our pipeline against existing digital watermarking methods (i.e., DCT-DWT-SVD [39] , RivaGan [37] , SSL [40] , Trusmark [56] ), and RoSteALS [57] ). We evaluate each method's performance using average watermark accuracy (Avg acc) and watermark distribution metrics (t@wd and t@k@100%wd, as defined in §4 (a) The watermark distribution across generated samples from 10 random protected units. To benchmark the effectiveness of the watermark, we primarily report the watermark distribution across 1000 image generations from all units of each protected dataset in the black-box validation scenario of AI mimicry, utilizing Avg acc and t@k@100%wd (i.e., the proportion of samples where watermark bit accuracy hits 100%) for evaluation.Table 1 : Main results. We enumerate the sample count within each range of watermark distribution (128 bits) from 1000 mimic images of all protected units for both mimic and non-mimic models. Main experimental findings regarding watermark distribution validation on both mimic and nomimic models across five datasets are detailed in Figure 3a and Table 1 . Our method outperforms the watermark distribution under random states, which are not exposed on the protected dataset. Specifically, our average accuracy exceeds 99%, significantly higher than the approximately 50% of the non-infringement state model. Such a significant difference in watermark distribution is one of the key pieces of evidence for copyright authentication. Additionally, we compare our method with digital watermarking approaches in Table 2 , where our method achieves an Avg acc of 99.83%, surpassing others that reach only around 60%. Notably, in cases of 100% watermark accuracy (t@k@100%wd), the proportion of samples using digital watermarking methods is deficient, whereas our method reaches 97.7%. This indicates the failure of digital watermarking in attributing ownership in AI mimicry scenarios, contrasting with the effectiveness of our proposed method. To benchmark the robustness of our watermark, we document its performance against various attack methods. These include identifier z error, second-stage fine-tuning with a 1:10 ratio between original and generated images, mixed clean fine-tuning with a blending rate of 0.1, watermark removal in latent attack [17] , and prompt attack with different description in black-box scenarios. Additionally, we utilize 7 data augmentations as attacks, consisting of 90 o rotation, 50% JPEG compression, 60% center cropping and scaling, Gaussian blur with a 3×3 filter size, color jitter with a hue factor of 100, along with adjustments to brightness by a factor of 1.5 and contrast by a factor of 2.0. In Table 2 , baseline methods fall short in attributing copyright ownership due to their inability to extract the complete watermark, achieving a likelihood of less than 0.1. Conversely, our methods demonstrate heightened reliability. Table 3 reveals that even under adversarial conditions, we can extract numerous samples with 100% bit accuracy, surpassing the performance of baseline models (i.e., t@k@100%wd less than 0.1%). Notably, while attacks decrease t@k@100%wd, only a certain proportion of t@k@100%wd samples is required for verification in AI mimicry copyright attribution. This is attributed to the improbable occurrence of such events in a natural state, as outlined in Eq.7.To benchmark the generalization capability of our watermark, we document its performance in copyright verification within the landscape of AI mimicry, considering an array of fine-tuning models and black-box APIs in Table 4 . We set the surrogate model to Stable Diffusion v1.5. Our primary focus lies on examining its generalization across Stable Diffusion v1.5 (i.e., Lora and Dreambooth) & v2.0, as well as the APIs of PixArt-α, PG-v2.5, DALL•E•3, and Imagen2. Each model and API is instructed to generate 20 images for inspection using the prompt "An art piece resembling the style of 'Starry Night'", aiming to discern whether the attack model has been exposed to Van Gogh's portfolios. In Table 4 , our method achieves 100% Avg acc on most suspicious models, with an overall average of 98.60%. The t@k@100%wd also reaches nearly 100% on most models, with an average level of 94.29%. Additionally, it achieves a 100% accuracy in True Positive (TP) detection, as well as 100% Avg acc and t@k@100%wd. As depicted in Figure 4 , our proposed method provides strong evidence of its ability to detect imitation behavior of commercial APIs like PixArt-α using a few suspicious samples. Attacker models/APIs FID CLIP TP TN Avg acc (%) t@k@100%wd (%) SD-v1. We hereby discuss the effects of several key hyperparameters involved in z-watermarking. Please find more experiments regarding other parameters and detailed settings in the Appendix.Model Component Study. We evaluate the effectiveness of our components: Disentangled Style Domain D, self-generalization module ϕ 1 , and mutual exclusivity module ϕ 2 . As shown in Figure 5a and 5d, we compare Avg acc and t@k@100%wd across five datasets. It demonstrates that omitting any proposed components leads to a decline in the model's performance. Notably, the Disentangled Style Domain enables the t@k@100%wd to rise from around 60% to over 90%, while the addition of ϕ 1 and ϕ 2 further optimizes the model to achieve performance exceeding 99%.Data Scale Study. We next explore the relationship between the training data scale and validation data scale in Figure 5b and 5e, which is crucial for real-world copyright protection scenarios. Experimental evidence suggests that a minimal set can propel our model to an above 99% Avg acc. Besides, high t@k@100%wd is observed on a limited scale of validation dataset, reflecting the effectiveness of practical validation settings, where access to the APIs of suspicious black-box mimic models.Bit Length Study. Watermark bit length serves as the bedrock of copyright verification reliability theory in AI mimicry scenarios. Figure 5c and 5f present experiments validating lengths from 32 to 512 bits across five datasets. Notably, experiments with a bit length of 512 demonstrated higher average accuracy and t@k@100%wd, highlighting the scalability and reliability of z-watermarking.  
discussion and limitation: 
 Ethical Statement: To enrich our experimental dataset, we gathered a lot of contemporary authorized artworks, historical artworks, and AI-generated pieces. We at this moment affirm that this collected data is specifically for the experiments related to our method and is not for any other use.Limitation: Our proposed method focuses on the disentangled style domains of protected units. Consequently, modifications to the deep features of style domains using existing techniques (e.g., style transfer or bias injection), are expected to influence our performance. Although our robustness study confirms our reliability against Latent Attacks, the potential impact of specific targeted attacks (e.g., Glaze) on performance degradation could not be overlooked. To address this challenge, adding specific adversarial samples for adversarial optimization could guide our future research. Additionally, the fine-tuning performance of the surrogate model may slightly affect the experimental results, so it is necessary to set the optimization parameters appropriately.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We have provided all training and testing details in the appendix (such as data splits, hyperparameters, selection methods, optimizer types, etc.). . 
conclusion: 
 This paper presents the first study on the disentangled style domain for implicit watermarking to detect unauthorized data usage of AI mimicry, from the perspective of entity protection in styles and contents. Extensive experiments demonstrate the superiority of z-watermarking compared to the baseline. Notably, our method achieves one-sample verification for copyright ownership of hybrid or partial AI infringements. We aspire for our work to advance the ethical evolution of future artificial intelligence, ensuring due respect for creators' copyrights.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: In our experiments conducted across five datasets, each repeated five times under consistent settings, we observed mean errors within 1%, leading us to employ 1-sigma error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We provide sufficient information on the computer resources required to reproduce each experiment in the appendix of the paper.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss both the potential positive societal impacts and negative societal impacts of our work in the paper.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper may poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We cite and provide detailed descriptions of benchmark datasets and some related works in the paper.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our paper dose not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our work does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: Our paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: • The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material.Justification: Our research conduct in the paper adheres to the NeurIPS Code of Ethics in every respect.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: 1x A and 3x BR. This paper proposes an implicit watermarking scheme using disentangled style domains to protect dataset copyrights in text-to-image models. The reviewers agree on the (1) important topic, (2) novel method, and (3) comprehensive experiments. Most of the concerns, such as the efficiency analysis, have been addressed by the rebuttal. Therefore, the AC leans to accept this submission.

Reviews: review: paper_summary: paper_summary: While text-to-image models excel in generating high-quality images, they also raise issues of unauthorized dataset copyright protection. This paper proposes a novel implicit watermarking scheme that detects and protects dataset copyrights by disentangling the style domain  to generate watermarks. The proposed method achieves One-Sample Verification, significantly improving existing copyright protection mechanisms.
strength_weakness: strength_weakness: 1. This paper involves many complex modules, and in Section 3 (Method), the authors use many symbols and subscripts. However, the annotations for these symbols and subscripts are not very clear. For instance, in line 144, $\mathcal{E}_z(z_x|(x, ϕ), z) = s$. It is not clear what the input to the style domain encoder is. If ‘|’ denotes a probabilistic condition, then the style domain encoder only has one input, but Figure 2 appears to show two inputs, which is confusing. For the numerous symbols and labels, it is recommended that the authors provide a unified introduction in each section.

2. The paper introduces the operational flow of the model in three parts. However, the authors seem to focus more on explaining each module rather than the connections between modules and the overall operational flow. As a result, after reading these parts, it remains difficult to grasp the overall process of the proposed method. The paper appears technically sound, but there is still significant room for improvement in introducing the technical flow.
questions: questions: 1. In line 150, “Moreover, we implement layer-wise guidance dropout by selectively zeroing out portions of  s_{1:m} , thereby diminishing the decoder’s dependency on sub-vector correlations.” Could you clarify the criteria used for this selection? Additionally, how does this approach effectively reduce the decoder’s dependency on sub-vector correlations? Is there existing literature that supports this conclusion, or can you provide a more detailed explanation?

2. In the experimental section, line 216, the authors tested “17 artists (e.g., Van Gogh and Monet) and 10 AI artworks.” Given that copyright protection is an incremental task, as new artists or AI artworks are created, the number of protected entities will increase. This presents two potential challenges:
	1. How does the proposed method handle the increasing number of protected entities? Does it require retraining with each addition, or is there a more cost-effective solution?
	2. As the number of protected entities grows, will the styles of these entities influence each other, potentially reducing the model’s detection performance?
limitations: limitations: The authors have addressed the limitations.
paper_summary: paper_summary: This paper introduces an innovative implicit $z$-watermarking scheme using disentangled style domains to protect dataset copyrights in text-to-image models. It achieves structured delineation of copyright boundaries, self-generalization, mutual exclusivity, and effective verification for hybrid or partial infringements. The method demonstrates high robustness and reliability against various challenges, marking a significant advancement in protecting copyrighted content in AI-generated visual data.
strength_weakness: strength_weakness: 1. In table I, it is suggested to compare with some recent and SOTA watermarking methods, such as Trustmark and RoSteALS.

[1] TrustMark: Universal Watermarking for Arbitrary Resolution Images.

[2] Rosteals: Robust steganography using autoencoder latent space. In CVPRW 2023.

2. Can $z$-watermarking resist some watermark removal or attack methods, such as DDIM inversion or VAE. The authors could provide some results to improve the completeness of the experiment.
questions: questions: Please refer to the weakness.
limitations: limitations: The author has clearly presented its limitations.
paper_summary: paper_summary: The paper introduces an implicit watermarking scheme that leverages disentangled style domains to detect unauthorized dataset usage in text-to-image models. The proposed method aims to address limitations in traditional watermarking techniques by using implicit z-watermarks for dataset copyright verification, achieving better robustness against various attacks.
strength_weakness: strength_weakness: 1. The writing of this paper has some problems. The description of its first section is totally in a chaos status. 
2. In some specific situations, I agree that the styles are necessary to be protected. However, in most situations, copyright law typically protects original works of authorship. An individual style, which may consist of a particular technique or aesthetic, is not considered a tangible, original creation. Styles are more akin to ideas or methods, which are generally not protected under copyright law. If the styles are not protected by the laws, why do we need such a method mentioned in this submission? 
3. Defining and enforcing copyrights for individual styles would be highly subjective and impractical. Styles evolve and are influenced by many sources, making it difficult to establish clear boundaries for what constitutes a protected style. 
4. Granting copyright protection to individual styles could stifle creativity and innovation. Artists and creators often build upon existing styles and techniques to develop new works. Restricting the use of styles could hinder the creative process and limit artistic freedom. For example, if the Cubist style were copyrighted, it could prevent new artists from experimenting with and developing this style further, thereby limiting artistic progress.
questions: questions: I have shown my concerns in the weakness part, Please address my concerns there.
limitations: limitations: I am uncertain about how practical it is to implement this method on a large scale. The experiments provided in this paper only cover limited data.
paper_summary: paper_summary: The current watermarks applied to AI-generated images rely on adding additional information, limiting their ability to detect unauthorized use of data. This paper introduces a new implicit watermarking scheme, which first utilizes the disentangled style domain to detect unauthorized dataset usage in text-to-image models., so as to achieve self-generalization and mutual exclusivity within the style domain anchored by protected units. In addition, this paper introduces the concept of watermarking distribution and establishes a verification mechanism for copyright ownership of hybrid or partial infringements. It is worth noting that this paper implements One-Sample-Verification for dataset copyright verification in AI mimic generation. This paper also designed experiments to verify the robustness, generalization and ablation of multiple data sets.
strength_weakness: strength_weakness: - This paper is really hard to follow. The method's modules are not explained very well, and the input, output and training details of each module are not clear.
- Too much mathematical description makes it difficult for readers to quickly understand the process and principle of the method, and the meaning of each mathematical symbol is not clearly described.
- The model structure diagram is less relevant to subsequent descriptions. The paper also lacks simple examples of the data used in the experiment.
-Only the watermark distribution concept is proposed, and the copyright ownership verification experiments of hybrid or partial infringements are lacking.
questions: questions: - The significant problem is the writing, too obscure.
- There is a problem with the icon marking in Figure 2, the marked color of central sample is inconsistent with that in the figure.
- There is an unclear sign in line 181. Should $s_k^+$ be $s_i^+$?
- Can you explain the meaning of each symbol of formula 7 in detail, and supplement the rationality and significance of the indicators proposed?
- In 5.2 Main result, can you give the value of the number of protected units (i.e., K) and how the 1000 images used were selected?
- It is observed that the avg acc is higher with the longer the watermark length in the ablation experiment. Can you give the details of the mapping from the contraction domain to the watermark in the extractor?
- The experiment is only compared with the digital watermarking method, can you add a comparison with other methods (such as backdoor-based)?
limitations: limitations: The authors addressed the limitations of this paper, which focuses only on the disentangled style domains of protected units, potentially making it difficult to resist attacks that modify deep features of style domains. Future research could incorporate specific adversarial samples.

Comments: title: Thanks and A Gentle Reminder of the Final Feedback
Comment
Thanks
Thank you and further explanations to reviewer's feedback
My feedback
comment: I do not have further questions. As we are also busy with our own submitted papers, the time is not enough for me to make my final decisions at this moment. Thanks. I will make my final decisions at the next stage.
Dear Reviewer **d18z**,

Please allow us to thank you again for your valuable time and constructive comments. Your comments have been instrumental in helping us clarify the significance of our work and enhance its quality. 

_As the reviewer-author discussion phase is nearing its end_, we would like to know whether our explanations and experiments have properly addressed your concerns. We are more than happy to answer any additional questions. Your feedback will be greatly appreciated.

Thank you again for your thorough review and for helping us improve our work!
Dear Reviewer **z2Ei**,

Thank you once again for your valuable time and constructive comments. We would like to kindly inform you that we have already addressed your concerns in our rebuttal.

As the reviewer-author discussion phase is nearing its end, we would like to know whether our explanations and experiments have properly addressed your concerns. We are more than happy to answer any additional questions. Your feedback will be greatly appreciated.
Dear Reviewer **NU2V**, thank you once again for your valuable feedback on our paper and for helping us improve our work. Your decision to raise our score is a recognition of our efforts. Regarding your concerns about the efficiency of the proposed framework, we have addressed this in our response to Reviewer d18z by analyzing the pipeline of our method across _the registration, computation, and inference stages_, supported by detailed experimental data. We will further clarify the efficiency of this framework in the revised paper. Thank you again for your thorough review and for helping us improve our work!
Thank you for your response. These details are important so the reader can understand your method clearly, I will raise my score. However, after reading other reviewer's comments, I also have the same concern: the efficiency of this proposed framework should also be analyzed.
Thank you for your detailed response. The additional experiments and explanations have addressed most of my concerns. I believe this paper presents an interesting and practically valuable work, with performance surpassing similar digital watermarking methods, making it applicable for copyright protection in generative AI. Considering the superior performance and the novelty of this approach, I decide to raise my score.
Thanks for the quick reply. I will make the final decisions based on the discussions with other reviewers.
Dear Reviewer **d18z**, please allow us to thank you again for reviewing our paper and for your valuable feedback. We understand your concerns and are providing additional explanations to address them.

---

First, our method is both simple and efficient for users: **they only need to register the _the identifier $z$, the watermark, and the protected data with a third-party regulatory body, as mentioned in  Section 3.1 (Defender Capability) of our paper_** . [1] notes that with the further commercialization of AIGC, the standardized data flow should involve _data owners, model providers, and public regulatory agencies (trusted third parties)_. The third party will jointly hold the unique identifier $z$ and watermark with the user, ensuring a one-to-one correspondence and non-redundancy between $z$, the watermark, and the protected data. Data copyright tracing and ownership should be initiated by the user through litigation, with processes involving judicial authorization and third-party verification. Together, the identifier $z$, watermark, and protected data constitute a personal copyright entity, enabling effective and secure copyright tracing in judicial proceedings. 

Second, **the _essential security, rigor, and accuracy_ of personal copyright verification in judicial procedures are effectively supported and demonstrated by the rigor and complexity of the algorithmic framework presented in our paper**. We have designed a comprehensive and well-rounded protection and verification mechanism with a focus on personal copyright security, and we provide extensive experimental results to validate the reliability of our method.

Third, **our framework is divided into three stages: registration, computation, and inference.** _In the registration stage_, data owners register their identifier $z$ and corresponding watermark with a third-party regulatory body. _In the computation stage_, the third party uses our algorithm to perform computations and store the results after receiving the registration list. _In the inference stage_, the framework performs copyright verification on suspicious samples and models. **We analyze the efficiency of the framework as follows**: **_On one hand_**, in terms of resource consumption, during the computation stage, using a single 3090 GPU, the average computation time per user is **1** minute, with proxy sample computations averaging **3-5** minutes and memory usage approximately **3**MB. In the inference stage, the average inference time for **1000** users ranges from **30 to 100 milliseconds** (i.e., **0.065** milliseconds per user). **_On the other hand_**, regarding copyright tracing accuracy, the ASR metric is close to **100%**, with an error rate controlled below **0.1%**, and the average watermark accuracy exceeds **99%**. Of 1000 suspicious AI mimic samples, about **97%** can be successfully verified and traced through judicial proceedings (as indicated by the t@k@100wd% metric mentioned in this paper).

Users such as artists only need to register the identifier $z$ and the corresponding watermark with the third party. The design and complexity of the algorithmic framework ensure the security, rigor, and accuracy of copyright protection. Overall, our framework demonstrates its practicality in judicial security validation, resource consumption, and efficiency.

[1] Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyond.
Thanks for the rebuttal. I have some concerns when reading other people's comments. Reviewer z2Ei mentions this proposed framework has many complex modules. By checking this paper again, I also agree with this point. If this work is for those artists, such a complex may hinder its usage. If people are reluctant to use it, any effective frameworks may become useless. How can you handle this issue? Besides, based on the provided descriptions, the efficiency of this proposed framework should also be analyzed, if the target users are those artist mentioned by the authors.