ZeroMark: Towards Dataset Ownership Verification without Disclosing Watermark

Authors: junfeng  guo
yiming  li
ruibo  chen
yihan  wu
chenxi  liu
heng  huang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 22:58:46

Abstract: High-quality public datasets significantly prompt the prosperity of deep neural networks (DNNs). Currently, dataset ownership verification (DOV), which consists of dataset watermarking and ownership verification, is the only feasible solution to protect their copyright by preventing unauthorized use. In this paper, we revisit existing DOV methods and find that they all mainly focused on the first stage by designing different types of dataset watermarks and directly exploiting watermarked samples as the verification samples for ownership verification. As such, their success relies on an underlying assumption that verification is a \emph{one-time} and \emph{privacy-preserving} process, which does not necessarily hold in practice. To alleviate this problem, we propose \emph{ZeroMark} to conduct ownership verification without disclosing dataset-specified watermarks. Our method is inspired by our empirical and theoretical findings of the intrinsic property of DNNs trained on the watermarked dataset. Specifically, ZeroMark first generates the closest boundary version of given benign samples and calculates their boundary gradients under the label-only black-box setting. After that, it examines whether the given suspicious method has been trained on the protected dataset by performing a hypothesis test, based on the cosine similarity measured on the boundary gradients and the watermark pattern. Extensive experiments on benchmark datasets verify the effectiveness of our ZeroMark and its resistance to potential adaptive attacks. The codes for reproducing our main experiments are publicly available at \href{https://github.com/JunfengGo/ZeroMark.git}{GitHub}.

Main paper: 
 introduction: 
 Deep neural networks (DNNs) have demonstrated their strong ability in widespread applications, such as face recognition [1, 2, 3] . Currently, there are many (high-quality) public datasets, such as CIFAR [4] and ImageNet [5] , that can be easily downloaded and used. Arguably, their availability is one of the key factors in the prosperity of DNNs, as developers can evaluate and improve their models upon them. In particular, these datasets are usually only freely available for non-commercial use since their collection and annotation are time-consuming and even expensive.To the best of our knowledge, dataset ownership verification (DOV) [6, 7, 8, 9, 10, 11] is currently the only feasible solution for protecting the copyright of public datasets. Specifically, DOV consists of two main stages, including dataset watermarking and ownership verification. In the first stage, dataset owners will introduce some imperceptible watermarked samples to generate the released watermarked version of the original dataset, so that all models trained on it will have specific distinctive prediction behaviors on particular samples (i.e., verification samples) while having normal behaviors on standard testing samples. In the second stage, given the API of a suspicious third-party deployed model, the dataset owners will detect whether it is trained on the protected dataset by examining its prediction behaviors on verification samples.In this paper, we revisit existing DOV methods. We find that they all mainly focused on the first stage by designing different types of dataset watermarks, no matter whether their watermark is backdoorbased [6, 7, 8] or not [9, 11] . All of them directly exploited watermarked samples as the verification samples in their second stage. Accordingly, their success relies on an underlying assumption that verification is a one-time and privacy-preserving process. Otherwise, as the watermark pattern is leaked during the (first) verification process, the malicious dataset users can easily remove the watermark from the model trained on the stolen dataset or from the verification samples. However, this assumption does not necessarily hold true in practice since adversaries can always update their unauthorized models. As such, an intriguing and critical question arises: Could we verify dataset ownership without disclosing dataset-specified watermarks to ensure a secure verification process?The answer to the aforementioned problem is positive. In this paper, we first delve into the intrinsic property of DNNs trained on the watermarked dataset. We empirically and theoretically demonstrate that the gradient of watermarked models calculated upon the closest samples located at the decision boundary (dubbed 'boundary gradient') has a similar direction to their corresponding watermark patterns, measured by their cosine similarity. Specifically, the distribution of cosine similarity of the watermarked model on the dataset-specified target class has significantly larger values than that of the remaining (benign) classes. In particular, samples located at the decision boundary (dubbed 'boundary samples') of watermarked DNNs contain limited information about the dataset-specified watermarks, measured by several metrics (e.g., mutual information). Motivated by these intriguing findings, we propose to conduct dataset ownership verification with closest boundary samples instead of samples containing dataset-specified watermarks (as shown in Figure 1 ). We call this method as ZeroMark. Specifically, our ZeroMark has three main steps. In the first step, ZeroMark generates the closest boundary version of given benign samples. The second step calculates the boundary gradients of generated closest boundary samples based on the Monte Carlo method. Both steps are conducted under the label-only black-box verification setting, where dataset owners can only query the suspicious model with verification samples via API and get its predicted labels. In the third step, ZeroMark examines whether the given suspicious method has been trained on the protected dataset via a hypothesis test, based on the distribution of cosine similarity measured between the boundary gradients and the corresponding watermark pattern.In conclusion, our main contributions are four-fold: (1) We revisit existing DOV methods and reveal their underlying assumption regarding the verification phase. It does not necessarily hold, hindering the protection of dataset copyright. (2) We empirically and theoretically discover an intrinsic property of watermarked DNNs regarding boundary samples (i.e., those located at the decision boundary). (3) We propose a simple yet effective method (i.e., ZeroMark) to verify dataset ownership without disclosing dataset-specified watermarks. (4) We conduct experiments on benchmark datasets, verifying the effectiveness of ZeroMark and its resistance to potential adaptive methods. 
related work: 
 2.1 Data Protection Classical Data Protection. Data protection is a classical and significant research area, which aims to prevent unauthorized data usage or protect private data. Existing classical data protection con-sists of three main categories, including (1) encryption, (2) digital watermarking, and (3) privacy protection. Specifically, encryption [12, 13, 14] encrypts the sensitive data so that only authorized users with a secret key can decrypt and use it. Digital watermarking [15, 16, 17] embeds an ownerspecified pattern to the protected data as the watermark to claim ownership. Privacy protection focuses on preventing the leakage of sensitive information of the data in both empirical [18, 19, 20] and certified manners [21, 22, 23] . Unfortunately, existing classical approaches can not directly protect the copyright of open-source datasets since they either hinder the dataset accessibility (e.g., encryption) or require the information of the training process of models trained on them (e.g., digital watermarking and privacy protection) that will not be disclosed by authorized dataset users.Dataset Ownership Verification. Dataset ownership verification (DOV) aims to verify whether a suspicious model is trained on the protected dataset. To the best of our knowledge, this is currently the only feasible method to protect the copyright of open-source datasets. Specifically, DOV intends to introduce specific prediction behaviors (towards verification samples) in models trained on the protected dataset while preserving their performance on benign testing samples. Dataset owners can verify ownership by examining whether the suspicious model has dataset-specified distinctive behaviors. Previous DOV methods exploit either backdoor attacks [6, 7, 8, 10] or others [9, 11] to watermark the original (unprotected) benign dataset. For example, recently, backdoor-based DOV [6, 8, 7] adopted poisoned-/clean-label backdoor attacks to watermark the protected dataset. Most recently, Guo et al. [9] adopted samples from the hardly-generalized domain as watermark samples without introducing any new security vulnerability. However, all existing dataset ownership verification (DOV) approaches [6, 8, 7, 9, 10, 11] mainly focus on designing watermarks with different properties (e.g., harmless and stealthy) and directly exploit the watermarked samples for verification. The security study of their verification stage remains blank and is worth further exploration.Currently, there are also a few works to safeguard the inference process of models. In the context of machine learning, secure inference is a two-party cryptographic protocol applied in the inference phase of machine learning models [24, 25, 26] . The server learns nothing about clients' input, while a client learns nothing about the server's machine learning model but can only get the results. Technically, it is implemented by having the server and client involved in a specific protocol and running the encrypted model over the encrypted input through cryptographic techniques such as homomorphic encryption [27] and secret sharing [28] . However, secure inference requires both the client and server to encrypt input data and adapt the machine learning model's operations accordingly through cryptographic mechanisms [27, 28] . As such, it is infeasible to protect the verification process of DOV methods since suspicious third-party models may not support these protocols.3 The Property of Models Trained on the Watermarked Dataset [29, 6] , where m ∈ {0, 1} C×W ×H is the trigger mask, δ ∈ [0, 1] C×W ×H is the trigger pattern, ⊙ denotes the element-wise product, and y t is the target label. In particular, γ ≜ |Dm| |Dw| is the watermarking rate. In the second phase (i.e., ownership verification), for a suspicious model C : X → Y that may be trained on D w , the dataset owners will investigate whether it conducts unauthorized training by querying it with verification samples under the black-box setting. In general, the verification process of existing DOV is to directly uses watermarked sample G x (x) as verification samples to examine whether C(G x (x)) = G y (y). In contrast, our goal is to perform the verification process without disclosing the watermark samples G x (x) during the inference phase of the suspicious classifier C. Boundary Samples. Let the logit margin of model f : X → [0, 1] K on the label y is denoted by:G y : Y → Y, i.e., D m = {(G x (x), G y (y))|(x, y) ∈ D s }. For example, G x = (1 -m) ⊙ ∆ + m ⊙ x and G y = y t in BadNets-based DOVEQUATIONIt can be observed that x can be classified as y by f (•; w) if and only if ϕ y (x; w) ≥ 0. As such, the set for boundary samples of class y can be denoted by B(y; w) = {x : ϕ y (x; w) = 0}. 
approach the closest boundary sample: 
 To obtain the boundary samples, we can easily use a gradient-free method (i.e., geometric search) to move each given sample x forward the decision boundary of f (•; w) under the label y, as follows:EQUATIONwhere α ∈ [0, 1] is a line search parameter and x y is a sample classified by the model f (•; w) as y.However, the obtained boundary sample of x would be varied according to different x y . As such, we use the closest boundary sample of x to study the characteristics of watermarked models.Following the previous work [30] , we define the closest boundary sample of x (i.e., x * ) as:EQUATIONwhere || • || 1≤p≤∞ is the ℓ p norm.Specifically, we can exploit the fast adaptive boundary attack (FAB) [31] to calculate the closest boundary sample. In particular, we adapt FAB to implement an iterative algorithm with gradient ascend using ∇ x ϕ y (x; w), whose update in (t + 1)-th iteration is as follows:EQUATIONwhere β t is a positive step size, x 0 is an initial point such that ϕ y (x 0 ; w) ≤ 0 and α t ∈ [0, 1] is chosen to ensure x t+1 lies in the decision boundary as Eq. (2). In practice, x 0 is randomly selected in the validation set whose label is different from y.In general, using the closest boundary samples generated via Eq. ( 4) is mostly because they are closely related to the intrinsic property of watermarked DNNs, as shown in the next subsection.In this section, we will show that the gradient of closest boundary samples ∇ x ϕ y (x * ; w) (dubbed 'boundary gradients') of watermarked DNNs is closely related to the watermark patterns. Before we present our technical details, we first define cos∠(x, ∇ x f (x t )) as follows:EQUATION)Following previous works [32, 9] , we use a model f (•; w) watermarked through the standard Bad-Nets backdoor attack (i.e., G x (x) = (1 -m) ⊙ x + m ⊙ ∆) [29] as a basic example to shed light on the intriguing characteristic of watermarked DNNs. β t = ||x t -x 0 || 2 • t q-1, there exists a constant c ≥ 0 such thatEQUATIONwhere q ∈ ( 1 2 , 1), y t is the target label (i.e., y t = C(G x (x))), δ is the watermark pattern (i.e., δ ≜ G x (x 0 ) -x 0 ), and t * is the number of convergence iterations of x * 's update. In general, since the right side of inequality (6) decreases with the increase of t * , Theorem 1 indicates that the cosine similarity between the watermark pattern δ and the gradient calculated on x t located in the watermark region increases along with its update process. Its proof is in the appendix.In the following parts, we empirically verify that the distribution of the cosine similarity between watermark patterns and boundary gradients of the closest boundary samples is different (i.e., has larger values) from that on their benign versions to further justify our Theorem 1.Settings. We hereby exploit BadNets-based dataset watermark [29, 6] with ResNet-18 [33] on the CIFAR-10 [4] dataset as an example for the discussion. Specifically, we watermark a sufficient amount of samples to achieve a high watermark success rate (i.e., ≥ 99%). The watermark pattern is a 4 × 4 square filled with random pixels, as shown in Figure 2 (a). We randomly select 400 benign samples and use Eq. ( 4) to generate their closest boundary version for the target label y t (i.e., class '0'). To reduce the randomness caused by the selection of watermark patterns, we introduce cos as the reference for normalization, as follows:EQUATIONand we calculate the normalized cosine similarity score as:EQUATIONwhere δ i is i-th random watermark pattern that is different from the original one (i.e., δ). We generate the gradient ∇ x ϕ y (x * , w) of the watermarked model on the target label (dubbed 'Grad (Target)') and benign labels (dubbed 'Grad (Benign)') others than the target one, as shown in Figure 2 (b) and Figure 2 (c), respectively. We then calculate their normalized cosine similarity scores with 400 samples. Since the values within the boundary gradient are sparse and not evenly distributed, we follow previous work [34] to select the largest 10 values within the m ⊙ δ and m ⊙ ∇ x ϕ yt (x * , w)) to calculate their corresponding cosine similarity score. More settings details are in the appendix.Results. As shown in Figure 2 (d), the normalized cosine similarity scores of the target label have significantly larger values compared with those of benign labels. However, their similarity scores still have some overlap (nearly 74% of the target label). It suggests that not all gradients calculated on the closest boundary samples can reflect the watermark pattern δ. It is mostly caused by the deviations introduced by the gradient estimation process under the black-box setting (as in Eq. ( 11)).Nevertheless, we can still distinguish between watermarked and benign models based on their similarity distributions by comparing their maximum instead of random values. Specifically, suppose we define a threshold τ > 0 as the maximum cosine similarity value for benign labels, and there exists P[ cos(•) > τ ] ≈ 0.26 for the watermark model. If we randomly sample m samples to calculate the (closest) boundary gradients and their corresponding normalized cosine similarity, we have:EQUATIONThere will be at least one sample having cos(•) > τ with a probability of1 -P[ cos(•) < τ ] m .As such, if we sample sufficient samples (e.g., 100), we will have a large chance (≥ 99%) to find at least one boundary gradient larger than τ to successfully identify the watermark models. These phenomena inspire the design of our ZeroMark method, as proposed in the next section.1 Estimate Gradient Following Eq. ( 9) 2 Update Following Eq. ( 4)Step I: Reach the Closest Boundary Point Suspicious DNNsStep II: Obtain Boundary Gradients Random WatermarkSuspicious DNNs Boundary Gradient Boundary Gradient Normalize Normalized SimilarityStep III: Boundary Gradient AnalysisFigure 3 : The main pipeline of our ZeroMark, which consists of three main steps. In the first steps, ZeroMark applies Eq. ( 4) and Eq. ( 10) to generate the closest boundary samples. In the second step, ZeroMark estimates the boundary gradients for generated boundary samples. In the third step, ZeroMark first leverages the ground truth watermark pattern and random watermark patterns to calculate the normalized cosine similarity following Eq. ( 7) and Eq. ( 8). After that, ZeroMark uses the distribution of similarity scores to conduct t-test for dataset ownership verification. In particular, in the third stage, patterns (i.e., random and ground truth watermarks) displayed in the gray background are only available to the defender and inaccessible to the suspicious model. 
methodology: 
 In this section, we describe the threat model and the technical details of our ZeroMark method.Threat Model. Consistent with previous DOV methods [7, 6, 8] , we assume that dataset owners will watermark the original dataset to generate its watermarked version. The dataset owner has full knowledge of watermark patterns and the validation samples used for ownership verification. In the verification stage, given a suspicious model, they will examine whether it was trained on the protected dataset under the label-only black-box setting, where they can only query the model with verification samples via model API and get its predicted labels, without accessing its intermediate results (e.g., gradients) and model parameters.The Main Pipeline of ZeroMark. In general, our ZeroMark consists of three main steps, as shown in Figure 3 . ZeroMark first generates the (closest) boundary samples of the suspicious model. Then, it calculates the boundary gradients of the generated boundary samples. ZeroMark conducts dataset ownership verification via boundary gradient analysis in the third step.Step 1. Generate Closest Boundary Samples. ZeroMark follows Eq. ( 4) to optimize the closest boundary samples x * iteratively with gradient decent. In particular, we exploit Monte Carlo method to estimate ∇ x ϕ y (x t ; w) to address the challenge in our considered label-only black-box scenarios, where the gradients for given inputs are inaccessible. The overall process is as follows:EQUATIONwhere β t is the step size, x 0 is an initial point such that ϕ y (x 0 ; w) ≤ 0, α t ∈ [0, 1] is chosen to ensure x t+1 lies in the decision boundary as Eq. ( 2), {µ i } N i=1 ∼ N (0, 1) are N random noises i.i.d sampled from the standard Gaussian distribution, and κ is a fixed positive parameter (i.e., 0.01).Step 2. Calculate Boundary Gradients. Once the closest boundary sample x * is generated, we can also exploit Monte Carlo method to estimate its gradient (dubbed 'boundary gradient'), as follows:EQUATION)Step 3. Boundary Gradient Analysis. After obtaining the boundary gradients, ZeroMark first calculates the cosine similarity based on the available watermark pattern δ and obtains boundary gradients. To further mitigate the variance caused by the watermark patterns, we create several random watermark patterns and follow Eq. ( 8) to normalize the calculated cosine similarity. After that, motivated by the characteristic described in Section 3.3, where the cosine similarity of the boundary gradients on the target label y t has a significantly larger value compared with that of benign labels, we design a hypothesis-test-guided method to conduct ownership verification based on the range of cosine similarity for verification, as follows. In general, we randomly select m (i.e., 500) validation samples evenly distributed across different classes to generate boundary gradients for the watermarked and benign labels. Then, we conduct the pairwise t-test [35] and calculate its p-value. The null hypothesis H 0 is rejected if the p-value is smaller than the significance level α. Besides, we also calculate the confidence score ∆P = P b -P d to represent the verification confidence. The larger the ∆P , the more confident the verification. 
experiments: 
 In this section, we conduct experiments on CIFAR-10 [4] and Tiny-ImageNet [36] datasets with ResNet18 and ResNet-34 [33] , respectively. More results with other settings are in our appendix.Settings. We hereby compare our ZeroMark method to two straightforward baseline methods, including (1) verification with minimal watermark (dubbed 'Minimal') and (2) verification with distorted watermark (dubbed 'Distortion'). These methods intend to protect the information of datasetspecified watermarks by perturbing the original watermarks. We also provide the results of verification with benign samples (dubbed 'Benign') and verification with original dataset-specified watermarked samples (dubbed 'Vanilla') for reference. We evaluate each verification method on four dataset watermark techniques, including two patch-based watermarks [29, 37] and two input-specific ones [38, 9] . Regarding the implementation of existing watermark techniques, we follow their default settings. The example of verification samples across different watermarks and verification methods is shown in Figure 4 . Please find more details in the appendix.Evaluation Metrics.We adopt mean square error (MSE), neuron activation similarity (NAS), and mutual information (MI) to measure the degree to which the dataset-specified watermarks are disclosed during the verification stage. Specifically, the MSE is defined as the mean square error between verification and their corresponding watermarked samples in the region of watermark patterns. NAS is calculated as the cosine similarity in the neuron activation map between verification and their corresponding watermarked samples. MI is calculated based on the distribution of verification and their corresponding watermarked samples. More details are in our appendix.Results. As shown in Table 1 , our approach produces larger cosine similarity scores of the target label than benign labels. We show the results regarding the watermark-disclosed degree for different approaches in Table 2-3. The results show that our approach can reach the most minor watermark- disclosed degree in almost all cases. The improvement is significant compared to the vanilla DOV methods. These results verify the effectiveness and security of our ZeroMark.Settings. We evaluate our ZeroMark for ownership verification under three scenarios, including (1) independent watermark (dubbed 'Independent-W'), (2) independent model (dubbed 'Independent-M'), and (3) unauthorized dataset training (dubbed 'Malicious'). In the first case, we used ZeroMark to verify the suspicious model affected with other watermark patterns; In the second case, we test the benign model with our ZeroMark; In the last case, we use ZeroMark to verify the watermarked model with corresponding ground truth watermark samples. Notice that only the last case should be regarded as having unauthorized dataset use. More setting detail are described in the appendix.Evaluation Metrics. Following the settings in [7, 6] , we use ∆P ∈ [-1, 1] and p-value ∈ [0, 1] for the evaluation. For the first two independent cases, a small ∆P and a large p-value are expected. In contrast, for the third one, the larger ∆P and the smaller the p-value, the better the verification.Results. As shown in Table 4 , our method can achieve accurate dataset ownership verification in all cases. Specifically, our approach can identify the unauthorized dataset usage with high confidence (i.e., p-value ≪ 0.01 for 'Malicious' case), while not misjudging when there is no unauthorized dataset utilization (i.e., p-value ≫ 0.1 for 'Independent-W' and 'Independent-M').We hereby study the effects of two key hyper-parameters. More results are in our appendix. Effects of Optimization Iterations t. We exploit BadNets and DW as the representatives of patch-based and sample-specific watermarks to study the effects of t on both CIFAR-10 and Tiny-ImageNet datasets. As shown in Figure 5 , the cosine similarity scores increase with the increase of t. With t ≥ 30, we can easily distinguish watermarked and benign models.Effects of the Largest Number of Verification Samples (i.e., m × Q%). As shown in Figure 6 , with the number of verification samples used in the t-test increases, the cosine similarity decreases while the benign models' cosine similarity remains stable on both datasets. However, our method can still have a promising cosine similarity even with many verification samples (e.g., 150).We hereby select domain watermark to evaluate the robustness of ZeroMark, as the domain watermark itself is sufficiently robust against this defense. Following the previous work [7] , we here evaluate the robustness of our ZeroMark under fine-tuning [39] and model pruning [40] . As shown in Figure 7 , fine-tuning has minor effects on our method. In Figure 8 , we can see ZeroMark performs resilient against model pruning as its efficacy decreases along with the domain watermark.The results of our resistance to other methods are in our appendix.In this section, we intend to further explore the mechanisms behind the effectiveness of our Zero-Mark. Specifically, we adopt t-SNE [41] to visualize the feature distribution of watermark samples, benign samples, and samples generated by ZeroMark of watermarked DNNs. As shown in Figure 9 , with varied optimization iterations t for generating (closest) boundary samples, samples generated by ZeroMark can always stay far away from the watermark samples' distribution, which demonstrates ZeroMark can prevent disclosing the watermark information from the watermark samples. 
potential limitations and future directions: 
 ZeroMark can conduct dataset ownership verification without disclosing the watermark pattern. However, as the first work towards a secure verification process of dataset ownership verification (DOV) methods, we have to admit that we still have some potential limitations.Firstly, we must admit that our method requires additional time to conduct ownership verification since it needs to generate some boundary samples and their gradients under the black-box setting. For example, it takes nearly 30 mins for verification on CIFAR-10. While this cost is acceptable in practice to a large extent, we will discuss how to accelerate our ZeroMark in our future work.Secondly, ZeroMark currently can only perform effectively for watermark techniques with a predefined target label (e.g., BadNets [29] , etc. For other watermark techniques, which have no predefined target label (i.e., UBW [7] ), ZeroMark can not conduct boundary gradient analysis and, therefore, is not a feasible solution. Moreover, through extensive experimental evaluation, Zero-Mark is shown to perform more effectively for Blended watermark, compared with other watermark techniques. Such observations inspire us to improve the effectiveness of ZeroMark by designing the potential watermark patterns in a blended manner. Finally, the boundary gradient analysis step in ZeroMark may incur variance among different labels' samples. In practice, we can release such variance by training a surrogate model with the protected dataset and calculating cosine similarity between the corresponding boundary gradients and the target watermark pattern with samples from different labels. Then we use the calculated cosine similarity using samples from different labels under the surrogate model to adjust the cosine similarity calculated under the suspicious model. Specifically, we can adjust the cosine similarity calculated with samples from a specific label t by subtracting it from the average cosine similarity calculated with samples from the same label t under the surrogate model. We will further discuss these issues in our future work.Thirdly, we can only empirically verify that malicious dataset users cannot recover dataset-specified watermarks based on our boundary samples. We will try to prove it theoretically in the future.Fourthly, we are currently focusing on convolutional neural networks (e.g., ResNet and VGG) and the continuous image modality. In general, the success of our approach on other model structures depends on two factors: (1) whether the studied dataset watermarking method (e.g., BadNets) can successfully watermark these models and (2) whether we can conduct effective 'adversarial attacks' to find boundary samples on these models. Based on existing work related to backdoor attacks/dataset watermarking [42, 9] and adversarial attacks [43, 44] , these factors are all met. As such, our method can fundamentally generalize to other models (e.g., transformer) as well. As for the generalizability to other (discrete) data formats like tabular or text, the main challenge lies in how to design effective adversarial attacks to them for finding the closest boundary samples (as in Eq.( 10)). In particular, there are already some relevant works [45, 46] confirming its feasibility. Accordingly, our ZeroMark can be naturally adapted to other discrete data formats (e.g., text and scientific data [47, 48, 49, 50, 51] ). We will discuss them in our future work.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We have detailed the training configurations and optimizers in Appendix G Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 In this paper, we revisited existing DOV methods and revealed that their underlying assumption regarding the verification phase does not necessarily hold in practice. Accordingly, directly using dataset-specified watermarks for verification is insecure. Motivated by these findings, we proposed ZeroMark to conduct ownership verification without disclosing them. Our method was inspired by our empirical and theoretical findings of the intrinsic property of DNNs trained on the watermarked dataset. We conducted experiments on benchmark datasets, verifying the effectiveness of our Zero-Mark and its resistance to potential adaptive methods. We hope our work can provide a new angle of dataset ownership verification to facilitate more secure and trustworthy dataset sharing.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We have reported the distribution of our results in Figure 2 and Appendix B Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We included the details for computer resources in the Appendix G.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We follow the NeurIPS Code of Ethics https://neurips.cc/public/ EthicsGuidelines Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We included the social impact in the Appendix.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]  
 
 Appendix: We follow the previous work [32] to use a model f (•; w) infected with the basic backdoor-based watermark (i.e., G x = (1 -m) ⊙ ∆ + m ⊙ x) [29] as a basic example to shed light on the intriguing characteristic of watermark model. Theorem 1 (Property of Boundary Gradient on the Closest Boundary Sample). Assume that ϕ y (x t ; w) is twice differentiable with a Lipschitz gradient, if |D m | → ∞ and by updating x t in Eq. ( 4) with step size β t = ||x t -x 0 || 2 • t q-1 , there exists a constant c ≥ 0 such thatEQUATION)where q ∈ ( 1 2 , 1), y t is the target label (i.e., y t = C(G x (x))), δ is the watermark pattern (i.e., δ ≜ G x (x 0 ) -x 0 ), and t * is the number of convergence iterations of x * 's update.Recall in Eq. ( 4), we update x t for each t -1 th iteration as:EQUATIONLet the step size β t in Eq. ( 4) as t -q ||x t -x 0 ||, we have the distance ratio for updating Eq. ( 4) as:EQUATION)With a second-order tylor expansion, we have:EQUATIONCombining these Eq. ( 4) and Eq. ( 15), we have:EQUATIONwhere we define v t as x t -x 0 + τ t ∇ x ϕ y (x t ; w) and τ t as t -q ||xt-x0|| ||∇xϕy(xt;w)|| . Solving for α, we have:EQUATION)Therefore, we can get:EQUATIONwhere d t = x t -x 0 and r t := cos∠(x t -x 0 , ∇ x ϕ y (x t ; w)) = Let k t := 3 2 L ||dt||2 ||∇t||2 . Then k t can be bounded when ||∇ t || 2 ≥ C and q ≥ 1 2 , thus we have:EQUATIONMotivated by previous work [52] , solve Eq. ( 20) and have :EQUATIONwhere c 1 , c 2 are two positive constants, thus the above equation is o(t -1).When q ∈ ( 1 2 , 1), we have:EQUATION)Therefore, we have:EQUATIONNotably, Eq. ( 4) with step size as t -q ||x t -x 0 || converges a stationary point of Eq. ( 3). Motivated by proof for Lemma 3 in [34] , when x t is optimized to a stationary point (i.e., x * ) in t * and if x t belongs to the watermark label y t , we have:EQUATIONandEQUATION)Hence, when |D| m → ∞, we have:EQUATIONtherefore, for the watermark label (i.e., y t ):EQUATIONB Detailed Settings for Empirical Study in Section 3.3In the Section. 3.3, we select ResNet-18 and CIFAR-10 as the evaluated model and benchmark. We select the class '0' as the watermark class and inject watermark samples with 10% watermark ratio to ensure the verification success rate ≥ 99%. We randomly select 300 samples from the validation set across classes. We use these selected validation samples to generate boundary points for watermark and benign labels labels following Eq. ( 4). In particular, for the boundary point of watermark label y t , we set x 0 as samples from classes different from y t and set x t as samples from the watermark label. As for the boundary point of benign labels, we set x 0 as samples from the watermark class and set x t as samples from the benign labels. As such , there should be 400 boundary gradients for the watermark or benign label. We then calculate the boundary gradients following the gradient estimation process as Eq. ( 10). We also conduct same empirical studies on additional three different types of watermarks (i.e., Blended [37] ,WaNet [38] and Domain Watermark [9] ) to validate the characteristic of boundary gradients for watermarked and benign labels. Specifically, we conduct empirical studies with CIFAR-10 using ResNet-18 and select '0' as the watermark label. We can see that in all these three watermark patterns, the distributions of cosine similarity for watermarked labels have significantly large ranges compared with that of benign labels. Interestingly enough, we find that the distribution of cosine similarity for Blended watermark has a more obvious separation with that of benign labels compared with other watermark patterns. We will speculate the reason for this phenomenon in our future work.We here describe how to mitigate the variance caused by the watermark patterns and the iterative gradient estimation in details.Mitigate the Variance Caused by the Watermark Patterns. Based on the available ground truth watermark pattern δ, we create several (i.e., 6) artifact watermark patterns {δ i } 10 i=1 which have the same location map (i.e.m) as the ground truth watermark but filled with different random noise. We calculate the baseline cos as:EQUATIONand we calculate the normalized cosine similarity score as:EQUATIONMitigatwe the Variance in the Iterative Gradient Estimation Procedure. During the procedure of gradient estimation for Eq. ( 10), the estimated gradients could yield variance among iterations for the gradient estimation process. Therefore, to mitigate such variance, we propose to average the estimated gradients over iterations for the gradient estimation process, which can be formulated as:EQUATIONWe here describe the metrics for evaluating each approach in details. With loss of generality, we here define each give watermark sample as:EQUATIONwhere x i and t i represent the benign sample and the corresponding watermark pattern. We suppose t i is located at the dims of [j : k](j < k). The evaluation metrics, including mean square error (MSE), neuron activation similarity (NAS) and mutual information (MI) are defined as below: •MSE: = 1 N N i=1 (x ′ i [j : k] -x v [j : k]) 2 . • NAS: = 1 N N i=1 cos∠(F (x ′ i ), F (x v ))). • MI:= E p(zv,z ′ ) [log p(z ′ |z v )] -E p(zv)p(z ′ ) [log p(z ′ |z v )].where x v , F (•) represent the verification samples, feature extractor for the corresponding watermark model. z represents the feature extracted by the watermark model. Since ZeroMark uses several perturbed boundary points for calculating the boundary gradient for each given sample, we average their values for computing each metric. Notably, we follow previous work [53] to estimate MI by calculating its upper bound, as follows:EQUATIONWe put the detailed algorithm for ZeroMark as follows:G Experiments DetailsWe evaluate our approach on two widely-adopted benchmark datasets (i.e., CIFAR-10 [4], Tiny-ImgaeNet [36] ). We here describe each benchmark dataset in detail. Evaluated Watermarks. In our experiments, we evaluate four types of watermark, including Bad-Nets [29] , Blended [37] , WaNet [38] , and domain watermark [9] . The visual demonstration for each watermark is shown in Sec. 4. For BadNets, we implement a 4 × 4 and 8 × 8 triggers for CIFAR-10 and Tiny-ImageNet. The trigger is filled with random noise. For Blended watermark, we implement a 4 × 4 and 8 × 8 triggers for CIFAR-10 and Tiny-ImageNet. We set the transparency ratio as 0.2 throughout experiments. As for WaNet, we use BackdoorBox2 [54] to build the watermarked model with its default configurations. For Domain Watermark, we implement it following its released code 3 . We set the watermark rate γ as 0.1 consistent with previous work [7] for training different watermark models.To train DNN models, we use Adam optimizer [55] with the initial learning rate as 0.01. The watermark models evaluated in our experiments can achieve ≥ 92.26% and 56.8% accuracy on validation dataset for CIFAR-10 and Tiny-ImageNet tasks. We use six NVIDIA RTX 2080 Ti GPUs for performing experiments.We here describe the comparison approaches in details, as follows.Existing DOV. We follow DOV approaches [29, 37, 38, 9] to directly exploit the watermark samples for verification.Minimal Watermark. Inspired by previous work [56] for Trojan detection, which applies reverse engineering to generate the pseudo trigger patterns with minimize size while preserving their attack efficacy. Specifically, we generate the minimal watermark for the watermark sample x ′ = x + δ following:EQUATIONwe follow Neural Cleanse [56] 's configurations for conducting optimization on Eq. (34) . For inputspecific watermark patterns (e.g., WaNet [38] , DW [9] ), we conduct Eq. ( 34) on each watermark sample. As such verification samples with the minimal watermark can be formulated as: Watermark with Distortion. Motivated by previous work [32] , which reveals that watermark samples can perform resilient against certain magnitude of random noise distortion, we thus propose to add maximum magnitudes of random noise for watermark samples to hinder the watermark pattern while preserving the watermark's efficacy. We generate the noisy watermark samples via:EQUATIONEQUATIONwhere µ ∼ N (0, 1) is Gaussian noise and P roj is the projection function to constrain x ′ + a * µ into [0, 1]. We solve a using grid search to find the largest a while preserving the verification success rate for watermark samples.We here show additional results in our experiments. We perform t-SNE clustering analysis for other three types of watermark (i.e., Blended, WaNet, Domain Watermark) with varied optimization iterations t. The results are shown in Figure 13 , Figure 14 and Figure 15 . The additional visual demonstrations for boundary samples within the verification procedure are shown in Figure 17 ,Figure 18 , Figure 19 and Figure 20 . We also evaluate ZeroMark with different model architectures. Specifically, we here evaluate ZeroMark using VGG-19 [57] with CIFAR-10 task and the configurations are consistent with Section. 3.3. The results are shown in Figure 16 , which demonstrates that ZeroMark can still perform effective on VGG-19 models.We here investigate whether ZeroMark can perform robustness against the potential adaptive attack for recovering or unlearning the watermark pattern.Recovering the Watermark Pattern from the Boundary Samples. We here explore the potential adaptive attack for recovering the watermark pattern. Since we leverage gradient estimation via aggregating the random noise for conducting boundary gradient analysis, we here consider an adaptive attack by aggregating corresponding boundary samples for each input sample x 0 to recover the watermark pattern. We conduct experiments using ResNet-18 under CIFAR-10 task and we evaluate the adaptive attack with Domain Watermark [9] . The results are shown in Figure 21 and Figure 22 . We find that the aggregated boundary samples can not reveal the watermark pattern from both visual and clustering analysis.Unlearn the Watermark through Boundary Samples. We here consider whether we can follow [56] to unlearn the watermark pattern via boundary samples. We retrain the suspicious model with 500 boundary samples and label them as their original label (i.e., label for x 0 ) along with the training data. We find that the accuracy of the suspicious model drops from 92.3% to 90.2% and the verification success rate for domain watermark drops from 88.6% to 75.1%, and the ZeroMark can still achieve the averaged largest Q% as 0.076 for the target label, which significantly larger than that of the benign labels(i.e.,0.028). This results demonstrate that ZeroMark can prevent disclosing watermark patterns during the verification procedure within DOV.In the appendix, we provide detailed descriptions of the datasets, models, training and evaluation settings, and computational facilities. We provide the codes and model checkpoints for reproducing the main experiments of our evaluation in the supplementary material.In this paper, we focus on the copyright protection of public datasets. Specifically, we reveal that the verification process of existing DOV methods is not secured and propose using boundary samples to conduct verification without disclosing the watermark. This work has no general ethical issues since our method is purely defensive and does not reveal any new vulnerabilities of DNNs. In this paper, all adopted samples are from the open-sourced datasets (i.e., CIFAR-10, Tiny-ImageNet). The Tiny-ImageNet dataset may contain a few human-related images. We admit that we modified a few samples for watermarking and verification. However, our research treats all samples the same and the verification samples and modified samples have no offensive content. Accordingly, our work fulfills the requirements of these datasets and has no privacy violation. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.https://github.com/THUYimingLi/BackdoorBox.githttps://proceedings.neurips.cc/paper_files/paper/2023/hash/ aa6287ca31ae1474ea802342d0c8ba63-Abstract-Conference.html 


Decision: The paper receives mostly positive comments on the importance of the addressed task and the novelty of the approach. One reviewer (s8Mh) raised several concerns and the authors discuss with the reviewers in multiple rounds by clarifying most of the reviewer's concerns and the reviewer did not follow up the final comments, which are quite important for the final decision. Reading all review comments and authors' follow-ups, the AC recommends to accept the submission. Please revise the manuscript by the all reviewers comments for the camera ready.

Reviews: review: paper_summary: paper_summary: This paper proposes ZeroMark, a novel scheme for dataset watermark verification. It is based on an observation that the boundary gradients (i.e., gradients of samples near the decision boundary of a watermarked model) of the watermark target class tend to have higher cosine similarities with the watermark pattern than the boundary gradients of benign classes. ZeroMark hence proposes to verify the watermark based on the cosine similarities between the boundary gradients and the watermark pattern. In this way the actual watermark pattern is not disclosed to the adversary during verification, thus making watermark verification more practical as it prevents the adversary from exploiting the leaked watermark pattern. Experimental results show that ZeroMark leaks little information about the original watermark pattern, achieves high verification performance and is relatively robust against removal attempts.
strength_weakness: strength_weakness: 1. The proposed method is primarily evaluated on image datasets and convolution networks (ResNet and VGG) and lacks a discussion on the potential applicability to other data formats (e.g., tabular or text) or model architectures (e.g., transformers).
2. The adaptive attacks discussed in the appendix lack details on experimental settings.
3. Some parts of the presentation in the manuscript are confusing or unclear.
questions: questions: 1. (Applicability to other data formats or model architectures) The evaluation on image datasets is quite comprehensive. However, it would be better if the authors could add a discussion on how ZeroMark extends to other data formats or model architectures. For example, for text data, it might be difficult to construct boundary examples using Eq.10 since texts are discrete.
2. (Adaptive attack setting) The paper mentions two adaptive attacks in appendix J. However, the appendix lacks some details for the adversary's setup (e.g., steps for recovering the watermark pattern, or hyper-parameters for unlearning the watermark).
3. (Clarity) The "largest Q% cosine similarity" used in the experiment is confusing: does this Q has a specific value in the experiments?
4. (Clarity) How the cosine similarity is computed seems confusing. In Eq.29 in the appendix, the cosine similarity is computed on the watermark patch, identified by a "location map $m$". However, this location map is not mentioned in Eq.8 in the main body.
5. (Clarity) Fig.4 presents 5 different samples constructed by ZeroMark, but it lacks an explanation on the differences among the 5 samples.
limitations: limitations: The authors have acknowledged some limitations of ZeroMark in Appendix K, including (1) additional time overhead due to the construction of boundary samples and (2) theoretical guarantee on the security of ZeroMark.
paper_summary: paper_summary: This paper explores how to conduct privacy-preserving dataset ownership verification without directly disclosing dataset watermarks. The proposed method is inspired by the characteristic of boundary gradient of watermarked DNNs. Specifically, it has three main steps, including (1) generate the (closest) boundary samples, (2) calculate the boundary gradients of the generated boundary samples, and (3) dataset ownership verification via boundary gradient analysis. The authors conduct experiments on CIFAR-10 and Tiny-ImageNet datasets with four baseline methods.
strength_weakness: strength_weakness: 1.	The motivation section is inspiring. However, the authors should provide more explanations about how they came up with this idea and how to further explore it. It is crucial to highlight follow-up research.
2.	The authors should justify the phenomenon in Figure 2 is not exists in benign models. 
3.	It would be better to explain more about how Theorem 1 relates to the proposed method.
4.	The authors also used BadNets in the experiments of Section 3.3. It would be better to also show results of other watermarking methods, especially those with non-patch-based watermarks.
5.	The authors introduce a normalization process in the calculation of cosine similarity score. The authors should conduct ablation study to verify its effectiveness.
6.	Missing setting details used for the zero-order gradient estimation (i.e., N and \beta_t).
7.	The authors should also analyze the visual similarities between the optimized verification samples and their benign version, as well as with the ground-truth dataset watermark, w.r.t. to the optimization process.
8.	The authors should also conduct experiments on methods with non-compact watermarks.
9.	The authors should provide more explanations in Section 5.5.

There are still some typos. For example, ‘method’ should be ‘model’ (Line 16, Page 1).
questions: questions: Please see the weaknesses part.
limitations: limitations: Yes
paper_summary: paper_summary: This work presents a method for dataset ownership verification, focusing on confidentiality during the verification phase. The authors highlight that adversaries can remove watermarked data by detecting it during verification. To address this issue, the paper proposes a new verification process based on the cosine similarity between the watermark pattern and the gradient at boundary samples. Building upon their observations and Theorem 1, the authors use the similarity between these two components as evidence of unauthorized dataset use.
strength_weakness: strength_weakness: However, I have some concerns as follows:

1) The authors state that "Theorem 1 indicates that the cosine similarity between the watermark pattern and the gradient increases along with the update process." However, I'm not convinced. As I understand it, cosine similarity must be within the range [-1, 1], so the left term in Eq (6) may be within [0, 2]. Then, 
t* represents the number of iterations, and the authors used 10-40 iterations as shown in Figure 5 (always a positive integer). This implies a simple condition where c>0. In this case, I question whether the lengthy proof is necessary. 


2) Furthermore, Theorem 1 does not imply a proportional relationship between the two terms. It suggests Lipschitz continuity, not a positively increasing relationship. Therefore, I am not convinced about the reliability of the proposed verification method.


3) Furthermore, I find this method to be rather impractical. The proposed approach necessitates the predicted logits for generating boundary data and conducting verification. However, full logits are generally not available in commercial services. This requirement appears impractical to me. In contrast, other methods, such as BadNet or Blended, can perform verification using only the one-hot encoded predictions, indicating the Top-1 prediction.  Although they can verify using only Top-1 predictions, all results in this paper rely on the logits. For a fair comparison, I believe the authors should compare their method with others that also use logits for verification.


4) I find the experiments (e.g., ResNet/VGG on CIFAR10/TinyImageNet without error bars) insufficiently convincing. In addition, the authors mention, "We have reported the distribution of our results in Figure 2, and Appendix B" at the checklist, but they only describe histograms of 400 samples. I don't consider the histograms to be a substitute for error bars because the results are achieved from a single model, which must represent reliability through multiple suspicious/clean models. 


5) The proposed method primarily uses backdoor attacks with label noise as a baseline. However, there are clean-labeled verification approaches, such as radioactive data [1]. I believe the property of clean labels is highly important in dataset watermarking, but I'm unsure whether the proposed method can work with clean-labeled data.


[1] Sablayrolles, A., Douze, M., Schmid, C., & Jégou, H. (2020, November). Radioactive data: tracing through training. In International Conference on Machine Learning (pp. 8326-8335). PMLR.
questions: questions: 1) For the confidentiality (or security) of the proposed method, isn't it possible to reconstruct the watermark (BadNet, Blended, and WaNet) using multiple boundary samples? According to Figure 4, the proposed verification requires multiple gradients for these boundary samples. If I were the adversary, I might attempt to reconstruct the watermark pattern using the multiple boundary samples and their corresponding gradients.

2) I expect that the proposed method is affected by data augmentation techniques such as Flip, MixUp, or random noise injection. Additionally, I think the proposed method can be compromised by random dropout. For example, if random flip was applied during training of the suspicious model, the boundary gradient would become less correlated to the watermark pattern because the model would have also seen its mirror pattern.
limitations: limitations: As I mentioned, the authors doesn't represent the error bars.
paper_summary: paper_summary: This paper proposes a new dataset ownership verification method, ZeroMark, which calculates the boundary gradient between the benign and reconstructed images to verify the authorship. Extensive experiments are conducted on two datasets with four backdoor attack methods to evaluate the ZeroMark.
strength_weakness: strength_weakness: 1) In Lines 40~44, the authors mention that the problem contained in existing methods is the leakage of the watermark pattern. This issue is not reasonable since keeping the safety of the watermark is a significant requirement for all watermarking methods. The embedded watermark can also be removed when ZeroMark releases the original watermark pattern. 

2) As shown in Fig.2 (d), there is no clear bond between the benign samples and target labels. Distinguishing the distribution to clarify the authorship is may not believable among various images.

3) The novelty of this paper is limited. Using backdoor behaviors [1] or their distributions to protect the dataset shows similar forms.

4) In Eq. (5), is it correct to calculate the cosine similarity between the image x and its gradient?

5) In Fig. (4), the trigger shown in Blended is wrong. Blended injects a Hello Kitty or a pair of sunglasses as the trigger.

6) Why does the smaller MSE denote the better performance? From my view, the MSE indicates the difference between the original watermark and the boundary region. The small value denotes that the boundary region is more like the watermark.

7) Although the authors discussed some limitations in future work, the issues of robustness and efficiency of ZeroMark also hinder the practicability. ZeroMark is a backdoor-based DOV method, the state-of-the-art backdoor defense methods should be included in the experimental section.

[1] Guo J, Li Y, Wang L, et al. Domain watermark: Effective and harmless dataset copyright protection is closed at hand[J]. Advances in Neural Information Processing Systems, 2024, 36.
questions: questions: In line 27, the authors believe that DOV is the only feasible way to protect the copyright of public datasets currently. However, Unlearnable examples have been studied to prevent unauthorized data exploitation [1]. How do you think about it?

[1] Huang H, Ma X, Erfani S M, et al. Unlearnable Examples: Making Personal Data Unexploitable[C]//International Conference on Learning Representations (2021).
limitations: limitations: The authors have discussed the limitations.

Comments: title: Responses to Post-rebuttal Comments (Round II)
Response to authors.
Thank You for Your Positive Feedback and Follow-up Response
A Gentle Reminder of the Post-rebuttal Discussion
A Gentle Reminder of the Post-rebuttal Discussion
Responses to Post-rebuttal Comments (Round 1) [Part 2]
Responses to Post-rebuttal Comments (Round 1) [Part 1]
Thanks to Reviewer s8Mh
Thanks to Reviewer RJcM
Thanks to Reviewer VV25
Thank You for Your Positive Feedback!
Responses to Post-rebuttal Comments (Round I) [Part 2]
Responses to Post-rebuttal Comments (Round I) [Part 1]
Response to rebuttal
comment: Dear Reviewer VV25,

We sincerely thank you for your timely follow-up feedback! We are so glad that our previous responses clarified potential misunderstandings and addressed your concerns. We also deeply thank you for your positive feedback, especially comments regarding our interesting topic and comprehensive evaluation. It encourages us a lot! We promise to add more discussions and experiments that we previously committed to in the revision.

We also fully understand your concern and your decision to keep the current score. Empirically, it is difficult for an attacker to recover the watermark patterns from our boundary samples because they contain limited trigger information, measured by mutual information, etc. The information used throughout the optimization process (i.e., the benign samples and the model) is also known to the attacker in advance. However, we do agree that a theoretical guarantee would have been more persuasive, yet we failed to provide it (as we admitted in our limitation analysis). We will further analyze it in our future work.

Thank you again for your feedback. Your valuable time and constructive comments are highly important to us!
The reviewer would like to thank the authors for the timely response. It has addressed the previous concern. Overall, the paper addresses an interesting topic and the evaluation on image datasets have been quite comprehensive. However, the reviewer decides to maintain the original score, primarily due to the following reason.

The proposed method lacks a theoretical guarantee on the privacy/secrecy of the trigger pattern. ZeroMark proposes to use boundary samples so that the actual trigger pattern is not directly exposed. However, this mainly relies on a heuristic that hiding the actual trigger pattern would hinder the adversary from reconstructing it. This is empirically verified with adaptive attacks, but a theoretical guarantee would have been more persuasive, especially for privacy-preserving tasks. (Note that Theorem 1 in the paper only proves the cosine similarity between the trigger pattern and boundary sample is related; it does not guarantee that it is difficult/impossible for an adversary to reconstruct the trigger.)
Dear Reviewer s8Mh,

Thank you for your follow-up comments. We greatly appreciate your efforts to help us improve our paper. However, we find that there are still some potential misunderstandings that may significantly mislead you. We hereby provide more explanations to address them.


---
**Q1**: ZeroMark is indeed a verification method, however, its effectiveness is restricted to the backdoor attack methods that are employed. If malicious users adopt defense methods during the model training process, then ZeroMark cannot verify the copyright successfully, resulting in low practicality.

**R1**: Thank you for your comments. We are deeply sorry that our previous rebuttal may have led you to some misunderstandings that we want to clarify here.

- As we have mentioned before, **our ZeroMark method is not restricted only to backdoor-based watermarks**. For example, it can also be used to Domain Watermark, which is non-backdoor-based.
- To the best of our knowledge, **there is no training-phase backdoor defense that can simultaneously defend against all types of backdoor attacks**. As such, even just for backdoor watermarks, we can't assume that users have been able to easily remove them completely through existing backdoor defenses.
- We argue that **it is unfair to completely dismiss our work simply because baseline watermarks may be removed**. This is just a probability and it is out the scope of this paper (PS: it is about the robustness of dataset watermarks). 


---

**Q2**: The main idea of ZeroMark is to employ the reconstructed backdoored samples (i.e., boundary samples) to activate the injected backdoor. This is a straightforward application of backdoor defense, such as using backdoor defense methods to reconstruct the backdoored samples first and then to initiate backdoor attacks. The important boundary samples are generated by the existing method, FAB.

**R2**: Thank you for your comments. We are deeply sorry that our previous rebuttal may have led you to some misunderstandings that we want to clarify here.
- **Boundary samples are not backdoor samples**, as shown in its definition (Eq.(1)&Eq.(3)). As such, **our ZeroMark does not have any direct relation to the backdoor sample synthesis (or backdoor trigger inversion)**.

---
**The Definition of Boundary Samples**. Let the logit margin of model $f: \mathcal{X} \rightarrow [0,1]^K$ on the label $y$ is denoted by:
\begin{align}
    \phi_{y}(x;w) = f_{y}(x;w) - \max_{y'\neq y} f_{y'}(x;w).
\end{align}
It can be observed that $x$ can be classified as $y$ by $f(\cdot;w)$ if and only if $\phi_{y}(x;w) \geq 0$. As such, the set for boundary samples of class $y$ can be denoted by $\mathcal{B}(y;w) = \{x: \phi_{y}(x;w) = 0\}$. 

---

- **The core contribution of the paper is not how to compute the boundary samples**, but rather the definition of this new research problem and our empirical and theoretical findings of the intrinsic property of DNNs trained on the watermarked dataset. 
- We argue that **it is unfair to completely dismiss our work simply because we need to use existing techniques to calculate some intermediate results**. Otherwise, we can easily infer that almost all existing DL works has limited novelty, as they all require updating the model with PGD/SGD, etc.



---

**Q3**: Please find each reviewer correctly and respond to their comments.

**R3**: Thank you for your comments. You may have this misunderstanding because we summarized the key points of your original question in our rebuttal due to word limitations. Although we have answered your original questions in the order they were asked and have tried to maintain enough key information, we apologize for any misunderstandings we may have caused you. 

Please kindly let us know if we missed any points. We are very happy to answer them before the discussion period ends.



---
Thanks for your response. I haven't recognized the strong novelty of ZeroMark ever instead of concerning this issue. Detailed reasons are as follows:
1. ZeroMark is indeed a verification method, however, its effectiveness is restricted to the backdoor attack methods that are employed. If malicious users adopt defense methods during the model training process, then ZeroMark cannot verify the copyright successfully, resulting in low practicality.
2. The main idea of ZeroMark is to employ the reconstructed backdoored samples (i.e., boundary samples) to activate the injected backdoor. This is a straightforward application of backdoor defense, such as using backdoor defense methods to reconstruct the backdoored samples first and then to initiate backdoor attacks. The important boundary samples are generated by the existing method, FAB. 

Please find each reviewer correctly and respond to their comments. Thank you for understanding and I hope this will help improve your work.
Dear Reviewer s8Mh,

We sincerely thank you for your timely follow-up feedback! We are so glad that our previous responses clarified potential misunderstandings and alleviated most of your concerns. We also deeply thank you for your positive feedback, which encourages us a lot! We promise to add more discussions and experiments that we previously committed to in the revision.

In this response letter, we hope to further alleviate your remain concerns. We believe this can further improve our work. Thank you again for giving us this chance :)


---

**Q1**: However, there is still one slight concern with regard to ZeroMark's generalizability to discrete data formats (e.g., texts). While one can indeed construct boundary samples using existing adversarial attack methods, from what this reviewer understands, constructing the boundary sample marks only the first step for ZeroMark, after which one still needs to obtain boundary gradients and compute the cosine similarity between the boundary gradient and the watermark pattern. However, for discrete data, the input lies in a discrete space, and thus taking/estimating the gradient w.r.t. the input might not always be feasible. Additionally, for discrete data, the trigger pattern is usually also discrete (e.g., a fixed trigger token or a certain text structure/style), and hence computing cosine similarity could also be infeasible. This reviewer understands that due to page budget and time limitation, it could be difficult to conduct extra evaluations. Nonetheless, this reviewer is still concerned that ZeroMark might not readily applicable to discrete data formats.

**R1**: Thank you for these insightful comments!

- **We admit that our ZeroMark is not currently fully ready for discrete data**. As you agree, it's unlikely that we'll be able to fully address this issue in this work not to mention during this rebuttal. 
- Nonetheless, prompted and further inspired by your detailed and constructive comments, **we hereby discuss how we can possibly extend our work to discrete data formats** (e.g., texts). We believe this provides a solid foundation for our subsequent and follow-up works.
    - **Estimating the gradients w.r.t. the input**. The estimation of gradients is central to obtaining adversarial examples under the black-box setting. Adversarial examples are obtained by optimizing multiple rounds of estimated gradients based on the original samples. As such, **a well-performing black-box adversarial attack usually means that relatively more accurate gradient estimates can be obtained**. Besides, we can also use a pre-trained word embedding to approximately estimate the gradients. Of course, we admit that this is still only an approximation and an estimate, and its process is subject to some inaccuracies.
    - **Calculating the similarity between gradients and triggers.** We admit that we cannot directly calculate cosine similarity if the trigger pattern is also discrete. Thank you for the reminder! However, we argue that we still have some potential methods to alleviate it.
        - **Using an alternative similarity measure** since gradients and triggers should have some correlations.
        - **Using a meta-classifier to learn these correlations** if we cannot find a proper surrogate measure.
        - **Using a pre-trained word embedding to approximately transfer this discrete optimization to continuous optimization**.


We will add more details and discussions in the appendix of our revision. 

---
I would like to thank the authors for their detailed response and clarifications. The response has addressed most of the concerns, including ZeroMark's applicability to other model architectures and a few confusions in the manuscript.

However, there is still one slight concern with regard to ZeroMark's generalizability to discrete data formats (e.g., texts). While one can indeed construct boundary samples using existing adversarial attack methods, from what this reviewer understands, constructing the boundary sample marks only the first step for ZeroMark, after which one still needs to obtain boundary gradients and compute the cosine similarity between the boundary gradient and the watermark pattern. However, for discrete data, the input lies in a discrete space, and thus taking/estimating the gradient w.r.t. the input might not always be feasible. Additionally, for discrete data, the trigger pattern is usually also discrete (e.g., a fixed trigger token or a certain text structure/style), and hence computing cosine similarity could also be infeasible.

This reviewer understands that due to page budget and time limitation, it could be difficult to conduct extra evaluations. Nonetheless, this reviewer is still concerned that ZeroMark might not readily applicable to discrete data formats.
Dear Reviewer VV25,

We would like to sincerely thank you for your helpful comments. We hope our response has adequately addressed your concerns. We take this as a great opportunity to improve our work. We would be very grateful if you could kindly give any feedback to our rebuttal :)

Best Regard,

Paper4797 Author(s)
Dear Reviewer s8Mh,

We would like to sincerely thank you for your helpful comments. We hope our response has adequately addressed your concerns. We take this as a great opportunity to improve our work. We would be very grateful if you could kindly give any feedback to our rebuttal :)

Best Regard,

Paper4797 Author(s)
---
**Q2**: I am curious about the method's applicability to clean-labeled backdoor attacks, such as Sleeper Agent.

**R2**: Thank you for this insightful question! We do understand your concern since we did not evaluate our method on clean-label backdoor watermarks, although we have evaluated our ZeroMark on the SOTA non-backdoor-based clean-label watermarks (i.e., DW). We hereby provide more explanations and results to further alleviate your concerns.

- In principle, **our ZeroMark can be used for all backdoor watermarks, regardless of whether they have dirty-label or clean-label settings**, since our work targets the inference instead of the training process. 
    - **The predictive behavior of both types of attacks is consistent**, i.e., making the attacked models have a specific prediction (i.e., target label) whenever the trigger pattern appears.
    - **Both types of attacks lead to similar changes in the feature space**: the poisoned samples are clustered into a single cluster and moved away from the respective clusters of the other types of benign samples.
    - **Both types of attacks lead to changes in the decision surface that are highly correlated with triggers**, since both cause samples with triggers to be misclassified, of attacked models.
    - **Our Zeromark makes no assumptions about the label type**.
- To further alleviate your concern, **we conduct additional experiments on CIFAR-10 with label-consistent attack and SleeperAgent**. They are the first and the advanced clean-label backdoor watermark, respectively.
    - As shown in the following Table 3-4, **our method is still effective under clean-label backdoor watermarks**.
    - It is worth mentioning that our method requires that models already learn the watermark, which may require a larger watermarking rate for clean-label backdoor watermarks. However, **this problem stems from the intrinsic properties of these watermarks** (e.g., the antagonistic effects of 'robust features' [3]) **and is irrelevant to our ZeroMark**.

**Table 3.** The top $Q\%$ cosine similarity scores of ZeroMark on CIFAR-10 with ResNet-18.

| Label$\downarrow$, Dataset$\rightarrow$ | Sleeper Agent [1]| Label-Consistent [2] | 
|----------------|---------|---------|
| Benign         |  0.033   |0.029   |
| Target         | 0.120  |  0.104   |



**Table 4.** The verification efficacy of ZeroMark with label-consistent attack and SleeperAgent.

| Watermark$\downarrow$ | Scenario$\rightarrow$   | Independent-D | Independent-M | Malicious |
|-----------|------------|---------------|---------------|-----------|
| Sleeper Agent [1]  |     $\Delta P$ / p-value       |    0.012  / 1.00         |     0.010   / 1.00       |  0.087 / $10^{-48}$         |
| Label-Consistent [2]  | $\Delta P$ / p-value |     0.014 / 1.00              |  0.009  / 1.00           |      0.075  / $10^{-15}$   |


We will add more details and discussions in the appendix of our revision.



### References

1. Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch. NeurIPS, 2022.
2. Label-Consistent Backdoor Attacks. arXiv, 2019.
3. Not All Samples Are Born Equal: Towards Effective Clean-Label Backdoor Attacks. Pattern Recognition, 2023.


---
Dear Reviewer s8Mh,

We sincerely thank you for your timely follow-up feedback! We are so glad that our previous responses clarified potential misunderstandings and alleviated your concerns to a large extent. We also deeply thank you for your positive feedback, especially comments regarding our research importance, method effectiveness, and a new protection aspect. It encourages us a lot! We promise to add more discussions and experiments that we previously committed to in the revision.

In this response letter, we hope to further alleviate your remain concerns. We believe this can further improve our work. Thank you again for giving us this chance :)

---
**Q1**: I feel that the limited settings (CIFAR-10 and Tiny ImageNet with ResNet and VGGNet) are not sufficient to concretely verify general applicability.

**R1**: Thank you for these insightful comments! We do agree with you that generalizability and flexibility across models and datasets are also important for the the general applicability of a method. We hereby provide more explanations and results to further alleviate your concerns.

- **Generalizability to Other Model Architectures** (e.g., Transformer). 
    - In general, the success of our approach on other model structures depends on two factors: **(1)** whether the studied dataset watermarking method (e.g., BadNets) can successfully watermark these models and **(2)** whether we can conduct effective 'adversarial attacks' to find boundary samples on these models. Based on existing work related to backdoor attacks/dataset watermarking [1,2] and adversarial attacks [3,4], **these factors are all met**. As such, **our method can fundamentally generalize to other models (e.g., transformer) as well**. 
    - To further alleviate your concern, as you suggested, we hereby also evaluate our ZeroMark on the transformer archietecture. We empirically evaluate the effectiveness of ZeroMark on the TinyImageNet dataset using SwinTransformer with a patch size of 4. Other settings are consistent with our main experiments. **As shown in the following Table 1-2, our method is highly effective under SwinTransformer**. 

**Table 1.** The top $Q\%$ cosine similarity scores of ZeroMark on Tiny-ImageNet with SwinTransformer.

| Label$\downarrow$, Dataset$\rightarrow$ | BadNets | Blended | WaNet | DW    |
|----------------|---------|---------|-------|-------|
| Benign         | 0.034   | 0.034   | 0.039 | 0.041 |
| Target         | 0.096   | 0.221   | 0.124 | 0.119 |


**Table 2.** The verification efficacy of ZeroMark with SwinTransformer.

| Watermark$\downarrow$ | Scenario$\rightarrow$   | Independent-D | Independent-M | Malicious |
|-----------|------------|---------------|---------------|-----------|
| BadNets   | $\Delta P$ / p-value |     0.010 / 1.00              |  0.011  / 1.00           |      0.062  / $10^{-9}$   |
| Blended   |     $\Delta P$ / p-value       |    0.011  / 1.00         |     0.014   / 1.00       |  0.187 / $10^{-61}$         |
| WaNet     |    $\Delta P$ / p-value        |  0.017 / 0.99             |     0.011 / 1.00           |   0.085 / $10^{-46}$        |
| DW        |     $\Delta P$ / p-value       | 0.021 / 0.90              |  0.010 / 1.00             |        0.078 / $10^{-19}$   |

- **Generalizability to Other Datasets** (e.g., NLP Datasets with Discrete Data Format). 
    - **We conduct experiments only on these two image datasets simply following existing works** [2, 5]. Due to the limitation of paper length, it is also difficult for us to provide a comprehensive evaluation on other data types. 
    - However, we do understand your concern. Arguably, the main challenge lies in how to design effective adversarial attacks to discrete data formats for finding the closest boundary samples (as in Eq.(10)). In particular, there are already some relevant works [6, 7] confirming its feasibility. Accordingly, **our ZeroMark can be naturally adapted to other discrete data formats** (e.g., tabular or text). Due to the limitation of time, we cannot provide sufficient experiments to verify it in our rebuttal. We will further discuss it in our future works.

### References

1. You Are Catching My Attention: Are Vision Transformers Bad Learners under Backdoor Attacks? CVPR, 2023.
2. Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Hand. NeurIPS, 2023.
3. On the robustness of vision transformers to adversarial examples. ICCV, 2021.
4. On the Adversarial Robustness of Vision Transformers. TMLR, 2022.
5. Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection. NeurIPS, 2022.
6. TextCheater: A Query-Efficient Textual Adversarial Attack in the Hard-Label Setting. TDSC, 2023.
7. Query-Efficient and Scalable Black-Box Adversarial Attacks on Discrete Sequential Data via Bayesian Optimization. ICML, 2022.

---
Thank you for the thoughtful reply.

First, I agree on the importance of protecting the verification data, and I acknowledge that the proposed method addresses this problem effectively. Then, thanks for this new aspect of protection. 

Regarding Theorem 1, I apologize for my earlier misunderstanding. I now understand that the right term decreases as $t^*$ increases.

For the experiments, thanks for the additional results and explanations. They help me to understand the propose method. However, I feel that the limited settings (CIFAR-10 and Tiny ImageNet with ResNet and VGGNet) are not sufficient to concretely verify general applicability. I believe the evaluation could be more robust if more diverse settings were included. For example, I am curious about the method's applicability to clean-labeled backdoor attacks, such as Sleeper Agent.

Many of my concerns have been addressed, so I have raised my rating.
Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the strengths of our paper in terms of *good soundness and presentation*, *extensive experiments*, and *novelty*. 

We also sincerely thank you for your timely follow-up feedback. Please kindly let us know if our response have properly addressed your remaining concerns. We are more than happy to answer any additional questions during the post-rebuttal discussion period. Your feedback will be greatly appreciated.
Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the strengths of our paper in terms of *good presentation* and *novel verification process*. 

Kindly let us know if our response and the new experiments have properly addressed your concerns. We are more than happy to answer any additional questions during the post-rebuttal discussion period. Your feedback will be greatly appreciated.
Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the strengths of our paper in terms of *novel and interesting research problem*, *interesting observation*, *method flexibility*, and *extensive and solid experiments*.

Kindly let us know if our response and the new experiments have properly addressed your concerns. We are more than happy to answer any additional questions during the post-rebuttal discussion period. Your feedback will be greatly appreciated.
Dear Reviewer RHDz,

Thank you so much for your positive feedback! It encourages us a lot!

We are also thrilled to answer all follow-up questions during the discussion period.
**Q5**: Please don't over-claim your research topic. Unlearnable examples are a more feasible way to prevent unauthorized dataset usage than ZeorMark. The ZeroMark can only check whether someone has trained their models on your released dataset. If you check it, unauthorized dataset usage has already occurred.


**R5**: Thank you for your remind! We are deeply sorry that our previous rebuttal may have led you to some misunderstandings that we want to clarify here.

- **We did not intend to claim that dataset ownership verification is a more important or feasible area than unlearnable samples**.
- As you mentioned, unlearnable examples and dataset ownership verification are active and passive defenses regarding preventing unauthorized dataset usage, respectively. They have different settings and usage scenarios.
- While dataset ownership verification is an after-the-fact method that does not directly prevent unauthorized use of a dataset, **DOV is, unfortunately, the only way in some scenarios (such as the one that is the focus of this paper)**. Specifically, when releasing open-sourced datasets (e.g., ImageNet) and selling commercial datasets, **we need to ensure that the datasets are available without compromising utilities** (and, therefore, cannot use unlearnable examples methods).


We will add unlearnable examples in our related work and provide more discussions and explanations in our revision.

---
Dear Reviewer s8Mh,

We sincerely thank you for your timely follow-up feedback! We hereby provide more explanations to clarify some potential misunderstandings and further alleviate your remaining concerns.

---
**Q1**: ZeroMark essentially injects a BadNet (or Blended) trigger into the dataset to achieve dataset authorship verification, while only the inputted watermarked samples have been changed to boundary samples during the verification phase. An important issue that should be considered is that these triggers have very low stealthiness. The effectiveness of this method would be significantly degraded if malicious users use Sota defense methods when training their models, which has already been demonstrated by current backdoor defense methods [1, 2].

**R1**: Thank you for these comments and we do understand your concerns. We hereby provide more explanations to clarify potential misunderstandings that our submission and rebuttal may lead to you.

- **ZeroMark focused only on the verification process of DOV methods**. It can be used to improve existing DOV methods by **replacing their verification process with ours while keeping their original watermarking process**.
- We argue that **all your mentioned issues**, including the stealthiness of the watermark in the protected dataset and whether it can be detected during the training process **are irrelevant to our approach** since they are due to the property of user-specified watermarks instead of the verification process.
- **We validated the effectiveness of our approach on different types of watermarking methods, not just on these two most basic ones (i.e., BadNets and Blended)**. 
    - Specifically, we also evaluate our method on WaNet and Domain Watermark. **These watermarks are highly stealthy**. Specifically, WaNet exploited warping-based image modification for watermarking. Its watermarks are sample-specific and imperceptible; Domain watermark used small additive noise for watermarking. Its watermarks are sample-specific and imperceptible, and under the clean-label setting.
    - We have also conducted evaluations on domain watermark, which is the first non-backdoor-based dataset watermarking. **This watermark naturally bypass the detection of backdoor defenses**. Please find more details in our R4.


---
**Q2**: For the unclear boundary, an important and in-depth ablation study about the Q% has been lost.

**R2**: Thank you for this constructive suggestion! We have included the ablation study about the Q in our Ablation Study (Figure 6). **The results show that our method has a promising cosine similarity with various Qs**. 

---

**Q3**: Keeping the correct explanation about existing works (e.g., Blended) and equations (e.g., Eq. (5)) is crucial. If you "never" implement Eq. (5) in your experiment, why do you list it in your paper?


**R3**: Thank you for pointing it out!
- We will clarify our settings of Blended in our revision, as we mentioned and promised in our previous rebuttal.
- We are deeply sorry that our previous rebuttal may lead you to some misunderstandings. **What we meant before was that we didn't calculate the similarity between a sample and its gradient; this formula is just used to explain specifically how the similarity is calculated**. In our method, we calculate the similarity between the trigger and the boundary gradient, as in our Theorem 1 (Eq.(6)). If we don't introduce the formula for similarity first up front, writing it directly later would make Theorem 1 very lengthy and more difficult to understand.


We will add more explanations near Eq.(5) to avoid potential misunderstandings in our revision.


---
**Q4**: Domain Watermark is a backdoor-based DOV method that employs clean-label backdoor attacks to implement dataset protection.


**R4**: Thank you for this comment. However, as you may have misremembered or confused it with another work, **domain watermark is the first non-backdoor-based dataset watermark and DOV method**. It is clearly stated in its [original paper](https://arxiv.org/pdf/2310.14942) (Page 2, the second contribution). Specifically, backdoor-based methods made the watermarked model misclassify ‘easy’ samples that can be correctly predicted by the benign model. In contrast, domain watermark intended to make the watermarked model can correctly classify some 'hard' samples that will be misclassified by the benign model.

We will add more details and explanations in the related works of our revision.

---
I thank the authors for clarifying some of the questions. As there is no reaction regarding my concerns and suggestions in the weakness section. I am sticking with my original rating.

1. ZeroMark essentially injects a BadNet (or Blended) trigger into the dataset to achieve dataset authorship verification, while only the inputted watermarked samples have been changed to boundary samples during the verification phase. An important issue that should be considered is that these triggers have very low stealthiness. The effectiveness of this method would be significantly degraded if malicious users use Sota defense methods when training their models, which has already been demonstrated by current backdoor defense methods [1, 2].
  
2. For the unclear boundary, an important and in-depth ablation study about the Q% has been lost.
  
3. Keeping the correct explanation about existing works (e.g., Blended) and equations (e.g., Eq. (5)) is crucial. If you "never" implement Eq. (5) in your experiment, why do you list it in your paper?
  
4. Domain Watermark is a backdoor-based DOV method that employs clean-label backdoor attacks to implement dataset protection. 
  
5. Please don't over-claim your research topic. Unlearnable examples are a more feasible way to prevent unauthorized dataset usage than ZeorMark. The ZeroMark can only check whether someone has trained their models on your released dataset. If you check it, unauthorized dataset usage has already occurred.

  
I am thrilled to hear various opinions from the authors and other reviewers.

[1] Gao K, Bai Y, Gu J, et al. Backdoor defense via adaptively splitting poisoned dataset[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 4005-4014.

[2] Zhu M, Wei S, Zha H, et al. Neural polarizer: A lightweight and effective backdoor defense via purifying poisoned features[J]. Advances in Neural Information Processing Systems, 2024, 36.
I would like to thank the authors for providing a detailed rebuttal containing additional experiments. It addressed all my concerns. Well done!

I have also carefully read the comments from other reviewers, especially Reviewer RJcM and Reviewer s8Mh, since they have different judgments of this paper. After reading their comments and the rebuttal, I think their negative opinions are mostly due to misunderstandings, which are caused by the loss of some technical details or explanations. I believe the authors did a good job of clearing up these misunderstandings in their rebuttal. In addition, other expert reviewers raised some concerns that I had not thought of. But I think the authors also addressed them in the rebuttal, at least to me.

Given all these considerations, as well as insightful motivation, novelty, and potential impact on this field, I slightly increase my score.  I would also be happy to hear from other reviewers and participate in discussions :)