SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Junfeng Guo $^{1, *}$ , Yiming Li $^{2,3, *}$ , Lixu Wang $^{4}$ , Shu-Tao Xia $^{5}$ , Heng Huang $^{1}$ , Cong Liu $^{6}$ , Bo Li $^{7,8}$
$^{1}$ Department of Computer Science, University of Maryland
$^{3}$ School of Cyber Science and Technology, Zhejiang University
$^{4}$ Department of Computer Science, Northwestern University
$^{5}$ Tsinghua Shenzhen International Graduate School, Tsinghua University
$^{6}$ Department of Electronic and Computer Engineering, UC Riverside
$^{7}$ Department of Computer Science, University of Illinois Urbana-Champaign
$^{8}$ Department of Computer Science, University of Chicago
{gjf2023,heng}@umd.edu; li-ym@zju.edu.cn;
lixuwang2025@u.northwestern.edu;
xiast@sz.tsinghua.edu.cn; congl@ucr.edu; lbo@illinois.edu
The prosperity of deep neural networks (DNNs) is largely benefited from open-source datasets, based on which users can evaluate and improve their methods. In this paper, we revisit backdoor-based dataset ownership verification (DOV), which is currently the only feasible approach to protect the copyright of open-source datasets. We reveal that these methods are fundamentally harmful given that they could introduce malicious misclassification behaviors to watermarked DNNs by the adversaries. In this paper, we design DOV from another perspective by making watermarked models (trained on the protected dataset) correctly classify some 'hard' samples that will be misclassified by the benign model. Our method is inspired by the generalization property of DNNs, where we find a hardly-generalized domain for the original dataset (as its domain watermark). It can be easily learned with the protected dataset containing modified samples. Specifically, we formulate the domain generation as a bi-level optimization and propose to optimize a set of visually-indistinguishable clean-label modified data with similar effects to domain-watermarked samples from the hardly-generalized domain to ensure watermark stealthiness. We also design a hypothesis-test-guided ownership verification via our domain watermark and provide the theoretical analyses of our method. Extensive experiments on three benchmark datasets are conducted, which verify the effectiveness of our method and its resistance to potential adaptive methods. The code for reproducing main experiments is available at https://github.com/JunfengGo/Domain-Watermark.
Deep neural networks (DNNs) have been applied to a wide range of domains and have shown human-competitive performance. The great success of DNNs heavily relies on the availability of various open-source datasets (e.g., CIFAR [1] and ImageNet [2]). With these high-quality datasets, researchers can evaluate and improve the proposed methods upon them. Currently, most of these datasets limit their usage to education or research purpose and are prohibited from commercial applications without authorization. How to protect their copyrights is of great significance [3, 4, 5, 6].
*The first two authors contributed equally to this work. Correspondence to Yiming Li.
Figure 1: The main pipeline of dataset ownership verification with backdoor-based dataset watermarks and our domain watermark, where BW Sample represents existing backdoor-watermarked sample while DW Sample represents our proposed domain-watermarked sample. Existing backdoor-based methods make the watermarked model (i.e., the backdoored DNN) misclassify 'easy' samples that can be correctly predicted by the benign model and therefore the verification is harmful. In contrast, our ownership verification is harmless since we make the watermarked model correctly predict 'hard' samples that are misclassified by the benign model.
Currently, there are many classical methods for data protection, such as encryption [7, 8, 9], differential privacy [10, 11, 12], and digital watermarking [13, 14, 15, 16]. However, they are not able to protect the copyrights of open-source datasets since they either hinder the dataset accessibility (e.g., encryption) or require the information of the training process (e.g., differential privacy and digital watermarking) of suspicious models that could be trained on it.
To the best of our knowledge, backdoor-based dataset ownership verification (DOV) [3, 4, 5] is currently the only feasible approach to protect them, where defenders exploit backdoor attacks [17, 18, 19] to watermark the original dataset such that they can verify whether a suspicious model is trained on the protected dataset by examining whether it has specific backdoor behaviors. Recently, Li et al. [4] first discussed the 'harmlessness' requirement of backdoor-based DOV that the dataset watermark should not introduce new security risks to models trained on the protected dataset and proposed untargeted backdoor watermarks towards harmless verification by making the predictions of watermarked samples dispersible instead of deterministic (as a pre-defined target label).
In this paper, we revisit dataset ownership verification. We argue that backdoor-based dataset watermarks can never achieve truly harmless verification since their fundamental mechanism is making watermarked model misclassifies 'easy' samples (i.e., backdoor-poisoned samples) that can be correctly predicted by the benign model (as shown in Figure 1). It is with these particular misclassification behaviors that the dataset owners can conduct ownership verification. An intriguing question arises: Is harmless dataset ownership certification possible to achieve?
The answer to the aforementioned problem is positive. In this paper, we design dataset ownership verification from another angle, by making the watermarked model can correctly classify some 'hard' samples that will be misclassified by the benign model. Accordingly, we can exploit this difference to design ownership verification while not introducing any malicious prediction behaviors to watermarked models that will be deployed by dataset users (as shown in Figure 1). In general, our method is inspired by the generalization property of DNNs, where we intend to find a hardly-generalized domain for the original dataset. It can be easily learned with the protected dataset containing modified samples. Specifically, we formulate the domain generation as a bile-level optimization and leverage a transformation module to generate domain-watermarked samples; We propose to optimize a set of visually-indistinguishable modified data having similar effects to domain-watermarked samples as our domain watermark to ensure the stealthiness of dataset watermarking; We design a hypothesis-test-guided method to conduct ownership verification via our domain watermark at the end. We also provide theoretical analyses of all stages in our method.
In conclusion, the main contributions of this paper are four-folds: 1) We revisit dataset ownership verification (DOV) and reveal the harmful drawback of methods based on backdoor attacks. 2) We explore the DOV problem from another perspective, based on which we design a truly harmless DOV method via domain watermark. To the best of our knowledge, this is the first non-backdoor-based DOV method. Our work makes dataset ownership verification become an independent research field instead of the sub-field of backdoor attacks. 3) We discuss how to design the domain watermark and provide its theoretical foundations. 4) We conduct experiments on benchmark datasets, verifying the effectiveness of our method and its resistance to potential adaptive methods.
2.1 Backdoor Attacks
Backdoor attack $^{2}$  [23, 24, 25] is a training-phrase threat of DNNs, where the adversary intends to implant a backdoor (i.e., the latent connection between the adversary-specified trigger pattern and the target label) into the victim model by maliciously manipulating a few training samples. The backdoored DNNs behave normally while their predictions will be maliciously changed to the target label whenever the testing samples contain the trigger pattern. In general, existing backdoor attacks can be divided into two main categories based on the property of the target label, as follows:
Poisoned-Label Backdoor Attacks. In these attacks, the target label of poisoned samples is different from their ground-truth labels. This is the most classical attack paradigm and is more easily to implant hidden backdoors. For example, BadNets [17] is the first backdoor attack, where the adversaries randomly modify a few samples from the original dataset by attaching a pre-defined trigger patch to their images and changing their labels to the target label. These modified samples (dubbed poisoned samples) associated with remaining benign samples are packed as the poisoned dataset that is released to victim users for training; After that, Chen et al. [26] improved the stealthiness of BadNets by introducing trigger transparency; Nguyen et al. [27] proposed a more stealthy backdoor attack whose trigger patterns were designed via image-warping; Recently, Li et al. [4] proposed the first untargeted (poisoned-label) backdoor attack (i.e., UBW-P) for dataset ownership verification.
Clean-Label Backdoor Attacks. In these attacks, the target label of poisoned samples is consistent with their ground-truth labels. Accordingly, these attacks are more stealthy, compared to the poisoned-label ones. However, they usually suffer from low effectiveness, especially on datasets with a high image resolution or many classes, due to the antagonistic effects of 'robust features' related to the target class contained in poisoned samples [28]. Label-consistent attack is the first clean-label attack where the adversaries introduced untargeted adversarial perturbations before adding trigger patterns; After that, a more effective attack (i.e., Sleeper Agent [29]) is proposed, which crafts clean-label poisoned samples via bi-level optimization; Recently, Li et al. [4] proposed UBW-C, which generated poisoned samples for leading untargeted misclassifications to attacked DNNs.
2.2 Data Protection
Classical Data Protection. Data protection is a classical and important research direction, aiming to prevent unauthorized data usage or protect data privacy. Currently, existing classical data protection can be roughly divided into three main categories, including (1) encryption, (2) digital watermarking, and (3) privacy protection. Specifically, encryption [30, 7, 8] encrypts the whole or parts of the protected data so that only authorized users who hold a secret key for decryption can use it; Digital watermarking [31, 32, 33] embeds an owner-specified pattern to the protected data to claim the ownership; Privacy protection focuses on preventing the leakage of sensitive information of the data in both empirical [34, 35, 36] and certified manners [10, 37, 12]. However, these traditional approaches are not feasible to protect the copyright of open-source datasets since they either hinder the dataset accessibility or require the information of the training process that will not be disclosed.
Dataset Ownership Verification. Dataset ownership verification (DOV) is an emerging topic in data protection, aiming to verify whether a given suspicious model is trained on the protected dataset. To the best of our knowledge, this is currently the only feasible method to protect the copyright of open-source datasets. Specifically, it intends to implant specific prediction (towards verification samples) behaviors in models trained on the protected dataset while not reducing their performance on benign samples. Dataset owners can conduct ownership verification by examining whether the suspicious model has these behaviors. Currently, all DOV methods [3, 4, 5] exploit backdoor attacks to watermark the unprotected benign dataset. For example, [3] adopted poisoned-label backdoor attacks while [5] adopted clean-label ones for dataset watermarking. Recently, Li et al. [4] first discussed the 'harmlessness' requirement of DOV that the dataset watermark should not introduce new security risks to models trained on the protected dataset and proposed the untargeted backdoor watermarks. However, there is still no definition of harmlessness and backdoor-based DOV methods
In this paper, we focus on poison-only backdoor attacks, where the adversaries can only modify a few training samples to implant backdoors. Only these attacks can be used as the dataset watermark for ownership verification. Attacks with more requirements (e.g., control model training) [20, 21, 22] are out of our scope.
can never achieve truly harmless verification for they introduce backdoor threats. How to design a harmless DOV method is still an important open question.
3 Domain Watermark
Threat Model. Following existing works in dataset ownership verification [3, 4, 5], we assume that the defenders (i.e., dataset owners) can only watermark the benign dataset to generate the protected dataset. They will release the protected dataset instead of the original benign dataset for copyright protection. Given a third-party suspicious model that may be trained on the protected dataset without authorization, we consider the black-box setting where defenders have no information about other training configurations (e.g., loss function and model architecture) of the model and can only access it to obtain predicted probability vectors via its model API.
The Main Pipeline of Dataset Watermark. Let  $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^N$  denotes the benign training dataset. Let we consider an image classification task with  $K$ -classes, i.e.,  $\pmb{x}_i \in \mathcal{X} = [0,1]^{C \times W \times H}$  represents the image with  $y_i \in \mathcal{Y} = \{1, \dots, K\}$  as its label. Instead of releasing  $\mathcal{D}$  directly, the dataset owner will generate and release its watermarked version (i.e.,  $\mathcal{D}_w$ ). Specifically,  $\mathcal{D}_w = \mathcal{D}_m \cup \mathcal{D}_b$ , where  $\mathcal{D}_m$  consists of the modified version of samples from a small selected subset  $\mathcal{D}_s$  of  $\mathcal{D}$  (i.e.,  $\mathcal{D}_s \subset \mathcal{D}$ ) and  $\mathcal{D}_b$  contains remaining benign samples (i.e.,  $\mathcal{D}_b = \mathcal{D} - \mathcal{D}_s$ ). The  $\mathcal{D}_m$  is generated by the defender-specified image generator  $G_x: \mathcal{X} \to \mathcal{X}$  and the label generator  $G_y: \mathcal{Y} \to \mathcal{Y}$ , i.e.,  $\mathcal{D}_m = \{(G_x(x), G_y(y)) | (x, y) \in \mathcal{D}_s\}$ . For example,  $G_x = (1 - m) \odot t + m \odot x$  and  $G_y = y_t$  in BadNets [17], where  $m \in \{0,1\}^{C \times W \times H}$  is the trigger mask,  $t \in [0,1]^{C \times W \times H}$  is the trigger pattern,  $\odot$  denotes the element-wise product, and  $y_t$  is the target label. In particular,  $\gamma \triangleq \frac{|D_m|}{|D_w|}$  is called the watermarking rate. All models trained on the protected dataset  $\mathcal{D}_w$  will have special prediction behaviors on  $G_x(x)$  for ownership verification. Specifically, let  $C: \mathcal{X} \to \mathcal{Y}$  denotes a third-party suspicious model that could be trained on the protected dataset, existing backdoor-based methods will examine whether it conduct unauthorized training by testing whether  $C(G_x(x)) = y_t$ . Since  $y_t \neq y$  in most cases, these backdoor-based watermarks are harmful.
As described in previous sections, existing backdoor-inspired dataset ownership verification (DOV) methods [3, 4, 5] would cause malicious misclassification on watermarked samples to all models trained on the protected dataset, therefore they are harmful. This limitation of backdoor-based DOV methods cannot be eliminated because their inherent mechanism is to lead the watermarked model to have particular mispredicted behaviors for verification, although the misclassification could be random and less harmful [18]. In this paper, we intend to design a truly harmless DOV method so that the watermarked models will correctly classify watermarked samples. Before we formally define the studied problem, we first provide the definition of harmful degree of a DOV method.
Definition 1 (Harmful and Relatively Harmful Degree). Let  $\hat{\mathcal{D}} = \{(\hat{x}_i, y_i)\}_{i=1}^N$  indicates a set of watermarked samples used for ownership verification of a DOV method, where  $\hat{x}_i$  is the verification sample with  $y_i \in \mathcal{Y}$  as its ground-truth label (instead of its given label). Let  $\hat{C}$  and  $C$  represent a classifier trained on the protected and unprotected datasets, respectively. The harmful degree is  $H \triangleq \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}\{\hat{C}(\hat{x}_i) \neq y_i\}$  and the relatively harmful degree is  $\hat{H} \triangleq \frac{1}{N} \left( \sum_{i=1}^{N} \mathbb{I}\{\hat{C}(\hat{x}_i) \neq y_i\} - \sum_{i=1}^{N} \mathbb{I}\{C(\hat{x}_i) \neq y_i\} \right)$  where  $\mathbb{I}\{\cdot\}$  is the indicator function.
To design a harmless DOV method, we intend to make watermarked DNNs correctly classify some 'hard' samples that will be misclassified by the model trained on the unprotected benign dataset. Inspired by the generalization property of DNNs, we intend to find a hardly-generalized domain of the benign dataset, which can be easily learned with the protected dataset containing the modified samples. In this paper, we call this watermarking method as domain watermark, defined as follows.
Definition 2 (Domain Watermark). Given a benign dataset  $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^N$ , let  $C: \mathcal{X} \to \mathcal{Y}$  denotes a model trained on  $\mathcal{D}$ . Assume that  $G_d$  denotes a domain generator such that  $G_d(x_i)$  owns the same ground-truth label as  $x_i$  but belongs to a hardly-generalized domain, i.e.,  $\sum_{(x_i, y_i) \in \mathcal{D}} \mathbb{I}\{C(x_i) = y_i\} \gg \sum_{(x_i, y_i) \in \mathcal{D}} \mathbb{I}\{C(G_d(x_i)) = y_i\}$ . We intend to find a watermarked
version of  $\mathcal{D}$  (i.e.,  $\mathcal{D}_d$ ) with watermarking rate  $\gamma$ , such that the watermarked model  $\hat{C}$  trained on it have two properties: (1)  $\frac{1}{N} \sum_{(\boldsymbol{x}_i, y_i) \in \mathcal{D}} \mathbb{I}\{\hat{C}(\boldsymbol{x}_i) = y_i\} \geq \beta$  and (2)  $\frac{1}{N} \sum_{(\boldsymbol{x}_i, y_i) \in \mathcal{D}} (\mathbb{I}\{\hat{C}(\boldsymbol{x}_i) = y_i\} - \mathbb{I}\{\hat{C}(G_d(\boldsymbol{x}_i)) = y_i\}) \leq \tau$ , where  $\beta, \tau \in [0, 1]$  are given parameters. In this paper,  $\mathcal{D}_d$  is defined as the domain watermark of the benign dataset  $\mathcal{D}$ .
3.3 Generating the Hardly-Generalized Domain
As illustrated in Definition 2, finding a hardly-generalized target domain  $\mathcal{T}$  (with domain generator  $G_{d}$ ) of the source domain  $\mathcal{S}$  is the first step of our domain watermark. To guide the construction of the domain  $\mathcal{T}$ , we have the following Lemma 1.
Lemma 1 (Generalization Bound [38]). The bound of expected risk on a given target domain  $\mathcal{T}$  is negatively associated with mutual information between features for source  $S$  and target  $\mathcal{T}$  domains:
$$
\mathcal {R} _ {\mathcal {T}} (f) \leq \mathcal {R} _ {\mathcal {S}} (f) - 4 I (\boldsymbol {z}; \hat {\boldsymbol {z}}) + 4 H (Y) + \frac {1}{2} d _ {\mathcal {H} \triangle \mathcal {H}} (p (\boldsymbol {z}), p (\hat {\boldsymbol {z}})), \tag {1}
$$
where  $\mathcal{R}_{\mathcal{T}}(f) = \mathbb{E}_{(\hat{x},y)\sim \mathcal{T}}[\mathbb{I}\{C(\hat{x})\neq y\} ],\mathcal{R}_{\mathcal{S}}(f) = \mathbb{E}_{(\boldsymbol {x},y)\sim \mathcal{S}}[\mathbb{I}\{C(\boldsymbol {x})\neq y\} ].$ $I(z;\hat{z})$  is mutual information between features from  $\mathcal{S}$  and  $\mathcal{T}$ .  $d_{\mathcal{H}\triangle \mathcal{H}}(p(\boldsymbol {z}),p(\hat{\boldsymbol{z}}))$  is  $\mathcal{H}\triangle \mathcal{H}$ -divergence for measuring the divergence of feature marginal distributions of two domains, and  $H(\cdot)$  is the entropy.
Lemma 1 reveals the upper bound of generalization performance on  $\mathcal{T}$ . Since  $d_{\mathcal{H}\triangle \mathcal{H}}(\cdot)$  is intractable and hard to directly optimize, as well as [38] shows that only a  $I(\cdot)$  is enough for generalization across domains, we propose to increase the expected risk on  $\mathcal{T}$  by minimizing  $I(z;\hat{z})$ .
Specifically, we formulate the design of the target domain  $\mathcal{T}$  (with the domain generator  $G_{d}(:,\pmb{\theta})$ ) as a bi-level optimization, as follows:
$$
\min  _ {\boldsymbol {\theta}} \mathbb {E} _ {p (\boldsymbol {z}, \hat {\boldsymbol {z}})} \left[ I \left(\boldsymbol {z} \left(\boldsymbol {w} ^ {*}\right); \hat {\boldsymbol {z}} \left(\boldsymbol {\theta}, \boldsymbol {w} ^ {*}\right)\right) + \lambda_ {1} \mathcal {L} _ {c} \left(\boldsymbol {z} \left(\boldsymbol {w} ^ {*}\right), \hat {\boldsymbol {z}} \left(\boldsymbol {\theta}, \boldsymbol {w} ^ {*}\right)\right) \right], \tag {2}
$$
$$
s. t. \boldsymbol {w} ^ {*} = \arg \min  _ {\boldsymbol {w}} \left[ \mathbb {E} _ {(\boldsymbol {x}, y) \sim \mathcal {D}} \left[ \mathcal {L} (f (G _ {d} (\boldsymbol {x}; \boldsymbol {\theta}); \boldsymbol {w}), y) + \mathcal {L} (f (\boldsymbol {x}; \boldsymbol {w}), y) \right] - \lambda_ {2} \mathbb {E} _ {p (\boldsymbol {z}, \hat {\boldsymbol {z}})} [ I (\boldsymbol {z} (\boldsymbol {w}); \hat {\boldsymbol {z}} (\boldsymbol {w})) ] \right.,
$$
where  $\lambda_1, \lambda_2$  are two positive hyper-parameters, and  $\mathcal{L}(\cdot)$  is the loss function (e.g., cross entropy).
Following previous works [38] in domain adaption and generalization, we propose to optimize the upper bound approximation for  $I(z; \hat{z})$  instead of itself and leverage a transformation module consisting of multiple convolutional operations as  $G_{d}(\cdot; \theta)$  to generate the domain-watermarked image  $\hat{x}$ . Specifically, we aim to craft  $\hat{x}$  via minimizing the upper bound approximation for mutual information [39] between  $x \in \mathcal{D}$  and  $\hat{x}$  in the latent feature space  $\mathbb{Z}$ :
$$
I (\boldsymbol {z}; \hat {\boldsymbol {z}}) = \mathbb {E} _ {p (\boldsymbol {z}, \hat {\boldsymbol {z}})} \left[ \log \frac {p (\hat {\boldsymbol {z}} | \boldsymbol {z})}{p (\hat {\boldsymbol {z}})} \right] \leq \mathbb {E} _ {p (\boldsymbol {z}, \hat {\boldsymbol {z}})} [ \log p (\hat {\boldsymbol {z}} | \boldsymbol {z}) ] - \mathbb {E} _ {p (\boldsymbol {z}) p (\hat {\boldsymbol {z}})} [ \log p (\hat {\boldsymbol {z}} | \boldsymbol {z}) ], \tag {3}
$$
where  $z$  and  $\hat{z}$  are the latent vectors obtained by passing  $x$  and  $\hat{x}$  through  $f(\cdot; w)$ 's feature extractor.
$\mathcal{L}_c(\cdot)$  is the class-conditional maximum mean discrepancy (MMD) computed on the latent space  $\mathbb{Z}$  and proposed to limit the potential semantic information distortion between  $\pmb{x}$  and  $\hat{\pmb{x}}$ , follows:
$$
\mathcal {L} _ {c} (z, \hat {z}) = \frac {1}{K} \sum_ {j = 1} ^ {K} \left(\left| \left| \frac {1}{n _ {s} ^ {j}} \sum_ {i = 1} ^ {n _ {s} ^ {j}} \phi (\boldsymbol {z} _ {\boldsymbol {i}} ^ {\boldsymbol {j}}) - \frac {1}{n _ {t} ^ {j}} \sum_ {i = 1} ^ {n _ {t} ^ {j}} \phi (\hat {\boldsymbol {z}} _ {\boldsymbol {i}} ^ {\boldsymbol {j}}) \right| \right| ^ {2}\right), \tag {4}
$$
where  $n_s^j$ ,  $n_t^j$  represent the number for  $\pmb{x}$  and  $\hat{\pmb{x}}$  from class  $j$ , and  $\phi(\cdot)$  is the kernel function.
The configurations, parameter selections, and model architectures are included in Appendix A.
3.4 Generating the Protected Dataset
Once we obtain the hard-generalized domain generator  $G_{d}$  with the method proposed in Section 3.3, the next step is to generate the protected dataset based on it. Before we present its technical details, we first deliver some insight into the data quantity impact for the domain watermark.
Theorem 1 (Data Quantity Impact). Suppose in PAC Bayesian [40], for a target domain  $\mathcal{T}$  and a source domain  $S$ , any set of voters (candidate models)  $\mathcal{H}$ , any prior  $\pi$  over  $\mathcal{H}$  before any training, any  $\xi \in (0,1]$ , any  $c > 0$ , with a probability at least  $1 - \xi$  over the choices of  $S \sim S^{n_s}$  and  $T \sim T_{\mathcal{X}}^{n_t}$ , for the posterior  $f$  over  $\mathcal{H}$  after the joint training on  $S$  and  $T$ , we have