Unlearnable Examples: Making Personal Data Unexploitable

1 INTRODUCTION . In recent years , deep learning has had groundbreaking successes in several fields , such as computer vision ( He et al. , 2016 ) and natural language processing ( Devlin et al. , 2018 ) . This is partly attributed to the availability of large-scale datasets crawled freely from the Internet such as ImageNet ( Russakovsky et al. , 2015 ) and ReCoRD ( Zhang et al. , 2018b ) . Whilst these datasets provide a playground for developing deep learning models , a concerning fact is that some datasets were collected without mutual consent ( Prabhu & Birhane , 2020 ) . Personal data has also been unconsciously collected from the Internet and used for training commercial models ( Hill , 2020 ) . This has raised public concerns about the “ free ” exploration of personal data for unauthorized or even illegal purposes . In this paper , we address this concern by introducing unlearnable examples , which aims at making training examples unusable for Deep Neural Networks ( DNNs ) . In other words , DNNs trained on unlearnable examples will have a performance equivalent to random guessing on normal test examples . Compared with preserving an individual ’ s privacy by obfuscating information from the dataset , what we aim to achieve here is different but more challenging . First , making an example unlearnable should not affect its quality for normal usage . For instance , an unlearnable “ selfie ” photo should be free from obvious visual defects so it can be used as a social profile picture . Ideally , this can be achieved by using imperceptible noise . In our setting , the noise can only be added to training examples on a single occasion ( when the data is uploaded to the internet ) prior to model training . However , DNNs are known to be robust to small noise either random ( Fawzi et al. , 2016 ) or adversarial ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Ma et al. , 2018 ) . It is still not clear whether small , imperceptible noise can stop the training of high-performance DNNs . The development of unlearnable examples should take full advantage of the unique characteristics , and more importantly , the weaknesses of DNNs . One well-studied characteristic of DNNs is that they tend to capture more of the high-frequency components of the data ( Wang et al. , 2020a ) . Surprisingly , by exploiting this characteristic , we find that small random noise when applied in a class-wise †Correspondence to : Xingjun Ma ( daniel.ma @ deakin.edu.au ) , Yisen Wang ( yisen.wang @ pku.edu.cn ) manner to the training data can easily fool DNNs to overfit to such noise ( shown in Section 4 ) . However , early stopping can effectively counteract this type of noise . DNNs are also known to be vulnerable to adversarial ( or error-maximizing ) noise , which are small perturbations crafted to maximize the model ’ s error at the test time ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ) . We find that error-maximizing noise can not stop DNN learning when applied in a sample-wise manner to the training examples . This motivates us to explore the opposite direction to error-maximizing noise . Specifically , we propose a type of error-minimizing noise that can prevent the model from being penalized by the objective function during training , and thus can trick the model into believing there is “ nothing ” to learn from the example ( s ) . We refer to an example that contains the errorminimizing noise as an unlearnable example . Error-minimizing noise can be generated in different forms : sample-wise and class-wise . Class-wise error-minimizing noise is superior to random noise and can not be circumvented by early stopping . Sample-wise error-minimizing noise is the only effective noise that can make training examples unlearnable compared to random ( Fawzi et al. , 2016 ) or error-maximizing noise ( Muñoz-González et al. , 2017 ) . Our main contributions are : • We present a type of error-minimizing noise that can create unlearnable examples to prevent personal data from being freely exploited by deep learning models . The noise is small , imperceptible to human eyes , thus it does not reduce general data utility . • We propose a bi-level optimization process to effectively generate different forms of error- minimizing noise : sample-wise and class-wise . • We empirically verify the effectiveness and flexibility of error-minimizing noise for cre- ating unlearnable examples . We also demonstrate the practical application of unlearnable examples in real-world scenarios via a case study on face recognition . 2 RELATED WORK . In this section , we briefly review most relevant works in data privacy , data poisoning , adversarial attacks against deep learning models . Data Privacy . Privacy issues have been extensively studied in the field of privacy-preserving machine learning ( Shokri & Shmatikov , 2015 ; Abadi et al. , 2016 ; Phan et al. , 2016 ; 2017 ; Shokri et al. , 2017 ) . While these works have made significant progress towards protecting data privacy , they are developed based on the assumption that the model can freely explore the training data and turn to protect the model from leaking sensitive information about the training data . In this paper , we consider a more challenging scenario where the goal of the defender is to make personal data completely unusable by unauthorized deep learning models . Fawkes ( Shan et al. , 2020 ) has made the first attempt towards this type of strict situation . By leveraging the targeted adversarial attack , Fawkes prevents unauthorized face tracker from tracking a person ’ s identity . This work is similar to ours as we share a common objective that prevents unauthorized data usage . In contrast to the targeted adversarial attack , we propose a novel error-minimizing noise to produce unlearnable examples which can be used as a generic framework for a wide range of data protection tasks . Data Poisoning . Data poisoning attacks aim to degrade the model ’ s performance on clean examples by modifying the training examples . Previous work has demonstrated a poisoning attack on SVM ( Biggio et al. , 2012 ) . Koh & Liang ( 2017 ) proposed to poison the most influential training examples using adversarial ( error-maximizing ) noise against DNNs , which has also been integrated into an endto-end framework ( Muñoz-González et al. , 2017 ) . Although data poisoning attacks can potentially prevent free data exploitation , these approaches are quite limited against DNNs and hard to operate in real-world scenarios . For example , poisoned examples can only slightly decrease DNNs ’ performance ( Muñoz-González et al. , 2017 ) , and often appear distinguishable to clean examples ( Yang et al. , 2017 ) which will reduce normal data utility . The backdoor attack is another type of attack that poisons training data with a stealthy trigger pattern ( Chen et al. , 2017 ; Liu et al. , 2020 ) . However , the backdoor attack does not harm the model ’ s performance on clean data ( Chen et al. , 2017 ; Shafahi et al. , 2018 ; Barni et al. , 2019 ; Liu et al. , 2020 ; Zhao et al. , 2020 ) . Thus , it is not a valid method for data protection . Different from these works , we generate unlearnable examples with invisible noise to “ bypass ” the training of DNNs . Adversarial Attack . It has been found that adversarial examples ( or attacks ) can fool DNNs at the test time ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Kurakin et al. , 2016 ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Jiang et al. , 2019 ; Wu et al. , 2020a ; Bai et al. , 2020 ; Croce & Hein , 2020 ; Wang et al. , 2020b ; Duan et al. , 2020 ; Ma et al. , 2020 ) . The adversary finds an error-maximizing noise that maximizes the model ’ s prediction error , and the noise can be crafted universally for the entire test set ( Moosavi-Dezfooli et al. , 2017 ) . Adversarial training has been shown to be the most robust training strategy against error-maximizing noise ( Madry et al. , 2018 ; Zhang et al. , 2019 ; Wang et al. , 2019 ; Wu et al. , 2020b ; Wang et al. , 2020c ) . Adversarial training can be formulated as a min-max optimization problem . In this paper , we explore the opposite direction of error-maximizing noise , i.e. , finding small noise that minimizes the model ’ s error via a min-min optimization process . 3 UNLEARNABLE EXAMPLES AND ERROR-MINIMIZING NOISE . 3.1 PROBLEM STATEMENT . Assumptions on Defender ’ s Capability . We assume the defender has full access to the portion of data which they want to make unlearnable . However , the defender can not interfere with the training process and does not have access to the full training dataset . In other words , the defender can only transform their portion of data into unlearnable examples . Moreover , the defender can not further modify their data once the unlearnable examples are created . Objectives . We formulate the problem in the context of image classification with DNNs . Given a typical K-class classification task , we denote the clean training and test datasets as Dc and Dt respectively , and the classification DNN trained on Dc as fθ where θ are the parameters of the network∗ . Our goal is to transform the training data Dc into unlearnable dataset Du such that DNNs trained on the Du will perform poorly on the test set Dt . Suppose the clean training dataset consists of n clean examples , that is , Dc = { ( xi , yi ) } ni=1 with x ∈ X ⊂ Rd are the inputs and y ∈ Y = { 1 , · · · , K } are the labels and K is the total number of classes . We denote its unlearnable version by Du = { ( x′i , yi ) } ni=1 , where x′ = x + δ is the unlearnable version of training example x ∈ Dc and δ ∈ ∆ ⊂ Rd is the “ invisible ” noise that makes x unlearnable . The noise δ is bounded by ‖δ‖p ≤ with ‖ · ‖p is the Lp norm , and is set to be small such that it does not affect the normal utility of the example . In the typical case , the DNN model will be trained onDc to learn the mapping from the input space to the label space : f : X → Y . Our goal is to trick the model into learning a strong correlation between the noise and the labels : f : ∆→ Y , ∆ 6= X , when trained on Du : arg min θ E ( x′ , y ) ∼DuL ( f ( x ′ ) , y ) ( 1 ) where , L is the classification loss such as the commonly used cross entropy loss . Noise Form . We propose two forms of noise : sample-wise and class-wise . For sample-wise noise , x′i = xi + δi , δi ∈ ∆s = { δ1 , · · · , δn } , while for class-wise noise , x′i = xi + δyi , δyi ∈ ∆c = { δ1 , · · · , δK } . Sample-wise noise needs to generate noise separately for each example . This may have more limited practicality . In contrast to sample-wise noise , a class of examples can be made unlearnable by the addition of class-wise noise , where all examples in the same class have the same noise added . As such , class-wise noise can be generated more efficiently and more flexibly in practical usage . However , we will see that class-wise noise may get more easily exposed .

Summary: The paper's motivation is based on protecting private data and preventing its being scraped and used to train models. Even though motivation is clear and very important, the problem is the same as the works in crafting adversarial samples (i.e., the ones under data poisoning and adversarial attacks parts of the related work). The key difference is to apply Projected Gradient Descent (Mandry et al. 2018) in the reverse direction iteratively to *minimize* the loss function.  Furthermore, the performance evaluation will be the margin between models trained on completely clean data and sample-wise/class-wise adversarially corrupted data (in contrast to fooling a pretrained network in adversarial attack benchmarks).