Robust Unlearnable Examples: Protecting Data Privacy Against Adversarial Learning

1 INTRODUCTION . Recent advances in deep learning largely rely on various large-scale datasets , which are mainly built upon public data collected from various online resources such as Flickr , Google Street View , and different search engines ( Lin et al. , 2014 ; Torralba et al. , 2008 ; Netzer et al. , 2011 ) . However , the process of data collection might be unauthorized , leading to the risk of privacy leakage and personal data misuse . For example , recent work demonstrates that individual information such as name or email address could be leaked from a pre-trained GPT-2 model ( Carlini et al. , 2020 ) . Another investigation reveals that a company has been continuously collecting facial images from the internet for training commercial facial recognition models ( Hill , 2020 ) . To protect the privacy and prevent the unauthorized use of the released data , some studies suggest poisoning the clean data with imperceptible noise so that the performances of models trained on the modified data be significantly downgraded ( Fowl et al. , 2021b ; a ; Yuan & Wu , 2021 ; Huang et al. , 2021 ) . Our work mainly focuses on the work by Huang et al . ( 2021 ) . Specifically , Huang et al . ( 2021 ) introduce unlearnable examples that are crafted from clean data via adding a type of imperceptible error-minimizing noise and show impressive data-protection ability . The error-minimizing noise is designed based on the intuition that an example of higher training loss may contain more knowledge to be learned . Thereby , the noise protects data from being learned via minimizing the corresponding loss to suppress the informative knowledge of data as much as possible . However , such an unlearnability conferred by error-minimizing noise is found vulnerable to adversarial training ( Huang et al. , 2021 ; Tao et al. , 2021 ) , a standard approach towards training robust deep learning models ( Madry et al. , 2018 ) . In adversarial training , a model trained on the unlearnable data can achieve the same performance as that trained on the clean data . Similar problems are also found in other poisoning-based data protection methods , e.g. , the targeted adversarial poisoning method ( Fowl et al. , 2021b ) ( see Fig . 1 ) . These findings bring great challenges to poisoning-based data privacy protection , and further raises the following question : Can we still make data unlearnable by adding imperceptible noise even under the adversarial training ? In this paper , we give an affirmative answer that imperceptible noise can indeed stop deep learning models from learning high-level knowledge in an adversarial manner . We first analyze the reason why the unlearnability conferred by error-minimizing noise fails under adversarial training . Adversarial training improves the robustness of models via training them on adversarial examples . However , adversarial examples usually correspond to higher loss , which according to Huang et al . ( 2021 ) may contain more knowledge to be learned . Though the error-minimizing noise is designed to minimize the training loss , adversarial examples crafted from unlearnable data could still significantly increase the loss scale ( see Fig . 2 ) . Therefore , adversarial training can break the protection brought by error-minimizing noise and still learn knowledge from these data . Based on the aforementioned analysis , we further deduce that the invalidation of unlearnable examples in adversarial training might be attributed to the error-minimizing noise could not effectively reduce the adversarial training loss . To this end , a new type of noise , named robust error-minimizing noise , is designed to protect data privacy from adversarial learners via minimizing the adversarial training loss . Inspired by the strategy of adversarial training and the generation of error-minimizing noise , a min-min-max optimization is designed to train the robust error-minimizing noise generator . Examples crafted by adding robust error-minimizing noise are called robust unlearnable examples . In summary , our work has three main contributions : ( 1 ) We present robust error-minimizing noise , which is the first and currently the only method that can prevent data from being learned in adversarial training to the best of our knowledge . ( 2 ) We propose to solve a min-min-max optimization problem to effectively generate robust error-minimizing noise . ( 3 ) We empirically verify the effectiveness of robust error-minimizing noise under adversarial training . 2 RELATED WORKS . Adversarial attacks . Adversarial examples are carefully crafted modifications of normal examples that can fool machine learning models to behave in unexpected ways . It has been found that adversarial examples widely exist and machine learning models are vulnerable to adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ) . Though minor data transformations can usually eliminate the adversaries of adversarial examples ( Lu et al. , 2017 ; Luo et al. , 2015 ) , many works have demonstrated that adversarial examples can further be made resistant against these transformations ( Evtimov et al. , 2017 ; Eykholt et al. , 2018 ; Wu et al. , 2020b ; Athalye et al. , 2018 ; Liu et al. , 2019 ; 2020a ) . This makes adversarial attacks realistic threatens . To tackle adversarial attacks , adversarial training is proposed to improve the robustness of deep learning models against adversarial examples . A standard adversarial training algorithm aims to solve a minimax problem , which minimizes the loss function while the most adversarial examples present ( Madry et al. , 2018 ) . Other advances include TRADE ( Zhang et al. , 2019 ) , FAT ( Zhang et al. , 2020 ) , GAIRAT ( Zhang et al. , 2021 ) , RLFAT ( Song et al. , 2020 ) , Jiang et al . ( 2021 ) , Stutz et al . ( 2020 ) , Singla & Feizi ( 2020 ) , Croce & Hein ( 2020 ) , Wu et al . ( 2020a ) , and Wu et al . ( 2021 ) . Tang et al . ( 2021 ) propose an adversarial robustness benchmark regarding architecture design and training techniques . Besides , there are also some works that attempt to interpret how machine learning models gain robustness through adversarial training ( Ilyas et al. , 2019 ; Zhang & Zhu , 2019 ) . Poisoning attacks . Poisoning attacks aim to manipulate the performance of a machine learning model via injecting malicious poisoned examples into its training data set ( Biggio et al. , 2012 ; Koh & Liang , 2017 ; Shafahi et al. , 2018 ; Liu et al. , 2020b ; Jagielski et al. , 2018 ; Yang et al. , 2017 ; Steinhardt et al. , 2017 ) . The generated poisoned examples usually appear different from the clear examples ( Biggio et al. , 2012 ; Yang et al. , 2017 ; Steinhardt et al. , 2017 ) . However , recent approaches show that poisoned examples can also be crafted imperceptible to their corresponding origins ( Koh & Liang , 2017 ; Shafahi et al. , 2018 ) . A special case of poisoning attacks is the backdoor attack . A model trained on backdoored data may perform well on normal data but wrongly behave on data that contain trigger patterns ( Chen et al. , 2017 ; Nguyen & Tran , 2020 ; 2021 ; Li et al. , 2021 ; Weng et al. , 2020 ) . Comparing to other poisoning attacks , backdoor attack is more covert and thus can be used to implement more harmful attacks , such as supply chain attacks ( Gu et al. , 2017 ) . Recent works start to employ data poisoning to protect data privacy and prevent unauthorized model training . Huang et al . ( 2021 ) propose to modify data with imperceptible error-minimizing noise to stop deep learning model learning knowledge from the modified data . Yuan & Wu ( 2021 ) adopt a Gaussian Process that is designed based on Neural Tangent Kernels to model the mean of a class of wide DNNs , and generate protective noise upon it . This method can enhance the transferability of the generated noise across different models . Fowl et al . ( 2021a ) and Fowl et al . ( 2021b ) employ gradient alignment ( Geiping et al. , 2020 ; 2021 ) and PGD method ( Madry et al. , 2018 ) to generate adversarial examples respectively , and then use the generated data as poisoned data to downgrade the performance of models trained on them . However , Tao et al . ( 2021 ) find that existing poisoningbased protection can be easily broken by adversarial training . 3 PRELIMINARIES . Suppose D = { ( x1 , y1 ) , · · · , ( xn , yn ) } is a dataset consists of n samples , where xi ∈ X is the feature of the i-th sample and yi ∈ Y is the corresponding label . A parameterized machine learning model is fθ : X → Y , where θ ∈ Θ is the model parameter . Suppose ` : Y × Y → [ 0 , 1 ] is a loss function . Then , the empirical risk minimization ( ERM ) aims to approach the optimal model by solving the following optimization problem , min θ 1 n n∑ i=1 ` ( fθ ( xi ) , yi ) . ( 1 ) Adversarial training ( Madry et al. , 2018 ) improves the robustness of models against adversarial attacks via training them on adversarial examples . A standard adversarial training aims to solve the following min-max problem , min θ 1 n n∑ i=1 max ‖δi‖≤ρa ` ( fθ ( xi + δi ) , yi ) , ( 2 ) where ρa > 0 is the adversarial perturbation radius and ( xi + δi ) is the most adversarial example within the ball sphere centered at xi with radius ρa . Usually , a larger radius ρa implies stronger adversarial robustness of the trained model . Unlearnable examples ( Huang et al. , 2021 ) is a type of data that deep learning models could not effectively learn informative knowledge from them . Models trained on unlearnable examples could only achieve severely low performance . The generation of unlearnable examples contains two steps . Firstly , train an error-minimizing noise generator f ′θ via solving the following optimization problem , min θ 1 n n∑ i=1 min ‖δi‖≤ρu ` ( f ′θ ( xi + δi ) , yi ) , ( 3 ) where ρu is the defensive perturbation radius that forces the generated error-minimizing noise to be imperceptible . Then , an unlearnable example ( x′ , y ) is crafted via adding error-minimizing noise generated by the trained noise generator f ′ ( θ ) to its clean counterpart ( x , y ) , where x′ = x + arg min‖δ‖≤ρu ` ( f ′ θ ( x + δ ) , y ) . The intuition behind unlearnable examples is , a smaller loss may imply less knowledge that could be learned from an example . Thus , error-minimizing noise aims to make data unlearnable via reducing the corresponding training loss . It is worth noting that once the unlearnable data is released publicly , the data defender could not modify the data any further . Projected gradient descent ( PGD ) ( Madry et al. , 2018 ) is a standard approach for solving the inner maximization and minimization problems in Eqs . ( 2 ) and ( 3 ) . It performs iterative projection updates to search for the optimal perturbation as follows , δ ( k ) = ∏ ‖δ‖≤ρ [ δ ( k−1 ) + c · α · sign ( ∂ ∂δ l ( fθ ( x+ δ ( k−1 ) ) , y ) ) ] , where k is the current iteration step ( K steps at all ) , δ ( k ) is the perturbation found in the k-th iteration , c ∈ { −1 , 1 } is a factor for controlling the gradient direction , α is the step size , and ∏ ‖δ‖≤ρ means the projection is calculated in the ball sphere { δ : ‖δ‖ ≤ ρ } . The final output perturbation is δ ( K ) . Throughout this paper , the coefficient c is set as 1 when solving maximization problems and −1 when solving minimization problems .

Summary: The paper proposes a min-min-max formulation to generate robust unlearnable examples in order to protect data privacy in adversarial learning. The basic idea of this paper builds upon a previous paper "Unlearnable examples: Making personal data unexploitable" where the error-minimization noise is generated to reduce the training loss such that the model performs badly on the clean data. This paper finds that such unlearnability is fragile in adversarial training and fragile to minor data augmentations. Therefore, it proposes to generate more robust unlearnable samples by considering adversarial loss and data augmentation in the noise generation process, and it shows the generated noise can effectively reduce the quality of the trained model through extensive empirical study.