ConfounderGAN: Protecting Image Data Privacy with Causal Confounder
Anonymous Author(s)
AfﬁliationAddress email
Abstract
The success of deep learning is partly attributed to the availability of massivedata downloaded freely from the Internet.
However, it also means that users’private data may be collected by commercial organizations without consent andused to train their models.
Therefore, it’s important and necessary to develop amethod or tool to prevent unauthorized data exploitation.
In this paper, we propose
ConfounderGAN , a generative adversarial network (GAN) that can make personalimage data unlearnable to protect the data privacy of its owners.
Speciﬁcally,the noise produced by the generator for each image has the confounder property.
It can build spurious correlations between images and labels, so that the modelcannot learn the correct mapping from images to labels in this noise-added dataset.
Meanwhile, the discriminator is used to ensure that the generated noise is smalland imperceptible, thereby, remaining the normal utility of the encrypted imagefor humans.
The experiments are conducted in six image classiﬁcation datasets,consisting of three natural object datasets and three medical datasets.
The resultsdemonstrate that our method not only outperforms state-of-the-art methods instandard settings, but can also be applied to fast encryption scenarios.
Moreover,we show a series of transferability and stability experiments to further illustrate theeffectiveness and superiority of our method.
Introduction
In recent years, deep learning has achieved great success in many ﬁelds, such as computer vision[10], natural language processing [6] etc .
An important factor is that large-scale datasets providerich and diverse training examples for deep neural networks.
However, many datasets are collectedfrom some free channels without mutual consent and this unauthorized collection of private datamay violate the rights of the data owner [3].
For example, Kashmir Hill from the New York Timesrecently reported that a company, named Clearview.
AI, used web crawler scripts to collect more than3 billion online photos from various multimedia websites ( i,e. , Facebook, YouTube, Venmo, andmillions of other websites) for training its own commercial models [34].
This large unauthorizeddataset goes far beyond anything ever constructed by the United States government or Silicon Valleygiants.
In addition, Google’s Nightingale Project has also been reported to collect medical careinformation from tens of millions of patients without their knowledge, and use advanced machinelearning technology to customize medical services for individual patients [22, 36].
In this context, various information privacy laws around the world have been published to describethe rights of natural persons to control who is using its data [40].
e.g. , General Data Protection
Regulation (GDPR) in the European Union [38], Electronic Communications Privacy Act (ECPA) inthe U.S. [39, 37], etc .
However, in addition to legal constraints, it’s necessary to develop a protectionmethod to actively prevent unauthorized data from being used by third parties to train commercialmodels.
In this paper, we aim to develop an image encryption method.
i.e. , the deep classiﬁcation [Figure] models cannot extract any exploitable knowledge from encrypted (or processed) images.
Moreover, we hope the modiﬁcation of the original data is strictly limited, thereby retaining thedata quality for normal usage.
e.g., an unlearnable photo should be free from obvious visualdefects so it can be shared with friends on socialmedia.
Unfortunately, the existing works are verysparse and have application limitations.
Shanet al.
[29] ﬁrst propose an error-maximizingnoise based on metric learning to make data unexploitable, but this method is mainly customized for face recognition systems.
Then Huang et al.[12] propose a bi-level optimization based on error-minimizing noise (EMN) to meet the above goal,and it can be extended to various image datasets.
However, these methods need multiple gradientbackpropagations to generate an effective noise for each labeled image, which means they cannotencrypt an unlabeled image on the ﬂy.
We emphasize that this setting is common and important forreal-world applications.
For example, a person takes a photo with a cell phone.
Then he/she wants toquickly encrypt the photo and share it on social media immediately.
To solve the above challenge, in this paper, we propose ConfounderGAN , a novel image dataencryption method that covers various practical scenarios.
Figure 1 demonstrates the core idea of ourmethod.
We suppose that the model can learn a correct causal graph from the image X to the label
Yin the original dataset as illustrated in Figure 1(a).
To prevent unauthorized models from extractingground truth relationship X
→ Y from the private dataset, we deliberately introduce a confoundernoise N that satisﬁes the relationship X ← N → Y in the dataset as shown in Figure 1(b).
Thisconfounder noise is correlated with both the image X and the label Y , creating a spurious correlationbetween X and Y .
Therefore, models trained on this data would ﬁt the negatively spurious correlationand get low test accuracy.
There are many possible ways that can achieve this confounder-basedencryption framework.
We instantiate this framework by generative adversarial networks (GAN)
[9],as this implementation is more efﬁcient and practical.
Speciﬁcally, we hope that the generator canproduce confounder noise that satisﬁes the above causal graph for each private image.
The mainchallenge is how to ensure that the noise is strongly associated with both the image and the label.
Since the noise is produced by the generator with the original image as input, the former is naturallysatisﬁed.
For the latter, we directly classify the noise as the ground truth class of the original imageduring generator training.
Meanwhile, we send the original image and its noise-added copy to thediscriminator, thus ensuring the invisibility of noise.
Once the generator is trained, it can quicklyproduce confounder noise for any image to protect its privacy, including those unlabeled ones.
Ourmain contributions are:
• We present a confounder-based framework for image data encryption.
The confounder inthis framework can create spurious correlations between images and labels, thus preventing
unauthorized models from learning exploitable knowledge on the encrypted data.
• By leveraging the property of generative adversarial networks, we implement the aboveframework as ConfounderGAN, where the generator is used to produce confounder noise
to encrypt the target image and the discriminator ensures that the noise is imperceptible tohuman eyes.• Extensive experiments on six image classiﬁcation datasets show that our proposed methodnot only outperforms state-of-the-art methods in standard settings, but can also be applied tofast encryption scenarios.
Also, we conduct a set of transferability and stability experiments
to highlight the effectiveness and superiority of our method.
Related work
In this section, we brieﬂy review some literature that is closely related to our work: data privacy, datapoisoning, causal confounder in deep learning.
Data privacy.
Privacy-preserving techniques have been widely explored in the machine learningcommunity and a lot of excellent works have been proposed to protect data privacy [31, 25, 26, 32].
However, these methods are mainly to prevent malicious agencies from de-inferring training setinformation in a trained model.
In this paper, we focus on a more challenging scenario.
i.e. , makingprivate data completely unlearnable by unauthorized deep neural networks.
Shan et al.
[29] ﬁrstpropose a solution for the face recognition system.
By utilizing the targeted adversarial attack togenerate error-maximizing noise, this method can close the representation of different identities sothat the model trained on this dataset can only obtain poor performance.
Then Huang et al.
[12]propose a bi-level optimization based on error-minimizing noise (EMN) to meet the same goal, andit is a general method that can be used to various image datasets.
However, these gradient-basedmethods need multiple optimization steps for encrypting one labeled image, and thus cannot quicklyencrypt an unlabeled one.
Instead, our ConfounderGAN does not have this limitation.
Because oncethe confounder noise generator is trained, any private image can be processed immediately.
Data poisoning.
The goal of traditional data poisoning is to reduce the test accuracy of the modelby modifying the training set.
Biggio et al. [2] ﬁrst introduce this type of attack in support vectormachines [4]. Then Muñoz-González et al. [20] propose a deep learning version by poisoning themost representative samples in the training examples.
Although data poisoning attacks seem toshare the same goal as ours, these methods have a limited impact on DNNs and are not suitablefor data protection tasks.
e.g., the model trained on poisoned examples can still achieve acceptableperformance [20], and the modiﬁed image can be easily distinguished from the original one [42].
Inaddition, the backdoor attack is another type of attack that poisons training data with a trigger pattern[4, 28, 19], but this attack does not prevent the model from learning useful knowledge in the naturaldata.
Therefore, traditional data poisoning methods cannot be used for data protection, while ourproposed method can produce unlearnable examples with imperceptible noise.
Confounder in deep learning.
In statistics, a confounder is a variable that inﬂuences both the causevariable and effect variable, causing a spurious association [23].
If there are confounders in thetraining set, the model may be hard to learn the correct causal relationship from input to output,resulting in sub-optimal performance [41].
A lot of works have been proposed to mitigate the negativeimpact of confounders in deep learning [30, 35, 13, 44].
e.g. , Shen et al.
[30] believe that thebackground in the dataset plays the role of a confounder and they propose to improve by samplereweighting.
Wang et al.
[35] point out that unbalanced data distribution will introduce confoundersand proposed to use do-operation [24] as a solution.
However, all these methods treat the confounderas a defect.
Instead, our proposed method utilizes the property of the confounder to achieve dataprivacy protection.
Problem statement
In this paper, we focus on the scenario of image data encryption.
There are two characters in thisscenario: data owner and unauthorized model trainer.
The data owner needs to encrypt their privatedata to prevent the model trainer from obtaining the high-performance classiﬁcation model on thisdata.
We consider two practical encryption settings:
(1) In-distribution data encryption (standard setting)
This setting follows [29, 12].
The data owner has n labeled natural (unencrypted) data D nat .
He/she can actively use a protection method to obtain encryption (unlearnable) images by addingimperceptible perturbation to the original images.
Since the protection method has access to eachdata to adaptively learn and generate custom unlearnable noises, these processed data are deﬁned asin-distribution encrypted data D in,en .
(2) Out-of-distribution data encryption (fast encryption setting)
The data owner trains a cryptographic tool ( e.g. , a generator) with labeled historical data.
Once theencryption tool is trained, it can be used for fast encryption on any data.
Speciﬁcally, if the data owneruses this tool to encrypt the historical data ( i.e. , the training set of the encryption tool), the processeddata is denoted as D in,en .
Instead, if they use the tool to encrypt some newly unlabeled data ( i.e. , thenon-training set of the encryption tool), the processed data is called out-of-distribution encrypted data
D out,en .
We believe this setting is practical for the real world.
e.g. , a person can install the encryptionapp on the phone in advance, and then their instant photos can be quickly encrypted by this app.
One unauthorized model trainer can download any data from public sources as the training set.
i.e. ,
D tr = {D in,en , D out,en , D nat } , where D nat is collected means that not everyone would encrypt [Figure] their data before publishing.
Then the trainer can label the collected examples x with classes t andtrain the model h with cross-entropy loss (cid:96) , as
[Equation]
where P data represents the distribution of the training set.
Note that data encryptors cannot predictthe information of model training in advance ( e.g. , model backbone, data augmentation), which posesa challenge to the encryption method.
Finally, we measure the effectiveness of the unlearnable example by evaluating the accuracy of modelh on the standard test set D test .
Intuitively, low test accuracy indicates that the model cannot learnexploitable knowledge from the encrypted dataset.
Methodology
In this section, we ﬁrst demonstrate how the native confounder ( i.e. , background) in the image dataaffects the performance of a classiﬁcation model.
Inspired by this phenomenon, we introduce ageneral framework for image data encryption with the causal confounder.
Then we instantiate thisframework as ConfounderGAN.
4.1 Motivation
We observe that the background in some datasets may play the role of the confounder in modellearning, thus reducing model performance.
Speciﬁcally, the image classiﬁcation task can be modeledas a causal graph.
The input image is the cause X , and the output prediction is the effect Y .
Thetraining process can be regarded as the process of using the ground truth information to regressrelationships from images to predictions X
→ Y .
Ideally, if the model can properly learn these truecausal relationships, the classiﬁcation accuracy will be optimal.
However, current data-driven deeplearning techniques are only good at learning correlations rather than causality, thus the co-occurrencefrequency of object and background may inﬂuence the model learning e.g. , if the dog is always onthe grass in the training set, the model may use grass as a key feature of the class dog instead of thedog itself.
In other words, the confounder background B ( i.e. , grass) constructs a spurious correlationbetween images and predictions
X ← B
→ Y so that a biased model is learned.
Nevertheless, wealso need to be aware that native confounders ( e.g. , background) in natural datasets do not play adominant role in training, as models trained on this dataset can still achieve acceptable performance.
4.2 Confounder-based encryption framework
Inspired by the above observations, if we can deliberately introduce a confounder C that is closelyrelated to image X and label Y
( i.e. , X ← C → Y ), the trained model would hardly learn the correctrelationship X
→ Y from this dataset.
We believe this is a general data encryption framework andthere are many possible ways to implement it.
For example, one straightforward approach is to addclass-wise patches (CWP) to the training set.
That is, the images of one class add the same patch,while the patches corresponding to each class are different.
As shown in Figure 2(a), since the patchis added to the image, P → X is constructed.
Meanwhile, the patch of each image is related tothe image’s class, so this builds the path P → Y .
To verify this idea, we select the ﬁrst 25 classes [Figure] from ImageNet to establish a tiny dataset ImageNet25 for experiments.
As illustrated in Figure 2(b),each 224 × 224 image in the training set adds a 36 × 36 patch by class in the lower right corner.
More experimental details are elaborated in Appendix G.2.
Figure 2(c) and Figure 2(d) shows thetraining and test accuracy curves during model training, where the ﬁnal test accuracies on the natural
ImageNet25 and the modiﬁed ImageNet25 are 70.9% and 35.4%, respectively (Figure 2(d)).
Thisdemonstrates that such a simple confounder-based method suppresses the learnable knowledge in thedataset to some extent.
Unfortunately, the patch is too conspicuous in the image, affecting its natural use.
A natural idea is toreplace class-wise patches with imperceptible class-wise noises (CWN).
However, this alternativecan be circumvented by early stopping [12], i.e. , the model trainer can train a high-accuracy modelon this noise-added dataset by early stopping.
Meanwhile, both CWP and CWN have two importantdrawbacks: (1) The patch or the noise is predeﬁned and not correlated with the image, so therelationship between the patch/noise (confounder) and
image X is not strong enough.
(2) Thesemethods are limited to in-distribution data encryption and cannot be applied to out-of-distributiondata encryption.
Hence, how to reasonably introduce a confounder into the training data to overcomethe above drawbacks needs to be further explored.
4.3 ConfounderGAN:Confounder generation method based on GAN
In order to develop an effective encryption algorithm that can cover various practical scenarios,we utilize adversarial generative networks (GAN) [9] to instantiate the above framework as ConfounderGAN.
Our goal is to train a generator whose output confounder noise N satisﬁes the causalrelationship shown in Figure 3(a-2).
Figure 3(a-1) illustrates our overall training pipeline, whichmainly consists of three parts: a generator G θ with learnable parameters θ , a discriminator D φ withlearnable parameters φ , and a pretrained classiﬁer f ω with ﬁxed parameters ω .
The generator G θtakes the private image x as its input and generates a noise G θ ( x ) .
The encrypted image x + G θ ( x )is obtained by combining the original image and noise, as highlighted in green in Figure 3(a-1).
We emphasize that this noise as a confounder is better than the predeﬁned class-wise patches andclass-wise noises, because its generation relies on the original image, so that the two variables onpath N → X have a very strong correlation (Figure 3(a-2)).
Then x + G θ ( x ) will be sent to thediscriminator D φ , which is used to distinguish the generated data and the original data x .
The trainingloss of the discriminator D φ is
[Equation]
where P data represents the distribution of the original data.
It can be found that the discriminator D φis a binary classiﬁer that encourages the generated noise to be imperceptible.
Meanwhile, in order to ensure that the generated noise has the confounder property, the relationship
N → Y in Figure 3(a-2) needs to be constructed.
As highlighted in red in Figure 3(a-1), we achieve Input a generator G θ with learnable parameters θ , a discriminator D φ with learnable parameters φ , a pretrained classiﬁer f ω with ﬁxed parameters ω , training epoch is M , is α . [Equation]
[Equation]
this by classifying noise into the ground truth class t of the original image. Thus training loss of theconfounder noise generator G θ is
[Equation]
where (cid:96) denotes the cross-entropy loss commonly used in classiﬁcation tasks.
Note that(cid:96) ( f ω ( G θ ( x )) , t ) in Equation (3) cannot be replaced with (cid:96) ( f ω ( x + G θ ( x )) , t ) , since f ω ( x ) usually has a high classiﬁcation accuracy for the original image x .
This means that the loss (cid:96) ( f ω ( x ) , t )is low enough, then (cid:96) (( f ω ( x + G θ ( x )) , t ) can be minimized as long as G θ ( x ) = 0 .
To bound the magnitude of the noise and stabilize the GAN’s training [14], we add a soft hinge losson the L 2 norm as
[Equation]
where c denotes a user-speciﬁed bound.
Finally, we train the generator G θ and discriminator D φ bysolving a max-min game, as
[Equation]
where α controls the weight of L confounderθ ( x, t ) and L hingeθ ( x ) during generator training.
See Algorithm 1 for the pseudo-code of ConfounderGAN.
Once G θ is trained, as illustrated in Figure 3(b), the data owner only needs to input the image intothe generator for one forward propagation to achieve the data encryption.
If private data is used totrain the generator, the processed data is called in-distribution encryption data.
Conversely, if privatedata is NOT used to train the generator, the processed data is called out-of-distribution encryptiondata.
The latter setting is especially useful when the user has some new unlabeled data to encrypt, asit does not need extra training.
After this private data is encrypted, the data owner can upload it tothe World Wide Web.
When the model trainer downloads these images, he/she cannot exploit usefulknowledge from these processed images, thus protecting the data privacy of the data owner.
Experiments
In this section, we ﬁrst introduce our experimental setup.
Then we demonstrate the effectiveness ofour proposed method in creating unlearnable examples under in-distribution and out-of-distributiondata encryption settings.
Next, we conduct a series of transferability and stability of our confoundernoise to further illustrate the effectiveness and superiority of our method.
Finally, we show somevisualizations to better understand our confounder noise.
5.1 Experimental setup
We select three natural object datasets and three medical datasets for algorithm evaluation, including
SVHN [21], CIFAR10 [17], ImageNet25 [5], BloodMNIST [43], Keratitis and ISIC [1].
See Appendix F for more details on these datasets.
We mainly focus on sample-wise noise since it canbe applied to various scenarios ( e.g. , encrypting unlabeled out-of-distribution data).
The proposedmethod is compared with the error-minimizing noise (EMN) [12], which is the strongest baseline for [Figure] [Figure] data privacy protection.
We employ ResNet18 [10] as the source model or pretrained classiﬁer forgenerating noise, and train ResNet18 on the unlearnable training sets.
Referring to Huang et al.
[12],the maximum perturbation is set to 8/255 in SVHN and CIFAR [Figure] and 16/255 in other datasets, thusensuring it is imperceptible to human observers.
These settings are ﬁxed for all experiments, unlessotherwise explicitly stated.
See Appendix G for more experimental details [Figure]
5.2 Effectiveness of ConfounderGAN5.2.1 In-distribution data encryption
We ﬁrst examine an extreme case, if all data collected by the model trainer is in-distributionencryption data ( i.e., D tr = 100% D in,en ) , how does the model trained on this data perform?
Theresults are illustrated in Figure 4(a) and Figure 4(b), where ‘natural’ represents the original image.
The corresponding test accuracies of EMN and our method demonstrate that the learnable knowledgeof images is effectively suppressed.
More importantly, all datasets processed by our method have thelowest accuracy, which proves the superiority of our method in data privacy protection.
However, in most situations, not all the training data collected by the model trainer are unlearnableexamples.
For example, only a certain number of web users have decided to use protection techniquesbut not all users.
Thus, one question is naturally raised: If the trainer uses p % in-distributionencryption data D in,en and (1 − p %) natural data D nat for model training, can these encryptiondata D in,en still maintain the unlearnable property in this mixed dataset?
For validation, we trainthe model on only the natural proportion (1 − p %)
D nat for comparison.
The results are shown in
Figure 5, where the X-axis represents the proportion p % of D in,en to the original total dataset and the
Y-axis represents the classiﬁcation accuracy on the same test set D test .
A quick glance at Polyline 3and Polyline 4 tells us that the effectiveness of both EMN and our method drops quickly when thedata are not made 100% unlearnable, because their accuracy is close to Beeline 1 ( i.e. , 100% naturaldataset).
This means that the encryption effect can be approximately ignored even when the noise isapplied to 50% of the data.
However, we can also observe that Polyline 2 is close to Polyline 3 and Polyline 4, which may indicate that the high test accuracy in Polyline 3 and Polyline 4 is caused by(1 − p %) natural data D nat .
To illustrate this, we take p = 90% in SVHN and BloodMNIST as examples and plot their testaccuracy curves during training as shown in Figure 4(c) and Figure 4(d).
We additionally add Curve4 and Curve 5 that only use 90% in-distribution encryption data D in,en for model training.
The resultof these two curves demonstrates that 90% D in,en is still unlearnable and the test accuracy curve ofour method is signiﬁcantly lower than that of EMN on BloodMNIST, which is consistent with the [Table] [Table] result of BloodMNIST in Figure 4(b) (100% D in,en ).
Meanwhile, Curve 1, Curve 2 and Curve 3are almost the same, showing that the exploitable knowledge in Curve 2 and Curve 3 mainly comesfrom 10% natural data.
Therefore, we conclude that both images encrypted by our method or EMNremain unlearnable property even as part of the dataset, and our method maintains its superiority tosome extent.
In addition, we also provide training gradient analysis on our encrypted images in the Appendix A to better understand the effectiveness of our method.
5.2.2 Out-of-distribution data encryption
In real-world scenarios, people sometimes want to quickly encrypt their instant photos with littlecomputing resources.
For example, a person takes a photo with a cell phone and plans to uploadit to social media immediately.
EMN is hard to deploy in this lightweight computing setting, asit needs multiple bi-level gradient optimizations to generate an effective noise.
In contrast, in our
ConfounderGAN, once the confounder noise generator G θ is trained, it only needs to input any imageinto G θ for one forward-propagation to achieve encryption, and no label information is required,which is ideal for the above practical scenario.
However, since the generator has not seen these newlycaptured images during training, the effectiveness of the generated noise on these out-of-distributionimages is unknown.
Therefore, we try to explore the following question: If the trainer uses p %in-distribution encryption data D in,en and (1 − p %) out-of-distribution encryption data D out,enfor model training, can these out-of-distribution encryption data D out,en effectively suppress thelearnable information?
For brevity, we plot the experimental results to Polyline 5 in Figure 5.
The generator for each point in Polyline 4 and Polyline 5 is trained with the same p % data, andperforms in-distribution encryption on these data after training.
The main difference between Polyline4 and Polyline 5 is that the former does not encrypt the remaining (1 − p %) natural data, whilethe latter performs out-of-distribution encryption for these remaining data.
It can ﬁnd that the testaccuracies of Polyline 5 are obviously lower than those of Polyline 4, which demonstrates that theout-of-distribution data encrypted by our method has signiﬁcantly less learnable knowledge than thenatural data D nat , thus proving the effectiveness of our method for out-of-distribution personal data. See Appendix E for more experiments.
5.3 A Series of Analytical Experiments about ConfounderGAN
For simplicity, the analytical experiments in this subsection are all conducted under the 100%in-distribution encryption setting ( i.e. , 100% D in,en ), unless otherwise stated.
5.3.1 Transferability analysis
Both EMN and our ConfounderGAN rely on a pretrained classiﬁer f to generate effective unlearnablenoise.
Speciﬁcally, EMN uses this classiﬁer as a source model to compute the gradient of error- minimizing noise, and our ConfounderGAN needs this classiﬁer to learn the correlation between theconfounder noise and the label.
Therefore, it can be considered that unlearnable noise is customizedfor this classiﬁer f .
If the pretrained classiﬁer f used by the data owner and the commercial model htrained by the unauthorized company share the same backbone, the protection performance shouldbe optimal and the above experiments follow this setting ( i.e. , both are ResNet18).
However, inpractical scenarios, the data owner cannot predict which model h
the trainer will choose as thebackbone.
A natural concern is raised: If the architecture of classiﬁer f and model h are different,can the unlearnable property of the noise trained on classiﬁer f be transferable to the model h ?
Table 1 shows the results of the transferability under in-distribution encryption setting ( i.e. , 100%
D in,en ), where the classiﬁer f employs ResNet18 and the model h selects VGG11 [33], ResNet50and DenseNet121 [11].
The results show our method outperforms EMN on all datasets and evaluatedmodels, especially for VGG11, indicating that our confounder noise is more practical and effective.
5.3.2 Stability analysis
Model trainers are likely to use some data augmentation techniques to improve their model performance during the training phase, so a question is introduced: Can noise still work well withdifferent data augmentations?
To answer this question, we select 7 commonly used and advanceddata augmentation techniques for evaluation on CIFAR10:
Random Horizontal Flip (RHF), Random
Crop (RC), Combination of Horizontal Flip and Crop (RHF+RC), CutOut [7], MixUp [46], CutMix[45], Fast Autoaugment (FA) [18].
See Appendix G.6 for detailed experimental setup.
The results areshown in Table 2.
It can be observed that the noise still maintains high effectiveness in most settings.
Meanwhile, the test accuracies of our method are also lower than those of EMN in all cases, whichfurther proves that our method is better than EMN.
In addition, we also investigate the stability of ConfounderGAN under early stopping and adaptive setting. See Appendix C and Appendix D formore details.
5.4 Visualizations
As shown in Figure 6, we randomly select two images from the ImageNet25 for visualization.
Sincenoises are all limited to a small range, all noises are scaled to the interval [0 , 255] according to theirmaximum and minimum values.
We can obverse that the noises of EMN are small disordered dots inmost cases, while the noises of our method show some high-level representations.
e.g. , feather-liketextures can be observed.
These high-level representations may lead to the better transferability ofour method (as mentioned in Section 5.3.1), since different backbones may share some commonrepresentations.
See Appendix B for more visualizations.
Conclusion
We focus on the image data encryption task, its goal is to use small and invisible noise to preventpersonal data from being freely exploited by deep neural networks.
By leveraging confounderproperty, we propose a confounder-based encryption framework for image data.
i.e. , introduce aconfounder that is strongly correlated with both images and labels in the dataset, so that the modeltrained on this dataset can only learn the spurious correlations between images and labels.
Basedon this framework, we develop confounderGAN, a novel image data encryption method that coversvarious practical scenarios.
Empirically, we demonstrate that our method outperforms the state-of-theart algorithm in the in-distribution data encryption and can be effectively applied to out-of-distributiondata encryption.
Moreover, we also verify that the transferability and robustness of our confoundernoise.
We believe this paper provides new insight into data privacy protection and could have a broadimpact on both the public and the deep learning community.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] The code will
be published after the paper is accepted.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Appendix G.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] In classiﬁcation tasks, the variance of each result issmall.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix G.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Appendix F.
(b) Did you mention the license of the assets?
[Yes] See Appendix F.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]