Data Redaction from Pre-trained GANs
Anonymous Author(s)
Affiliation
Address
Email
Abstract —Large pre-trained generative models are known to occasionally output undesirable samples, which undermines their trustworthiness.
The common way to mitigate this is to re-train them differently from scratch using different data or different regularization – which uses a lot of computational resources and does not always fully address the problem.
In this work, we take a different, more compute- friendly approach and investigate how to post-edit a model after training so that it “redacts”, or refrains from outputting certain kinds of samples.
We show that redaction is a fundamentally different task from data deletion, and data deletion may not always lead to redaction.
We then consider Generative Adversar- ial Networks (GANs), and provide three different algorithms for data redaction that differ on how the samples to be forgotten are described.
Extensive evaluations on real-world image datasets show that our algorithms out-perform data deletion baselines, and are capable of redacting data while retaining high generation quality at a fraction of the cost of full re-training.
I. I NTRODUCTION
Generative Adversarial Networks (GANs) are large neural generative models that learn a complicated probability distribution from data and then generate samples from it.
These models have been immensely successful in many large scale tasks from multiple domains, such as images [Zhu et al., 2020,
Karras et al., 2020, 2021], point clouds [Zhang et al., 2021], video [Tulyakov et al., 2018], text [de Masson d’Autume et al., 2019] and speech [Kong et al., 2020].
However, it is also well-known that many deep generative models frequently output undesirable samples, which makes them less reliable and trustworthy.
Image models generate blurred samples [Kaneko and Harada, 2021] or checkerboard artifacts [Odena et al., 2016, Zhang et al., 2019, Wang et al., 2020, Schwarz et al., 2021], speech models produce unnatural sound [Donahue et al., 2018, Thiem et al., 2020], and language models emit offensive text [Abid et al., 2021, Perez et al., 2022].
Thus, an important question is how to mitigate these artifacts, which would improve the trustworthiness of these models.
One way to mitigate undesirable samples is to re-design the entire training pipeline including data augmentation, model architecture and loss functions, and then re-train the entire model from scratch [Isola et al., 2017, Aitken et al., 2017, Kaneko and
Harada, 2021] – a strategy that has been used in prior work.
This approach is very compute-intensive as modern GANs can be extremely expensive to train.
In addition, other problems may become apparent after training, and resolving them may require multiple re-trainings.
To address this challenge, we consider post-editing , which means modifying a pretrained model in a certain way rather than training it differently from scratch.
This is a much more computationally efficient process that has shown empirical success in many supervised learning tasks [Frankle and Carbin, 2018, Zhou et al., 2021, Taha et al., 2021], but has not been studied much for unsupervised learning.
In particular, we propose a post-editing framework to redact undesirable samples that might be generated by a GAN, which we call data redaction .
A second plausible solution for mitigating undesirable samples is to use a classifier to filter them out after generation.
This approach, however, has several drawbacks.
Classifiers can take a significant amount of space and time after deploymen.
Additionally, if the generative model is handed to a third party, then the model trainer has no control over whether the filter will ultimately be used.
Data redaction via postediting, on the other hand, offers a cleaner solution which does not suffer from these limitations.
A third plausible solution is data deletion or machine unlearning – post-edit the model to approximate a re-trained model that is obtained by retraining from scratch after removing the undesirable samples from the training data.
However, this does not always work – as we show in Section IV-B, deletion does not necessarily lead to redaction in constrained models.
Additionally, the undesirable samples may simply due to inductive biases of the neural generative model and may not exist in the training data; examples include unnatural sounds emitted by speech models and blurred images from image models.
Data redaction, in contrast, can address all these challenges.
There are two major technical challenges that we need to resolve in order to do effective data redaction.
The first is how to describe the samples to be forgotten.
This is important as data redaction algorithms need to be tailored to specific descriptions.
The second challenge is that we need to carefully balance data redaction with retaining good generation quality, which means the latent space and the networks must be carefully manipulated.
In this work, we propose a systematic framework for redacting data from pre-trained generative models (see Section II).
We model data redaction as learning the data distribution restricted to the complement of a redaction set Ω .
We then formalize three ways of describing redaction sets, namely data-based (where a pre-specified set is given), validity-based (where there is a validity checker), and classifier-based (where there is a differentiable classifier).
Then, we introduce three data redaction algorithms, one for each description (see Section III).
Prior works have looked at avoiding negative samples in the re-training setting with different descriptions and purposes [Sinha et al., 2020, Asokan and
Seelamantula, 2020].
They introduce fake distributions to penalize the generation of negative samples.
We extend this idea to data redaction by defining the fake distribution as a mixture of the generative distribution and a redaction distribution supported on Ω .
We prove the optimal generator can recover the target distribution when label smoothing [Salimans et al., 2016, Szegedy et al., 2016, Warde-Farley and
Goodfellow, 2016] is used.
Based on our theory, we introduce the data-based redaction algorithm (Alg. 1).
We then combine this algorithm with an improper active learning algorithm by Hanneke et al.
[2018] and introduce the validity-based redaction algorithm (Alg. 2).
Finally, we propose to use a guide function to guide the discriminator via a classifier, and introduce the classifier-based redaction algorithm (Alg. 3).
Finally, we empirically evaluate these redaction algorithms via experiments on real-world image datasets (see Section IV).
We show that these algorithms can redact quickly while keeping high generation quality.
We then investigate applications of data redaction, and use our algorithms to remove different biases that may not exist in the training set but are learned by the pre-trained model.
This demonstrates that data redaction can be used to reduce biases and improve generation quality, and hence improve the trustworthiness of generative models.
In summary, our contributions are as follows:
• We formalize the problem of post-editing generative models to prevent them from outputting undesirable samples as “data redaction” and establish its differences with data deletion.
• We propose three data augmentation-based algorithms for redacting data from pre-trained GANs as a function of how the inputs to be forgotten are described.
• We theoretically prove that data redaction can be achieved by the proposed algorithms.
• We extensively evaluate our algorithms on real world image datasets.
We show these algorithms can redact data quickly while retaining high generation quality.
Moreover, we find data redaction performs better than data deletion in a de-biasing
II. A F ORMAL F RAMEWORK FOR D ATA R EDACTION
Let p data be the data distribution on R d and
X ∼ p data be i.i.d.
training samples.
Let A be the learning algorithm of generative modelling and
M = A ( X ) be the pre-trained model on X , which learns p data .
In this paper, we consider A to be a
GAN learning algorithm
[Goodfellow et al., 2014a], and M contains two networks, D (discriminator) and G (generator), which are jointly trained to optimize
[Equation]
where p G is the push-forward distribution
G # N (0 , I ) defined as the distribution of G ( Z ) where Z ∼ N (0 , I ) .
A. Data Redaction Framework
Let the redaction set Ω ⊂ R d be the set of samples we would like the model to redact.
Formally, the goal is to develop a redaction algorithm D such that M ′
=
D ( M , Ω) learns the data distribution restricted to the complement ¯Ω = R d \ Ω , i.e. p data | ¯Ω .
Examples of Ω include inconsistent, blurred, unrealistic, or banned samples that are possibly generated by the model.
The redaction set Ω , in addition to the pre-trained model, is considered as an input to the redaction algorithm.
We consider three kinds of Ω , namely data-based, validity-based, and classifier-based.
B. Redaction Set Descriptions
We propose three different descriptions for the redaction set Ω .
First, the data-based Ω is a predefined set of samples in R d , such as a transformation applied on all training samples [Sinha et al., 2020].
Second, the validity-based Ω is defined as all invalid samples according to a validity function v : R d → { 0 , 1 } , where 0 means invalid andmeans valid.
This is similar to the setting in Hanneke et al.
Finally, let f : R d → [0 , 1] be a soft classifier that outputs the probability that a sample belongs to a certain binary class, and τ ∈ (0 , 1) be a threshold.
Then, the classifier-based Ω is defined as { x : f ( x ) < τ } .
For example, f can be an offensive text classifier in language generation tasks [Pitsilis et al., 2018].
These descriptions are general and apply to any kind of generative models.
C. Data Deletion versus Data Redaction
Motivated by privacy laws such as the GDPR and the CCPA, there has been a recent body of work on data deletion or machine unlearning [Cao and Yang, 2015, Guo et al., 2019, Schelter, 2020, Neel et al., 2021, Sekhari et al., 2021, Izzo et al., 2021, Ullah et al., 2021].
In data deletion, we are given a subset set X ′ ⊂
X of the training set to be deleted from an already-trained model, and the goal is to approximate the re-trained model
A ( X \ X ′ ) .
While there are some superficial similarities – in that the goal is to post-edit models in order to “remove” a few data points, there are two very important differences.
The first is that data redaction requires the model to assign zero likelihood to the redaction set Ω in order to avoid generating samples from this region; this is not the case in data deletion – in fact, we present an example below which shows that data deletion of a set X ′ may not cause a generative model to redact X ′ .
Specifically, in Fig.
1, the entire data distribution p data =
N (0 , 1) (blue line) is the standard Gaussian distribution on R .
We set the redaction set Ω =
( −∞ , − 1 . 5] ∪ [1 . 5 , ∞ ) , so the blue samples fall in Ω and orange samples outside.
The learning algorithm
A is the maximum likelihood Gaussian learner that fits the mean and variance of the data.
With n =samples, the learnt density A ( X ) is shown in green.
If the blue samples were deleted , and the model refitted, the newly learnt density A ( X \ X ′ ) would be the red line.
Notice that this red line has considerable density on the blue points – and so these points are not forgotten.
In contrast, the correct redaction solution that redacts samples in Ω would be the orange density.
Thus deletion does not necessarily lead to redaction.
The second difference is that the redaction set Ω may have a zero intersection with the training data, but may appear in the generated data due to artifacts of the model.
Examples include unnatural sounds emitted by speech models, and blurred images from image models.
Data redaction, in contrast to data deletion, can address this challenge.
4 samples / data support 0.
d e n s i t y p data p data | pre-trained density ( X ) re-trained density
( X X 0 ) deletion set Xremaining set X X
III. M ETHODS
In this section, we describe algorithms for each kind of redaction set described in Section II.
We also provide theory on the optimality of the generator and the discriminator.
Finally, we generalize the algorithms to situations where we would like the model to redact the union of multiple redaction sets.
A. Data-based Redaction Set
The data-based redaction set Ω is a pre-defined set of samples we would like the model to redact.
One example is a transformation function NegAug applied to all training samples, where NegAug makes realistic images unrealistic or inconsistent [Sinha et al., 2020].
Another example can be visually nice samples outside data manifold when the training set is small [Asokan and Seelamantula, 2020].
In our framework, the redaction set Ω can be any set of carefully designed or selected samples depending on the purpose of redacting them – which includes but does not limit to improving the generation quality of the model.
For example, we expect the model to improve on fairness, bias, ethics or privacy when Ω is properly constructed with unfair, biased, unethical, or atypical samples.
To redact Ω , we regard both generated samples and Ω to be fake samples, and all training samples that are not in Ω to be real samples [Sinha et al., 2020, Asokan and Seelamantula, 2020].
Let p Ω be a redaction distribution such that supp ( p Ω )
= Ω .
Then, the fake data distribution p fake is a mixture of the generative distribution p G and the redaction distribution p Ω :
[Equation]
where λ ∈ (0 , 1) is a hyperparameter.
We also apply the common label smoothing [Salimans et al., 2016,
Szegedy et al., 2016, Warde-Farley and Goodfellow, 2016] technique to the minimax loss function in order to improve robustness of the discriminator.
Let α + ∈
( 12 , 1] be the positive target (such as 0 . 9 ) and α − ∈ [0 , 12 ) be the negative target (such as 0 . 1 ).
Then, the loss function is
[Equation]
Theorem 1.
The optimal solution to min G max D L ( G, D ) is
[Equation]
We provide the proof and theoretical extension to the more general f
-GAN [Nowozin et al., 2016] setting in Appendix A. In the data-based setting, we let p Ω =
U (Ω) , the uniform distribution on Ω .
We assume Ω has positive, finite Lebesgue measure in R d so that U (Ω) is well-defined.
The proposed method is summarized in Alg. 1.
Our objective function is connected to Sinha et al.
[2020] and Asokan and Seelamantula [2020] in the sense that p Ω is an instance of the negative distribution described in their frameworks.
However, there are several significant differences between our method and theirs: (1) we start from a pre-trained model, (2) we aim to learn p data | ¯Ω rather than p data and therefore do not require Ω ∩ supp ( p data ) to be the empty set, and (3) we consider the common label smoothing technique and provide theory for this setting.
These differences are also true in the following sections.
B. Validity-based Redaction Set
Let v : R d → { 0 , 1 } be a validity function that indicates whether a sample is valid.
Then, validitybased redaction set Ω is the set of all invalid samples { x : v ( x ) = 0 } .
For example, M is a code generation model, and v is a compiler that indicates whether the code is free of syntax errors [Hanneke et al., 2018].
Different from the data-based setting, the validity-based Ω may have infinite Lebesgue measure, such as a halfspace, and consequently U (Ω) may not be well-defined.
To redact Ω , we let p Ω in (2) to be a mixture of p data | Ω and p G | Ω .
This corresponds to a simplified version of the improper active learning algorithm introduced by Hanneke et al.
[2018] with our Alg.
1 as their optimization oracle.
The idea is to apply
Alg. 1 for R rounds.
After each round, we query the validity of T newly generated samples and use invalid samples to form a data-based redaction set Ω ′ .
In contrast to the data-based approach, this active algorithm focuses on invalid samples that are more likely to be generated, and therefore efficiently penalizes generation of invalid samples.
The proposed method is summarized in Alg. 2.
The total number of queries to the validity function v is | X | + T × R .
In case v is expensive to run, we would like to achieve better data redaction within a limited number of queries.
From the datadriven point of view, we hope to collect as many invalid samples as possible.
This is done by setting
R = 1 and T maximized if we assume less invalid samples are generated after each iteration.
However, this may not be the case in practice.
We hypothesis some samples are easier to redact while others harder.
By setting R > 1 , we expect an increasing fraction of invalid generated samples to be hard to redact after each iteration.
Focusing on these hard samples can potentially help the generator redact them.
Since it is hard to directly analyze neural networks, we leave the rigorous study to future work.
In Appendix B, we study a much simplified dynamical system corresponding to Alg. 2, where we show the invalidity (the mass of p G on Ω ) converges to zero, and provide optimal T and R values.
C. Classifier-based Redaction Set
We would like the model to redact samples with certain (potentially undesirable) property.
Let f : R d → [0 , 1] be a soft binary classifier on the property (0 means having the property and 1 means not having it), and τ ∈ (0 , 1) be a threshold.
The classifier-based redaction set Ω is then defined as { x : f ( x ) < τ } .
For example, the property can be being offensive in language generation, containing no speech in speech synthesis, or visual inconsistency in image generation.
We consider f to be a trained machine learning model that is fully accessible and differentiable.
To redact Ω , we let p Ω be a mixture of p data | Ω and p G | Ω , similar to the validity-based approach.
We use f to guide the discriminator and make it able to easily detect samples from Ω .
Let guide ( D, f ) be a guided discriminator that assigns small values to x when f ( x ) <
τ or D ( x ) is small (i.e. x ∼ p fake ), and large values to x when f ( x ) >
τ and D ( x ) is large (i.e. x ∼ p data | ¯Ω ).
Instead of optimizing
L ( G, D ) in (3), we optimize L ( G, guide ( D, f )) .
This will effectively update G by preventing it from generating samples in Ω .
According to Theorem 1, the optimal discriminator is the solution to
[Equation]
Therefore, the design of the guide function must make (5) feasible.
In this paper, we let
[Equation]
The feasibility of (5) is discussed in Appendix C.
The proposed method is summarized in Alg. 3.
The classifier-based Ω generalizes the validity-based Ω .
First, any validity-based Ω can be represented by a classifier-based Ω if we let f = v and τ = 12 .
Next, we note there is a trivial way to deal with classifierbased Ω via the validity-based approach – by setting
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , redaction set Ω .
Initialize G =
G 0 , D = D 0 .
Define the fake data distribution p fake according to (2) with p Ω =
U (Ω) .
Train G, D to optimize (3): min G max D L ( G, D ) .
return M ′ =
( G, D ) .
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , validity function v .
Initialize Ω ′ =
{ x ∈ X : v ( x ) = 0 } , M 0 = M .
for i = 0 , · · · , R − 1 do
Initialize G
=
G i , D
= D i .
Draw T samples X ( i ) gen from G i .
Query v and add invalid samples to Ω ′ :
Ω ′ ← Ω ′ ∪ { x ∈ X
( i ) gen : v ( x ) = 0 } .
Define the fake data distribution p fake according to (2) with p Ω =
U (Ω ′ ) .
Let M i +
=
( G i +1 , D i +1 ) optimize (3): min G max D L ( G, D ) .
end for return M ′ =
( G R , D R )
Algorithm
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , differentiable classifier f .
Initialize G =
G 0 , D = D 0 .
Define the fake data distribution p fake according to (2) with p Ω
= U ( { x ∈ X : f ( x ) < τ } ) .
Train G, D to optimize (3): min G max D L ( G, guide ( D, f )) , where guide ( · , · ) is defined in (6).
return M ′ =
( G, D ) .
v ( x ) = 1 { f ( x ) < τ } .
However, potentially useful information such as values and gradients of f are lost, and we will evaluate this effect in experiments.
In addition, the classifier-based approach does not maintain the potentially large set of invalid generated samples, as this step is automatically done in the guide function.
D. Generalization to Multiple Redaction Sets
Let { Ω k } Kk =1 be disjoint sets in R d , and we would like the model to redact Ω = (cid:83)
Kk =1 Ω k .
In the databased setting, we let p Ω =
U (Ω) =
U ( (cid:83)
Kk =1 Ω k ) .
In the validity-based setting, each Ω k is associated with a validity function v k .
We let the overall validity function to be v ( x ) =
min k v k ( x ) .
In the classifierbased setting, each Ω k is associated with a classifier f k .
Similar to the validity-based setting, we let the overall f to be f ( x ) =
min k f k ( x ) .
IV. E XPERIMENTS
In this section, we aim to answer the following questions.
• How well can the algorithms in Section III redact samples in practice?
• Can these algorithms be used to de-bias pretrained models?
• Can these algorithms be used to understand training data?
In this section, we examine these questions by focusing on several real-world image datasets, including
MNIST ( 28 × 28 )
[LeCun et al., 2010], CIFAR ( 32 × 32 )
[Krizhevsky et al., 2009], CelebA ( 64 × 64 )
[Liu et al., 2015] and STL-10 ( 96 × 96 )
[Coates et al., 2011] datasets.
In Section IV-A, we investigate how well these algorithms can redact samples with a specific label.
In Section IV-B, we investigate how well these algorithms can de-bias pre-trained models and improve generation quality.
In Section IV-C, we use these algorithms to understand training data through the lens of data redaction.
The pre-trained model for each dataset is a
DCGAN [Radford et al., 2015] trained forepochs (see details in Appendix D).
We use one
NVIDIA
GPU to train these models and run experiments.
Evaluation Metrics: invalidity and generation
quality.
The invalidity is defined as the mass of the generation distribution on the redaction set Ω :
Inv ( p G ) =
(cid:82) x ∈ Ω p G ( x ) dx .
In practice, we measure invalidity by generating 50K samples and computing the fraction of these samples that fall into Ω .
The generation quality is measured in Inception
Score ( IS ) [Salimans et al., 2016] and Frechet
Inception Distance ( FID )
[Heusel et al., 2017].
Higher IS or lower FID indicates better quality.
We compute IS for grey-scale images and FID for RGB images.
When measuring quality, we compute IS or FID between 50K generated samples and X ∩ ¯Ω .
Therefore, this score is not comparable with the score w.r.t.
the pre-trained model if the redaction set includes samples in the training set, such as samples with a specific label in Section IV-A. Detailed setup is in Appendix D.
A. Redacting Labels
Question.
How well can the algorithms in Section
III redact samples in practice?
Methodology.
We investigate how well the proposed algorithms can redact samples with a specific label y .
In the data-based setting (Alg. 1), we express this as Ω = { x ∈ X : label ( x ) = y } .
In the validity-based setting (Alg. 2), we express this by setting v ( x ) = 1 { arg max i logit ( x ) i ̸
= y } , where logit is the output of the softmax layer of a pre-trained label classifier
[Chen, 2020].
In the classifier-based setting (Alg. 3), we set f ( x ) = 1 − logit ( x ) y .
In Table I, we compare invalidity and generation quality among different algorithms and datasets when we redact label 0.
We plot invalidity during data redaction in Fig.
We also compare invalidity after one epoch of data redaction in Appendix E-A2.
Mean and standard errors for 5 random runs are reported.
Results for different hyper-parameters and redacting other labels are in Appendix E.
Results.
We find all the algorithms in Section III work quite well with a much fewer number of epochs used for training the pre-trained model (which is 200).
These algorithms are generally comparable.
Therefore, we conclude that the simplest data-based algorithm is good enough to redact samples when those training samples to be forgotten ( X ∩ Ω ) can characterize the redaction set ( Ω ) well.
We also find invalidity rapidly drops after only one epoch of data redaction, indicating these algorithms are very efficient in penalizing invalidity.
While different algorithms perform better on different datasets, they are highly comparable with each other.
The reason why the classifier-based algorithm performs the best on MNIST is possibly that the label classifier on MNIST is almost perfect so its gradient information is accurate.
Visualization.
We sample latents z ∼ N (0 , I ) and choose those corresponding to invalid samples, i.e. G 0 ( z ) ∈ Ω where G 0 is the pre-trained generator.
We select visually good G 0 ( z ) for demonstration.
We visualize G ( z ) during data redaction in Fig.
3, and more visualizations are in Appendix E-C.
This demonstrates how the latent space is manipulated: the label to be forgotten is gradually pushed to other labels, and there is high-level visual similarity between the final G ( z ) and the original G 0 ( z ) .
Effects of other hyper-parameters.
In Table
II, we compare different T (#queries after each epoch) in the validity-based redaction algorithm (Alg.
We fix the total number of queries by setting
T ×
#epochs to be a constant.
Results indicate that a large T may lead to worse invalidity, and there is trade-off between invalidity and quality when setting
T to be small or moderate.
In Appendix E-A4, we compare different λ (hyperparameter in (2)) in the classifier-based redaction algorithm (Alg. 3).
We find there exists a clear tradeoff between invalidity and quality when alternating λ : a larger λ tends to produce better quality, and a smaller λ tends to have better invalidity.
Redacting multiple sets.
We then investigate how well the proposed algorithms can generalize to multiple redaction sets with methods in Section III-D. [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
We focus on the CelebA dataset [Liu et al., 2015], which has 40 labeled attributes.
We use proposed algorithms to redact a combination of these attributes: Ω 1 = { Black_hair and Blurry } , Ω 2 = { Brown_hair and Wear_eyeglasses } , and Ω = Ω 1 ∪ Ω 2 .
These attributes are randomly selected from those easy to capture.
See detailed setup in
Appendix E-D. Results after 1 or 5 epochs are reported in Table III.
Consistent with results on redaction just one label, all algorithms can reduce invalidity and retain generation quality and are comparable, while the classifier-based algorithm achieves the best invalidity after one epoch.
B. Model De-biasing
There can be different artifacts in GAN generated samples, and these could harm the overall generation quality.
These artifacts may not exist in training samples, but are caused by inductive biases of the model, and become obvious after training.
We can post-edit a pre-trained model to remove these artifacts, which we call model de-biasing .
In this section, we investigate how well Alg. 2 and Alg.apply to this task.
We assume training samples are not biased so Alg.
1 does not apply to de-biasing.
To use these algorithms for de-biasing, we assume the target artifact or bias can be automatically detected by a classifier f or a validity function v .
[Table]
Specifically, we survey two kinds of biases: boundary artifacts and label biases.
Boundary artifacts.
A GAN trained on MNIST might generate samples that have numerous white pixels on the boundary (see Appendix F-A).
We call this phenomenon the boundary artifact .
We use the validity-based algorithm (Alg. 2) to de-bias boundary artifacts.
The validity function is defined as v ( x ) = 1 { (cid:80) ( i,j ) ∈ boundary pixels x ij < τ b } , where boundary pixels are those within a certain margin to the boundary, and threshold τ b satisfies no training image is invalid.
Results are reported in Table IV.
It is clear that the invalidity reduces in order after data redaction, indicating boundary artifacts are largely removed.
Consistent with Table II, a small or moderate T leads to better results.
We visualize samples before and after de-biasing in Appendix F-A.
Label biases.
Neural networks may generate visually smooth but semantically ambiguous samples [Kirichenko et al., 2020], e.g. samples that look like multiple objects (see Appendix F-B).
We call this phenomenon the label bias .
We use the classifier-based algorithm (Alg. 3) to de-bias label biases.
The classifier is defined as f ( x ) =
−
Entropy( logit ( x )) / log (#classes) , where the logit function is the same as in Section
IV-A. We also compare to a data deletion baseline, where we delete invalid samples and fully re-train the model.
Results are reported in Table V and VI.
After de-biasing, we can improve the generation quality by a significant gap ( ∼ 0 . 3 in IS and ∼in FID ). There is also a clear drop in terms of invalidity. In contrast, we find that data deletion does not help removing label biases.
C. Understanding Training Data through the Lens of Data Redaction
Large datasets can be hard to analyze.
In this section, we investigate how data redaction can help
[Table] [Table] [Table]
us understand these data.
Specifically, we ask: which samples are easy or hard to redact?
In order to quantify the difficulty to redact a sample, we define the redaction score FS to be the difference of discriminator outputs before and after data redaction. Formally, let x ∈ Ω be a sample to redact, M = ( G 0 , D 0 ) be the pretrained model, and M ′ = ( G ′ , D ′ ) be a model after data redaction.
Then, the redaction score is
FS ( x ) =
D 0 ( x ) − D ′
( x ) .
A larger FS means it is easier to redact x .
To investigate sample-level redaction difficulty, we redact a particular label at one time using Alg. 1.
We then demonstrate scatter plots of redaction scores
FS ( x ) versus pre-trained discriminator outputs
D 0 ( x ) for all samples x with this label.
We also fit linear regression and report R 2 values (larger means stronger linear relationship).
Scatter plots for some labels in MNIST and CIFAR-100 and distribution of R 2 for all labels are shown in Fig.
We also visualize the most and least difficultto-redact samples in Appendix G.
We find there is positive correlation between FS ( x ) and D 0 ( x ) , indicating on-manifold (large D 0 ( x ) ) samples are easier to be forgotten, while off-manifold (small
D 0 ( x ) )
ones are harder to be forgotten.
This analysis further provides a way to investigate label-level redaction difficulty.
By averaging redaction scores for samples associated with each label, we can survey which labels are easy or hard to redact in general.
The results are in Appendix G.
We find some labels are harder to redact than others.
0.34 forgetting score 0.44 0.
p r e t r a i n e d D o u t p u t linear reg ( R 2 =0.41) (a) MNIST (0) 0.
0.9 forgetting score 0.
p r e t r a i n e d D o u t p u t linear reg ( R 2 =0.71) (
[Figure] [Figure] aquarium [Figure]
fish ) 0.
{+0.8 Rmnist cifar
+}
d a t a s e t
[Figure]
D. Discussion: Relationship to Adversarial Samples
An adversarial sample for a classifier f and a sample x is another sample ˜ x
≈ x but f (˜ x ) ̸ = f ( x ) .
Generating and defending these samples have become one of the most important directions of deep learning [Goodfellow et al., 2014b, Madry et al., 2018].
In this section, we show a variant of Alg.can potentially be used to define a specific type of adversarial samples.
In detail, we fix the discriminator D and only update the generator G while running Alg. 3.
Then, G is trained to fool D and the classifier f at the same time.
Notice that fooling
D means generating on-manifold (visually similar to training data) samples, and fooling f means finding adversarial samples of f .
By combining these objectives we can force G to produce on-manifold adversarial samples, which may be significant in many real-world applications.
We visualize some samples in Appendix H.
V. R ELATED W ORK
Although deep generative models have been highly successful at many domains, it has long been known that they often emit undesirable samples and samples with different types of artifacts that make them untrustworthy.
Examples include blurred image samples [Kaneko and Harada, 2021], fairness issues [Tan et al., 2020, Karakas et al., 2022], and checkerboard artifacts [Odena et al., 2016, Zhang et al., 2019, Wang et al., 2020, Schwarz et al., 2021] in image generation, offensive text in language models [Abid et al., 2021, Perez et al., 2022], and unnatural sound in speech models [Donahue et al., 2018, Thiem et al., 2020].
Some prior works have used post-editing to remove artifacts and improve GANs.
Examples include improving fairness [Tan et al., 2020, Karakas et al., 2022], rule rewriting [Bau et al., 2020], discovering interpretability [Härkönen et al., 2020], and fine-tuning [Mo et al., 2020, Li et al., 2020,
Zhao et al., 2020].
The purpose, use cases, and editing methods of these papers are different from our paper, where we focus on data redaction.
While our problem definition and formalization is novel, the technical solutions that we propose are related to three prior works that use these techniques in different contexts.
These are NDA [Sinha et al., 2020], Rumi-GAN [Asokan and Seelamantula, 2020], and Hanneke et al.
The first two papers look at how to avoid generating negative samples while training a generative model from scratch.
This is done by defining new fake distributions to penalize the generation of these samples.
However, their purposes are different from us: NDA is used to characterize the boundary of the support of the generative distribution more precisely, and Rumi-GAN is used to handle unbalanced data.
We extend their idea and theory to data redaction in Section III.
1 Hanneke et al.
[2018] propose an active learning approach to avoid generating invalid samples, also while training a generative model from scratch.
Their work however is entirely theoretical and apply to discrete distributions.
In our paper, the validity-based redaction algorithm (Alg. 2) is based on a simplified version of their algorithm.
We also use their definition of invalidity as an evaluation method.
Our work is also related to data deletion or machine unlearning [Cao and Yang, 2015, Guo et al., 2019, Schelter, 2020, Neel et al., 2021, Sekhari et al., 2021, Izzo et al., 2021, Ullah et al., 2021,
Bourtoule et al., 2021].
However, there are two important differences between data deletion and data redaction.
First, data deletion aims to approximate the re-trained model when some training samples are removed – mostly due to privacy reasons – while in data redaction we penalize the model from knowing samples that should be forgotten.
Another difference is that in data redaction, the redaction set Ω may have a zero intersection with training data.
These two differences are discussed in Section II-C in detail.
In addition, most data deletion techniques are for supervised learning or clustering, and is much less studied for generative models.
There is also a related line of work on catastrophic forgetting in supervised learning [Kirkpatrick et al., 2017] and generative models [Thanh-Tung and Tran, 2020].
This concept is different from data redaction in that we would like the generative model to redact certain data after training, while catastrophic forgetting means knowledge learned in previous tasks is destroyed during continual learning.
VI. C ONCLUSION
In this paper, we propose a systematic framework for redacting data from pre-trained generative models.
We provide three different algorithms for GANs that differ on how the samples to be forgotten are described.
We provide theoretical results that data redaction can be achieved.
We then empirically investigate data redaction on real-world image datasets, and show that our algorithms are capable of redacting data while retaining high generation quality at a fraction of the cost of full re-training.
One limitation or our paper is that the proposed framework only applies to unconditional generative models.
It is an important future direction to define data redaction and propose algorithms for conditional generative models, which are more widely used in downstream deep learning applications.
A PPENDIX A P ROOF OF T HEOREM 1 AND E XTENSION TO f -GAN
a) Background of f -GAN [Nowozin et al., 2016].
: Let ϕ be a convex, lower-semicontinuous function such that ϕ (1) = 0 .
In f -GAN, the following ϕ -divergence is minimized:
[Equation]
According to the variational characterization of ϕ -divergence
[Nguyen et al., 2010],
[Equation]
where the optimal T is obtained by T = ϕ ′ (cid:16) PQ (cid:17) .
b) The objective function (3) corresponds to an f -GAN.
: Let α = α −
+ α + .
We can rewrite (3) as
[Equation]
where
[Equation]
Let
[Equation]
{+Then, ϕ (1) = 0 , and ϕ ′′ ( u ) = α (2 − α ) u ( αu − α +2) > 
+}
so ϕ is convex.
Its convex conjugate function ϕ ∗ is
[Equation]
Let T ( x ) =
α log D ( x ) .
Then,
[Equation]
Optimal D
[Equation]
: We have
[Equation]
Therefore, the optimal discriminator is
[Equation]
or
[Equation]
Finally, the optimal discriminator in (4) is obtained by inserting (2) into the above equation.
d) Optimal G .
: For conciseness, we let
[Equation]
Then, we have
P
[Equation]
We also have
[Equation]
Because supp ( P 1 ) ∩ supp ( P 3 ) is the empty set
[Equation]
we have
[Equation]
Let
[Equation]
We have
[Equation]
Let
[Equation]
According to Jensen’s inequality,
[Equation]
Therefore
[Equation]
we have
[Equation]
η
[Equation]
Now
[Equation]
we show the η term is non-negative.
We write
[Equation]
It suffices to prove the function ψ ( u ) =
ϕ ( u ) /u
satisfies
[Equation]
We use the Mathematica software [Inc.] to compute the difference:
[Equation]
The minimum value of the above difference for α − ∈ [0 , 12 ] , α + ∈ [0 , 12 ]
[Equation]
and λ ∈ [0 , 1] is obtained at α − =
α + = 12 , where the difference equals zero.
This makes us able to conclude
[Equation]
Finally
[Equation]
we let P 2 = P 1 .
In this case,
[Equation]
Therefore, the optimal generator is p G = p data | ¯Ω .
e) Extension to f -GAN.
: We can extend the objective (3) to any type of f -GAN.
Let ϕ be a convex, lower-semicontinuous function such that ϕ (1) = 0 .
Let
[Equation]
We jointly optimize
[Equation]
Then, the optimal discriminator is D = ϕ ′ (cid:16) PQ (cid:17) .
If ψ (cid:16) β 3 γ 3 (cid:17) ≥ ψ (cid:16) 1 − β 3 1 − γ 3 (cid:17) , then the optimal generator is p G = p data | ¯Ω .
Remark 1.
{+When α − = 0 and α + = 
+}
(i.e. there is no label smoothing), Theorem 1 in Sinha et al.
[2020] implies the above optimal generator.
Our theorem also extends their theorem to the label smoothing setting.
A PPENDIX B T HEORETICAL A NALYSIS OF A S IMPLIFIED D YNAMICAL S YSTEM ON I NVALIDITY
In this section, we provide theoretical analysis to a simplified, ideal dynamical system that corresponds to Alg. 2 and Section III-B.
In this dynamical system, we assume there are only two types of invalid samples: those easy to redact, and those hard to redact.
We assume after each iteration, the generator will generate a less but positive fraction of invalid samples.
Formally, let { Ω easy , Ω hard } be a split of Ω , where Ω easy is the set of invalid samples that are easy to redact, and Ω hard is the set of invalid samples that are hard to redact.
We let
[Equation]
Then, m easy is the fraction of invalid generated samples that are easy to redact, and m hard is the fraction of invalid generated samples that are hard to redact.
m easy
+ m hard is the fraction of invalid generated samples over all generated ones, which we call invalidity .
We use superscript to represent each iteration.
We consider the following dynamical system:
[Equation]
In other words, the improvement of m easy and m hard (in terms of multiplication factor) is only affected by m ratio and T .
We make this assumption because in practice, the number of invalid samples to optimize the loss function is always fixed.
As for boundary conditions, we assume m 0easy > m 0hard .
We assume for η ∈ { η easy , η hard } , 0 < η ( m, T ) ≤ 1 , where equality holds only in these situations:
[Equation]
We also assume a larger T leads to smaller η , but this effect degrades as T increases:
[Equation]
To distinguish between samples that are easy or hard to redact, we assume
[Equation]
We can now draw some conclusions below.
a) As i → ∞ , invalidity converges to 0.
: Because η easy ( T ) < 1 and η hard ( T ) < 1 when T > 0 , we have m i +1easy ≤ m i easy and m i +1hard ≤ m i hard .
According to the monotone convergence theorem, there exists m ∞ easy ≥ 0 and m ∞ hard ≥ 0 such that
[Equation]
We now prove m ∞ easy = m ∞ hard = 0 .
If otherwise, there exists m ∞ ratio =
[Equation]
such that m i ratio → m ∞ ratio .
We then have
[Equation]
If m ∞ easy > 0 , then m ∞ ratio > 0 , and η easy ( m ∞ ratio , T ) < 1 , contradiction.
Similarly, if m ∞ hard > 0 , then m ∞ ratio < 1 , and η hard ( m ∞ ratio , T ) < 1 , contradiction.
Therefore, we conclude both m i easy and m i hard converge to 0.
This indicates the invalidity converges to zero.
b) Simplifying the dynamical system.
: To further simplify the problem, we make a strong assumption that η is linear in m .
Then, we must have
[Equation]
where ξ ∈ [0 , 1] , ξ (0) = 0 , ξ ′ > 0 , ξ ′′ < 0 for ξ ∈ { ξ easy , ξ hard } .
We also have ξ ′ easy > ξ ′ hard and therefore ξ easy > ξ hard .
c) Optimal T and R from bounds.
: We have
[Equation]
Because ξ easy ( T ) ≥ ξ hard ( T ) , we have ξ easy
[Equation]
This leads to
[Equation]
and therefore
[Equation]
[Equation]
R .
Assume the number of queries, T × R , is fixed.
Then, the optimal T from the lower bound is
[Equation]
By setting the derivative to be zero, we have T ∗ low is the solution to − Tξ
[Equation]
Similarly, the optimal T from the upper bound is
[Equation]
By setting the derivative to be zero, we have T ∗ upp is the solution to − T
[Equation]
A PPENDIX C F EASIBILITY OF D ISCRIMINATOR IN THE C LASSIFIER - BASED S ETTING
The solution to (5) and (6) is:
[Equation]
which satisfies D ∗ ∈ [0 , 1]
Therefore, (5) is feasible with the guide function defined in (6).
A
D E XPERIMENTAL S ETUP
Pre-training.
We use DCGAN [Radford et al., 2015] with latent dimension = 128 as the model.
The pre-trained model is trained with label smoothing ( α + = 0 .
9 , α − = 0 .
[Equation]
D ( x ) + (
[Equation]
α +
[Equation]
log(
[Equation]
D ( x ))]
[Equation]
We use Adam optimizer with learning rate = 2 × 10 − 4 , β 1 = 0 .
5 , β 2 = 0 .
999 to optimize both the generator and the discriminator.
The networks are trained for 200 epochs with a batch size of 64.
For each iteration over one mini-batch, we let K D be the number of times to update the discriminator, and K G the number of times to update the generator.
We use K D = 1 and K G = 5 to train.
Data redaction.
The setup is similar to the pre-training except for two differences.
The number of epochs is much smaller: 8 for MNIST, 30 for CIFAR, and 40 for STL-10.
We let K G = 1 for MNIST and CIFAR and K G = 5 for STL-10.
Evaluation.
To measure invalidity, we generate 50K samples, and compute the fraction of these samples that are not valid (e.g., classified as the label to be forgotten, or with pre-defined biases).
It is the lower the better.
The invalidity for redacting labels is measured based on label classifiers.
We use pre-trained classifiers on these datasets.
The other evaluation metric is generation quality.
The inception score ( IS ) [Salimans et al., 2016] is computed based on logit distributions from the above pre-trained classifiers.
It is the higher the better.
The
Frechet Inception Distance ( FID )
[Heusel et al., 2017] is computed based on an open-sourced PyTorch implementation.
3 It is the lower the better.
When computing these quality metrics, we generate 50K samples, and compare to the set of valid training samples: { x ∈ X : x / ∈ Ω } .
Therefore, when X ∩ Ω is not the empty set (such as redacting labels in Section IV-A), the quality measure of the model after data redaction is not directly comparable to the pre-trained model, but these scores among different redaction algorithms are comparable and give intuition to the generation quality.
When X ∩ Ω is the empty set (such as de-biasing in Section IV-B), the quality measures of the pre-trained model and the model after data redaction are directly comparable.
A. Redacting Label
1) Main results: We include results for redacting label 0 in this section.
We look at MNIST, CIFAR-10, and STL-10 datasets with different sets of hyper-parameters.
With the base set of hyper-parameters, while different redaction algorithms perform better on different datasets, they are highly comparable with each other.
We find the results are worse when there is no label smoothing ( α + = 1 , α − = 0 ), indicating label smoothing is important for data redaction.
We discuss results after one epoch in Appendix E-A2, the effect of λ in Appendix E-A4, and the effect of T in Table II
[Table]
Results for MNIST:
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table]
2) Invalidity after one epoch: We compare invalidity after only one epoch of data redaction. These redaction algorithms are highly comparable to each other. We hypothesis that the classifier-based algorithm performs the best on MNIST because a label classifier on MNIST (and its gradient information) can be very accurate, while this may not be true for CIFAR-10 and STL-10.
[Table]
3) Quality during data redaction: We plot quality measure of different data redaction algorithms on different datasets during the redaction process, complementary to the invalidity in Fig.
We find the variances of quality measure is higher than the invalidity, but different redaction algorithms are generally comparable.
[Figure]
4) Trade-off by alternating λ : We study the effect of λ (hyper-parameter in (2)) in Table XVII and Fig. There is a trade-off by alternating λ : a larger λ (less fake data from the redaction set) leads to better quality measure, and a smaller λ (more fake data from the redaction set) leads to better invalidity. [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
B. Redacting Other Labels
We also demonstrate results for redacting other labels with our data redaction algorithms. We use the base set of hyper-parameters in Appendix E-A. Similar to redacting label 0, all redaction algorithms can largely reduce invalidity, and they are highly comparable to each other. The classifier-based redaction algorithm achieves slightly better generation quality on MNIST and CIFAR-10. In terms of different labels, we find some labels are harder to redact in the sense that the invalidity scores for these labels are higher than other scores, such as label 9 in MNIST
[Table] and label 3 in CIFAR-10 and STL-10. [Table] [Table]
on STL- [Table]
C. Visualization
D. Detailed Setup of Redacting Multiple Sets
We use 30K images from CelebA-64 as the training set. All other hyper-parameters are the same as the base set for STL-10 in Appendix E-A, except that we run data redaction algorithms for only 5 epochs.
We train attribute classifiers for each attribute separately. The attribute classifiers are fine-tuned from open-sourced pre-trained ResNet [He et al., 2016].
4 We fine-tune the network for 20 epochs using the SGD optimizer with learning rate = 1 × 10 − 3 , momentum = 0 . 9 , and a batch size of 64. A PPENDIX F
M ODEL D E - BIASING A. Boundary Artifacts Let the image size be W × H (the number of channels is 1 for MNIST).
For an integer margin, the boundary pixels are defined as { ( i, j ) : 1 ≤ i ≤ margin or W − margin < i ≤ W, 1 ≤ j ≤ margin or H − margin < j ≤ H } . Then, the validity function for boundary artifacts is defined as
[Equation]
where τ b = 4 .
25 for margin = 1 and {+10 . 
+}
for margin = 2 . For these values, no training data has the boundary artifact. Quantitative results are in Tabel IV.
We visualize some samples with boundary artifacts in Fig. 8a. We run the validity-based redaction algorithm with λ = 0 .
98 , α + = 0 . 95 , α − = 0 .
05 forepochs. After de-biasing via data redaction, these samples have less boundary pixels, as shown in Fig. 8b. [Figure]
B. Label Biases
We use classifier-based redaction algorithm to de-bias label biases.
For MNIST, we use λ = 0 . 8 , α + = 0 . 95 , α − = 0 .
and run for 8 epochs. For CIFAR-10, we use λ = 0 .
9 , α + = 0 . 9 , α − = 0 . and run for 30 epochs.
Quantitative results are in Table V and VI.
We visualize semantically ambiguous samples generated by the pre-trained model in Fig.
9a.
After de-biasing via data redaction, these samples become less semantically ambiguous, as shown in Fig.
9b.
[Figure]
U
We visualize some most and least difficult-to-redact samples according to the redaction scores in
Fig.
10 and Fig.
We find the most difficult-to-redact samples are visually atypical, while the least difficult-to-redact samples are visually more common.
[Figure]
[Figure]
B. Label-level redaction difficulty
We sort all labels according to their average redaction scores.
This tells us which labels are easier or harder to redact.
The results for MNIST are in Fig.
Consistent with Table XVIII, label 9 is the most difficult label to redact.
The most and least difficult-to-redact labels for CIFAR-100 are shown in Fig. 13a and 13b.
[Figure] [Figure]
[-fish ) 0. 0.8 Rmnist cifar
 d a t a s e t-]D output (pretrained) D output (after forgetting) forgetting
[Figure] [Figure]
C. Relative redaction score
We also study the relative redaction score FS rel ( x ) = ( D 0 ( x {+) − D ′ ( x )) /D 
+}
( x ) , and find results are similar to the redaction scores.
Some visualizations are shown below: [Figure] [Figure]
A PPENDIX H R ELATIONSHIP TO A DVERSARIAL S AMPLES
Consider the classifier-based redaction algorithm. We fix the discriminator and only update the generator.
Then, the generator is trained to fool both the discriminator and the classifier at the same time.
We may define generated samples from this generator as on-manifold adversarial samples to ( D, f ) . Note that on-manifold samples are not necessarily visually clear samples; instead, they could be high likelihood samples according to the inductive bias of the generative model.
When training, we use the classifier-based redaction algorithm with τ = 0 .
5 , α + = 0 .
95 , α − = 0 .
05 , λ = 0 . 85 , and a batch size of 64.
We “redact” one label at a time, similar to experiments in Appendix E-B.
After each iteration over one mini-batch, we generate samples with the same latents.
The visualization is shown below.
There are several interesting findings shown in the figures.
First, the generated samples tend to have less pixels.
Second, the generated samples tend to be dis-connected.
Third, there are some general patterns across these generated samples (for each label): for example, there are pixels in the middle of zeroes, the bottom of sevens vanish, and nines are split from the middle.
We conjecture that these observations correspond to the inductive bias of the discriminator and adversarial samples of the classifier.