Data Redaction from Pre-trained GANs
Anonymous Author(s)
Affiliation
Address
Email
Abstract —Large pre-trained generative models are known to occasionally output undesirable samples, which undermines their trustworthiness.
The common way to mitigate this is to re-train them differently from scratch using different data or different regularization – which uses a lot of computational resources and does not always fully address the problem.
In this work, we take a different, more compute- friendly approach and investigate how to post-edit a model after training so that it “redacts”, or refrains from outputting certain kinds of samples.
We show that redaction is a fundamentally different task from data deletion, and data deletion may not always lead to redaction.
We then consider Generative Adversar- ial Networks (GANs), and provide three different algorithms for data redaction that differ on how the samples to be forgotten are described.
Extensive evaluations on real-world image datasets show that our algorithms out-perform data deletion baselines, and are capable of redacting data while retaining high generation quality at a fraction of the cost of full re-training.
I. I NTRODUCTION
Generative Adversarial Networks (GANs) are large neural generative models that learn a complicated probability distribution from data and then generate samples from it.
These models have been immensely successful in many large scale tasks from multiple domains, such as images [Zhu et al., 2020,
Karras et al., 2020, 2021], point clouds [Zhang et al., 2021], video [Tulyakov et al., 2018], text [de Masson d’Autume et al., 2019] and speech [Kong et al., 2020].
However, it is also well-known that many deep generative models frequently output undesirable samples, which makes them less reliable and trustworthy.
Image models generate blurred samples [Kaneko and Harada, 2021] or checkerboard artifacts [Odena et al., 2016, Zhang et al., 2019, Wang et al., 2020, Schwarz et al., 2021], speech models produce unnatural sound [Donahue et al., 2018, Thiem et al., 2020], and language models emit offensive text [Abid et al., 2021, Perez et al., 2022].
Thus, an important question is how to mitigate these artifacts, which would improve the trustworthiness of these models.
One way to mitigate undesirable samples is to re-design the entire training pipeline including data augmentation, model architecture and loss functions, and then re-train the entire model from scratch [Isola et al., 2017, Aitken et al., 2017, Kaneko and
Harada, 2021] – a strategy that has been used in prior work.
This approach is very compute-intensive as modern GANs can be extremely expensive to train.
In addition, other problems may become apparent after training, and resolving them may require multiple re-trainings.
To address this challenge, we consider post-editing , which means modifying a pretrained model in a certain way rather than training it differently from scratch.
This is a much more computationally efficient process that has shown empirical success in many supervised learning tasks [Frankle and Carbin, 2018, Zhou et al., 2021, Taha et al., 2021], but has not been studied much for unsupervised learning.
In particular, we propose a post-editing framework to redact undesirable samples that might be generated by a GAN, which we call data redaction .
A second plausible solution for mitigating undesirable samples is to use a classifier to filter them out after generation.
This approach, however, has several drawbacks.
Classifiers can take a significant amount of space and time after deploymen.
Additionally, if the generative model is handed to a third party, then the model trainer has no control over whether the filter will ultimately be used.
Data redaction via postediting, on the other hand, offers a cleaner solution which does not suffer from these limitations.
A third plausible solution is data deletion or machine unlearning – post-edit the model to approximate a re-trained model that is obtained by retraining from scratch after removing the undesirable samples from the training data.
However, this does not always work – as we show in Section IV-B, deletion does not necessarily lead to redaction in constrained models.
Additionally, the undesirable samples may simply due to inductive biases of the neural generative model and may not exist in the training data; examples include unnatural sounds emitted by speech models and blurred images from image models.
Data redaction, in contrast, can address all these challenges.
There are two major technical challenges that we need to resolve in order to do effective data redaction.
The first is how to describe the samples to be forgotten.
This is important as data redaction algorithms need to be tailored to specific descriptions.
The second challenge is that we need to carefully balance data redaction with retaining good generation quality, which means the latent space and the networks must be carefully manipulated.
In this work, we propose a systematic framework for redacting data from pre-trained generative models (see Section II).
We model data redaction as learning the data distribution restricted to the complement of a redaction set Ω .
We then formalize three ways of describing redaction sets, namely data-based (where a pre-specified set is given), validity-based (where there is a validity checker), and classifier-based (where there is a differentiable classifier).
Then, we introduce three data redaction algorithms, one for each description (see Section III).
Prior works have looked at avoiding negative samples in the re-training setting with different descriptions and purposes [Sinha et al., 2020, Asokan and
Seelamantula, 2020].
They introduce fake distributions to penalize the generation of negative samples.
We extend this idea to data redaction by defining the fake distribution as a mixture of the generative distribution and a redaction distribution supported on Ω .
We prove the optimal generator can recover the target distribution when label smoothing [Salimans et al., 2016, Szegedy et al., 2016, Warde-Farley and
Goodfellow, 2016] is used.
Based on our theory, we introduce the data-based redaction algorithm (Alg. 1).
We then combine this algorithm with an improper active learning algorithm by Hanneke et al.
[2018] and introduce the validity-based redaction algorithm (Alg. 2).
Finally, we propose to use a guide function to guide the discriminator via a classifier, and introduce the classifier-based redaction algorithm (Alg. 3).
Finally, we empirically evaluate these redaction algorithms via experiments on real-world image datasets (see Section IV).
We show that these algorithms can redact quickly while keeping high generation quality.
We then investigate applications of data redaction, and use our algorithms to remove different biases that may not exist in the training set but are learned by the pre-trained model.
This demonstrates that data redaction can be used to reduce biases and improve generation quality, and hence improve the trustworthiness of generative models.
In summary, our contributions are as follows:
• We formalize the problem of post-editing generative models to prevent them from outputting undesirable samples as “data redaction” and establish its differences with data deletion.
• We propose three data augmentation-based algorithms for redacting data from pre-trained GANs as a function of how the inputs to be forgotten are described.
• We theoretically prove that data redaction can be achieved by the proposed algorithms.
• We extensively evaluate our algorithms on real world image datasets.
We show these algorithms can redact data quickly while retaining high generation quality.
Moreover, we find data redaction performs better than data deletion in a de-biasing
II. A F ORMAL F RAMEWORK FOR D ATA R EDACTION
Let p data be the data distribution on R d and X ∼ p data be i.i.d.
training samples.
Let A be the learning algorithm of generative modelling and
M = A ( X ) be the pre-trained model on X , which learns p data .
In this paper, we consider A to be a
GAN learning algorithm
[Goodfellow et al., 2014a], and M contains two networks, D (discriminator) and G (generator), which are jointly trained to optimize
[Equation]
where p G is the push-forward distribution
G # N (0 , I ) defined as the distribution of G ( Z ) where Z ∼ N (0 , I ) .
A. Data Redaction Framework
Let the redaction set Ω ⊂ R d be the set of samples we would like the model to redact.
Formally, the goal is to develop a redaction algorithm D such that M ′
=
D ( M , Ω) learns the data distribution restricted to the complement ¯Ω = R d \ Ω , i.e. p data | ¯Ω .
Examples of Ω include inconsistent, blurred, unrealistic, or banned samples that are possibly generated by the model.
The redaction set Ω , in addition to the pre-trained model, is considered as an input to the redaction algorithm.
We consider three kinds of Ω , namely data-based, validity-based, and classifier-based.
B. Redaction Set Descriptions
We propose three different descriptions for the redaction set Ω .
First, the data-based Ω is a predefined set of samples in R d , such as a transformation applied on all training samples [Sinha et al., 2020].
Second, the validity-based Ω is defined as all invalid samples according to a validity function v : R d → { 0 , 1 } , where 0 means invalid andmeans valid.
This is similar to the setting in Hanneke et al.
Finally, let f : R d → [0 , 1] be a soft classifier that outputs the probability that a sample belongs to a certain binary class, and τ ∈ (0 , 1) be a threshold.
Then, the classifier-based Ω is defined as { x : f ( x ) < τ } .
For example, f can be an offensive text classifier in language generation tasks [Pitsilis et al., 2018].
These descriptions are general and apply to any kind of generative models.
C. Data Deletion versus Data Redaction
Motivated by privacy laws such as the GDPR and the CCPA, there has been a recent body of work on data deletion or machine unlearning [Cao and Yang, 2015, Guo et al., 2019, Schelter, 2020, Neel et al., 2021, Sekhari et al., 2021, Izzo et al., 2021, Ullah et al., 2021].
In data deletion, we are given a subset set X ′ ⊂
X of the training set to be deleted from an already-trained model, and the goal is to approximate the re-trained model
A ( X \ X ′ ) .
While there are some superficial similarities – in that the goal is to post-edit models in order to “remove” a few data points, there are two very important differences.
The first is that data redaction requires the model to assign zero likelihood to the redaction set Ω in order to avoid generating samples from this region; this is not the case in data deletion – in fact, we present an example below which shows that data deletion of a set X ′ may not cause a generative model to redact X ′ .
Specifically, in Fig.
1, the entire data distribution p data =
N (0 , 1) (blue line) is the standard Gaussian distribution on R .
We set the redaction set Ω =
( −∞ , − 1 . 5] ∪ [1 . 5 , ∞ ) , so the blue samples fall in Ω and orange samples outside.
The learning algorithm
A is the maximum likelihood Gaussian learner that fits the mean and variance of the data.
With n =samples, the learnt density A ( X ) is shown in green.
If the blue samples were deleted , and the model refitted, the newly learnt density A ( X \ X ′ ) would be the red line.
Notice that this red line has considerable density on the blue points – and so these points are not forgotten.
In contrast, the correct redaction solution that redacts samples in Ω would be the orange density.
Thus deletion does not necessarily lead to redaction.
The second difference is that the redaction set Ω may have a zero intersection with the training data, but may appear in the generated data due to artifacts of the model.
Examples include unnatural sounds emitted by speech models, and blurred images from image models.
Data redaction, in contrast to data deletion, can address this challenge.
4 samples / data support 0.
d e n s i t y p data p data | pre-trained density ( X ) re-trained density
( X X 0 ) deletion set Xremaining set X X
III. M ETHODS
In this section, we describe algorithms for each kind of redaction set described in Section II.
We also provide theory on the optimality of the generator and the discriminator.
Finally, we generalize the algorithms to situations where we would like the model to redact the union of multiple redaction sets.
A. Data-based Redaction Set
The data-based redaction set Ω is a pre-defined set of samples we would like the model to redact.
One example is a transformation function NegAug applied to all training samples, where NegAug makes realistic images unrealistic or inconsistent [Sinha et al., 2020].
Another example can be visually nice samples outside data manifold when the training set is small [Asokan and Seelamantula, 2020].
In our framework, the redaction set Ω can be any set of carefully designed or selected samples depending on the purpose of redacting them – which includes but does not limit to improving the generation quality of the model.
For example, we expect the model to improve on fairness, bias, ethics or privacy when Ω is properly constructed with unfair, biased, unethical, or atypical samples.
To redact Ω , we regard both generated samples and Ω to be fake samples, and all training samples that are not in Ω to be real samples [Sinha et al., 2020, Asokan and Seelamantula, 2020].
Let p Ω be a redaction distribution such that supp ( p Ω )
= Ω .
Then, the fake data distribution p fake is a mixture of the generative distribution p G and the redaction distribution p Ω :
[Equation]
where λ ∈ (0 , 1) is a hyperparameter.
We also apply the common label smoothing [Salimans et al., 2016, Szegedy et al., 2016, Warde-Farley and Goodfellow, 2016] technique to the minimax loss function in order to improve robustness of the discriminator.
Let α + ∈
( 12 , 1] be the positive target (such as 0 . 9 ) and α − ∈ [0 , 12 ) be the negative target (such as 0 . 1 ).
Then, the loss function is
[Equation]
Theorem 1.
The optimal solution to min G max D L ( G, D ) is
[Equation]
We provide the proof and theoretical extension to the more general f
-GAN [Nowozin et al., 2016] setting in Appendix A. In the data-based setting, we let p Ω =
U (Ω) , the uniform distribution on Ω .
We assume Ω has positive, finite Lebesgue measure in R d so that U (Ω) is well-defined.
The proposed method is summarized in Alg. 1.
Our objective function is connected to Sinha et al.
[2020] and Asokan and Seelamantula [2020] in the sense that p Ω is an instance of the negative distribution described in their frameworks.
However, there are several significant differences between our method and theirs: (1) we start from a pre-trained model, (2) we aim to learn p data | ¯Ω rather than p data and therefore do not require Ω ∩ supp ( p data ) to be the empty set, and (3) we consider the common label smoothing technique and provide theory for this setting.
These differences are also true in the following sections.
B. Validity-based Redaction Set
Let v : R d → { 0 , 1 } be a validity function that indicates whether a sample is valid.
Then, validitybased redaction set Ω is the set of all invalid samples { x : v ( x ) = 0 } .
For example, M is a code generation model, and v is a compiler that indicates whether the code is free of syntax errors [Hanneke et al., 2018].
Different from the data-based setting, the validity-based Ω may have infinite Lebesgue measure, such as a halfspace, and consequently U [-(Ω ′ ) . Let M i +
-](Ω) may not be well-defined.
To redact Ω , we let p Ω in (2) to be a mixture of p data | Ω and p G | Ω .
This corresponds to a simplified version of the improper active learning algorithm introduced by Hanneke et al.
[2018] with our Alg.
1 as their optimization oracle.
The idea is to apply
Alg. 1 for R rounds.
After each round, we query the validity of T newly generated samples and use invalid samples to form a data-based redaction set Ω ′ .
In contrast to the data-based approach, this active algorithm focuses on invalid samples that are more likely to be generated, and therefore efficiently penalizes generation of invalid samples.
The proposed method is summarized in Alg. 2.
The total number of queries to the validity function v is | X | + T × R .
In case v is expensive to run, we would like to achieve better data redaction within a limited number of queries.
From the datadriven point of view, we hope to collect as many invalid samples as possible.
This is done by setting
R = 1 and T maximized if we assume less invalid samples are generated after each iteration.
However, this may not be the case in practice.
We hypothesis some samples are easier to redact while others harder.
By setting R > 1 , we expect an increasing fraction of invalid generated samples to be hard to redact after each iteration.
Focusing on these hard samples can potentially help the generator redact them.
Since it is hard to directly analyze neural networks, we leave the rigorous study to future work.
In Appendix B, we study a much simplified dynamical system corresponding to Alg. 2, where we show the invalidity (the mass of p G on Ω ) converges to zero, and provide optimal T and R values.
C. Classifier-based Redaction Set
We would like the model to redact samples with certain (potentially undesirable) property.
Let f : R d → [0 , 1] be a soft binary classifier on the property (0 means having the property and 1 means not having it), and τ ∈ (0 , 1) be a threshold.
The classifier-based redaction set Ω is then defined as { x : f ( x ) < τ } .
For example, the property can be being offensive in language generation, containing no speech in speech synthesis, or visual inconsistency in image generation.
We consider f to be a trained machine learning model that is fully accessible and differentiable.
To redact Ω , we let p Ω be a mixture of p data | Ω and p G | Ω , similar to the validity-based approach.
We use f to guide the discriminator and make it able to easily detect samples from Ω .
Let guide ( D, f ) be a guided discriminator that assigns small values to x when f ( x ) <
τ or D ( x ) is small (i.e. x ∼ p fake ), and large values to x when f ( x ) >
τ and D ( x ) is large (i.e. x ∼ p data | ¯Ω ).
Instead of optimizing
L ( G, D ) in (3), we optimize L ( G, guide ( D, f )) .
This will effectively update G by preventing it from generating samples in Ω .
According to Theorem 1, the optimal discriminator is the solution to
[Equation]
Therefore, the design of the guide function must make (5) feasible.
In this paper, we let
[Equation]
The feasibility of (5) is discussed in Appendix C.
[-NVIDIA 
-]
The proposed method is summarized in Alg. 3.
The classifier-based Ω generalizes the validity-based Ω .
First, any validity-based Ω can be represented by a classifier-based Ω if we let f = v and τ = 12 .
Next, we note there is a trivial way to deal with classifierbased Ω via the validity-based approach – by setting
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , redaction set Ω .
[-0.8 Rmnist cifar
-]
Initialize G =
[-Then, ϕ (1) = 0 , and ϕ ′′ ( u ) = α (2 − α ) u ( αu − α +2) > 
-]
G 0 , D = D 0 .
Define the fake data distribution p fake according to (2) with p Ω =
[-When α − = 0 and α + = 
-]
[-25 for margin = 1 and 10 . 
-]
U (Ω) .
Train G, D to optimize (3): min G max D L ( G, D ) .
[-We extend their idea and theory to data forgetting in Section 3. 
-]
return M ′ =
( G, D ) .
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , validity function v .
Initialize Ω ′ =
{ x ∈ X : v ( x ) = 0 } , M 0 = M .
for i = 0 , · · · , R − 1 do
Initialize G
=
G i , D
= D i .
Draw T samples X ( i ) gen from G i .
Query v and add invalid samples to Ω ′ : Ω ′ ← Ω ′ ∪ { x ∈ X ( i ) gen : v ( x ) = 0 } .
Define the fake data distribution p fake according to (2) with p Ω =
U (Ω ′ ) .
Let M i +
=
( G i +1 , D i +1 ) optimize (3): min G max D L ( G, D ) .
end for return M ′ =
( G R , D R )
Algorithm
Inputs : Pre-trained model M = ( G 0 , D 0 ) , train set X , differentiable classifier f .
Initialize G =
G 0 , D = D 0 .
Define the fake data distribution p fake according to (2) with p Ω
= U ( { x ∈ X : f ( x ) < τ } ) .
Train G, D to optimize (3): min G max D L ( G, guide ( D, f )) , where guide ( · , · ) is defined in (6).
return M ′ =
( G, D ) .
v ( x ) = 1 { f ( x ) < τ } .
However, potentially useful information such as values and gradients of f are lost, and we will evaluate this effect in experiments.
In addition, the classifier-based approach does not maintain the potentially large set of invalid generated samples, as this step is automatically done in the guide function.
D. Generalization to Multiple Redaction Sets
Let { Ω k } Kk =1 be disjoint sets in R d , and we would like the model to redact Ω = (cid:83)
Kk =1 Ω k .
In the databased setting, we let p Ω =
U (Ω) =
U ( (cid:83)
Kk =1 Ω k ) .
In the validity-based setting, each Ω k is associated with a validity function v k .
We let the overall validity function to be v ( x ) =
min k v k ( x ) .
In the classifierbased setting, each Ω k is associated with a classifier f k .
Similar to the validity-based setting, we let the overall f to be f ( x ) =
min k f k ( x ) .
IV. E XPERIMENTS
In this section, we aim to answer the following questions.
• How well can the algorithms in Section III redact samples in practice?
• Can these algorithms be used to de-bias pretrained models?
• Can these algorithms be used to understand training data?
In this section, we examine these questions by focusing on several real-world image datasets, including
MNIST ( 28 × 28 )
[LeCun et al., 2010], CIFAR ( 32 × 32 )
[Krizhevsky et al., 2009], CelebA ( 64 × 64 )
[Liu et al., 2015] and STL-10 ( 96 × 96 )
[Coates et al., 2011] datasets.
In Section IV-A, we investigate how well these algorithms can redact samples with a specific label.
In Section IV-B, we investigate how well these algorithms can de-bias pre-trained models and improve generation quality.
In Section IV-C, we use these algorithms to understand training data through the lens of data redaction.
The pre-trained model for each dataset is a
DCGAN [Radford et al., 2015] trained forepochs (see details in Appendix D).
We use one NVIDIA {+
+} GPU to train these models and run experiments.
quality.
The invalidity is defined as the mass of the generation distribution on the redaction set Ω : Inv ( p G ) =
(cid:82) x ∈ Ω p G ( x ) dx .
In practice, we measure invalidity by generating 50K samples and computing the fraction of these samples that fall into Ω .
The generation quality is measured in Inception
Score ( IS ) [Salimans et al., 2016] and Frechet
Inception Distance ( FID )
[Heusel et al., 2017].
Higher IS or lower FID indicates better quality.
We compute IS for grey-scale images and FID for RGB images.
When measuring quality, we compute IS or FID between 50K generated samples and X ∩ ¯Ω .
Therefore, this score is not comparable with the score w.r.t.
the pre-trained model if the redaction set includes samples in the training set, such as samples with a specific label in Section IV-A. Detailed setup is in Appendix D.
A. Redacting Labels
Question.
How well can the algorithms in Section III redact samples in practice?
Methodology.
We investigate how well the proposed algorithms can redact samples with a specific label y .
In the data-based setting (Alg. 1), we express this as Ω = { x ∈ X : label ( x ) = y } .
In the validity-based setting (Alg. 2), we express this by setting v ( x ) = 1 { arg max i logit ( x ) i ̸
= y } , where logit is the output of the softmax layer of a pre-trained label classifier [Chen, 2020].
In the classifier-based setting (Alg. 3), we set f ( x ) = 1 − logit ( x ) y .
In Table I, we compare invalidity and generation quality among different algorithms and datasets when we redact label 0.
We plot invalidity during data redaction in Fig.
We also compare invalidity after one epoch of data redaction in Appendix E-A2.
Mean and standard errors for 5 random runs are reported.
Results for different hyper-parameters and redacting other labels are in Appendix E.
Results.
We find all the algorithms in Section III work quite well with a much fewer number of epochs used for training the pre-trained model (which is 200).
These algorithms are generally comparable.
Therefore, we conclude that the simplest data-based algorithm is good enough to redact samples when those training samples to be forgotten ( X ∩ Ω ) can characterize the redaction set ( Ω ) well.
We also find invalidity rapidly drops after only one epoch of data redaction, indicating these algorithms are very efficient in penalizing invalidity.
While different algorithms perform better on different datasets, they are highly comparable with each other.
The reason why the classifier-based algorithm performs the best on MNIST is possibly that the label classifier on MNIST is almost perfect so its gradient information is accurate.
Visualization.
We sample latents z ∼ N (0 , I ) and choose those corresponding to invalid samples, i.e. G 0 ( z ) ∈ Ω where G 0 is the pre-trained generator.
We select visually good G 0 ( z ) for demonstration.
We visualize G ( z ) during data redaction in Fig.
3, and more visualizations are in Appendix E-C.
This demonstrates how the latent space is manipulated: the label to be forgotten is gradually pushed to other labels, and there is high-level visual similarity between the final G ( z ) and the original G 0 ( z ) .
Effects of other hyper-parameters.
In Table II, we compare different T (#queries after each epoch) in the validity-based redaction algorithm (Alg.
We fix the total number of queries by setting
T ×
#epochs to be a constant.
Results indicate that a large T may lead to worse invalidity, and there is trade-off between invalidity and quality when setting T to be small or moderate.
In Appendix E-A4, we compare different λ (hyperparameter in (2)) in the classifier-based redaction algorithm (Alg. 3).
We find there exists a clear tradeoff between invalidity and quality when alternating λ : a larger λ tends to produce better quality, and a smaller λ tends to have better invalidity.
Redacting multiple sets.
We then investigate how well the proposed algorithms can generalize to multiple redaction sets with methods in Section III-D.
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] We focus on the CelebA dataset [Liu et al., 2015], which has 40 labeled attributes.
We use proposed algorithms to redact a combination of these attributes: Ω 1 = { Black_hair and Blurry } , Ω 2 = { Brown_hair and Wear_eyeglasses } , and Ω = Ω 1 ∪ Ω 2 .
These attributes are randomly selected from those easy to capture.
See detailed setup in
Appendix E-D. Results after 1 or 5 epochs are reported in Table III.
Consistent with results on redaction just one label, all algorithms can reduce invalidity and retain generation quality and are comparable, while the classifier-based algorithm achieves the best invalidity after one epoch.
B. Model De-biasing
There can be different artifacts in GAN generated samples, and these could harm the overall generation quality.
These artifacts may not exist in training samples, but are caused by inductive biases of the model, and become obvious after training.
We can post-edit a pre-trained model to remove these artifacts, which we call model de-biasing .
In this section, we investigate how well Alg. 2 and Alg.apply to this task.
We assume training samples are not biased so Alg.
1 does not apply to de-biasing.
To use these algorithms for de-biasing, we assume the target artifact or bias can be automatically detected by a classifier f or a validity function v . [Table]
Specifically, we survey two kinds of biases: boundary artifacts and label biases.
Boundary artifacts.
A GAN trained on MNIST might generate samples that have numerous white pixels on the boundary (see Appendix F-A).
We call this phenomenon the boundary artifact .
We use the validity-based algorithm (Alg. 2) to de-bias boundary artifacts.
The validity function is defined as v ( x ) = 1 { (cid:80) ( i,j ) ∈ boundary pixels x ij < τ b } , where boundary pixels are those within a certain margin to the boundary, and threshold τ b satisfies no training image is invalid.
Results are reported in Table IV.
It is clear that the invalidity reduces in order after data redaction, indicating boundary artifacts are largely removed.
Consistent with Table II, a small or moderate T leads to better results.
We visualize samples before and after de-biasing in Appendix F-A.
Label biases.
Neural networks may generate visually smooth but semantically ambiguous samples [Kirichenko et al., 2020], e.g. samples that look like multiple objects (see Appendix F-B).
We call this phenomenon the label bias .
We use the classifier-based algorithm (Alg. 3) to de-bias label biases.
The classifier is defined as f ( x ) =
−
Entropy( logit ( x )) / log (#classes) , where the logit function is the same as in Section
[-0.8 Rmnist cifar
-]IV-A. We also compare to a data deletion baseline, where we delete invalid samples and fully re-train the model.
Results are reported in Table V and VI.
After de-biasing, we can improve the generation quality by a significant gap ( ∼ 0 .
3 in IS and ∼in FID ).
There is also a clear drop in terms of invalidity.
In contrast, we find that data deletion does not help removing label biases.
C. Understanding Training Data through the Lens of Data Redaction
Large datasets can be hard to analyze.
In this section, we investigate how data redaction can help [Table]
[Table] [Table] us understand these data. Specifically, we ask: which samples are easy or hard to redact?
In order to quantify the difficulty to redact a sample, we define the redaction score FS to be the difference of discriminator outputs before and after data redaction.
Formally, let x ∈ Ω be a sample to redact, M =
( G 0 , D 0 ) be the pretrained model, and M ′ =
( G ′ , D ′ ) be a model after data redaction.
Then, the redaction score is FS ( x ) =
D 0 ( x ) − D ′
( x ) .
A larger FS means it is easier to redact x .
To investigate sample-level redaction difficulty, we redact a particular label at one time using Alg. 1.
We then demonstrate scatter plots of redaction scores FS ( x ) versus pre-trained discriminator outputs D 0 ( x ) for all samples x with this label.
We also fit linear regression and report R 2 values (larger means stronger linear relationship).
Scatter plots for some labels in MNIST and CIFAR-100 and distribution of R 2 for all labels are shown in Fig.
We also visualize the most and least difficultto-redact samples in Appendix G.
We find there is positive correlation between FS ( x ) and D 0 ( x ) , indicating on-manifold (large D 0 ( x ) ) samples are easier to be forgotten, while off-manifold (small D 0 ( x ) )
ones are harder to be forgotten.
This analysis further provides a way to investigate label-level redaction difficulty.
By averaging redaction scores for samples associated with each label, we can survey which labels are easy or hard to redact in general.
The results are in Appendix G.
We find some labels are harder to redact than others.
0.34 forgetting score 0.44 0. p r e t r a i n e d D o u t p u t linear reg ( R 2 =0.41) (a) MNIST (0) 0. 0.9 forgetting score 0. p r e t r a i n e d D o u t p u t linear reg ( R 2 =0.71) (
[Figure] [Figure] aquarium [Figure]
fish ) {+0. 0.8 Rmnist cifar
 d a t a s e t [Figure]+}
An adversarial sample for a classifier f and a sample x is another sample ˜ x ≈ x but f (˜ x ) ̸ = f ( x ) .
Generating and defending these samples have become one of the most important directions of deep learning [Goodfellow et al., 2014b, Madry et al., 2018].
In this section, we show a variant of Alg.can potentially be used to define a specific type of adversarial samples.
In detail, we fix the discriminator D and only update the generator G while running Alg. 3.
Then, G is trained to fool D and the classifier f at the same time.
Notice that fooling D means generating on-manifold (visually similar to training data) samples, and fooling f means finding adversarial samples of f .
By combining these objectives we can force G to produce on-manifold adversarial samples, which may be significant in many real-world applications. We visualize some samples in Appendix H. V. R ELATED W ORK
Although deep generative models have been highly successful at many domains, it has long been known that they often emit undesirable samples and samples with different types of artifacts that make them untrustworthy. Examples include blurred image samples [Kaneko and Harada, 2021], fairness issues [Tan et al., 2020, Karakas et al., 2022], and checkerboard artifacts [Odena et al., 2016, Zhang et al., 2019, Wang et al., 2020, Schwarz et al., 2021] in image generation, offensive text in language models [Abid et al., 2021, Perez et al., 2022], and unnatural sound in speech models [Donahue et al., 2018, Thiem et al., 2020]. Some prior works have used post-editing to remove artifacts and improve GANs. Examples include improving fairness [Tan et al., 2020, Karakas et al., 2022], rule rewriting [Bau et al., 2020], discovering interpretability [Härkönen et al., 2020], and fine-tuning [Mo et al., 2020, Li et al., 2020,
Zhao et al., 2020]. The purpose, use cases, and editing methods of these papers are different from our paper, where we focus on data redaction. While our problem definition and formalization is novel, the technical solutions that we propose are related to three prior works that use these techniques in different contexts. These are NDA [Sinha et al., 2020], Rumi-GAN [Asokan and Seelamantula, 2020], and Hanneke et al.
The first two papers look at how to avoid generating negative samples while training a generative model from scratch.
This is done by defining new fake distributions to penalize the generation of these samples.
However, their purposes are different from us: NDA is used to characterize the boundary of the support of the generative distribution more precisely, and Rumi-GAN is used to handle unbalanced data.
We extend their idea and theory to data redaction in Section III.
1 Hanneke et al.
[2018] propose an active learning approach to avoid generating invalid samples, also while training a generative model from scratch.
Their work however is entirely theoretical and apply to discrete distributions.
In our paper, the validity-based redaction algorithm (Alg. 2) is based on a simplified version of their algorithm.
We also use their definition of invalidity as an evaluation method.
Our work is also related to data deletion or machine unlearning [Cao and Yang, 2015, Guo et al., 2019, Schelter, 2020, Neel et al., 2021, Sekhari et al., 2021, Izzo et al., 2021, Ullah et al., 2021,
Bourtoule et al., 2021]. However, there are two important differences between data deletion and data redaction. First, data deletion aims to approximate the re-trained model when some training samples are removed – mostly due to privacy reasons – while in data redaction we penalize the model from knowing samples that should be forgotten. Another difference is that in data redaction, the redaction set Ω may have a zero intersection with training data.
These two differences are discussed in Section II-C in detail. In addition, most data deletion techniques are for supervised learning or clustering, and is much less studied for generative models. There is also a related line of work on catastrophic forgetting in supervised learning [Kirkpatrick et al., 2017] and generative models [Thanh-Tung and Tran, 2020].
This concept is different from data redaction in that we would like the generative model to redact certain data after training, while catastrophic forgetting means knowledge learned in previous tasks is destroyed during continual learning.
VI. C ONCLUSION
In this paper, we propose a systematic framework for redacting data from pre-trained generative models.
We provide three different algorithms for GANs that differ on how the samples to be forgotten are described.
We provide theoretical results that data redaction can be achieved.
We then empirically investigate data redaction on real-world image datasets, and show that our algorithms are capable of redacting data while retaining high generation quality at a fraction of the cost of full re-training.
One limitation or our paper is that the proposed framework only applies to unconditional generative models.
It is an important future direction to define data redaction and propose algorithms for conditional generative models, which are more widely used in downstream deep learning applications.