D ECENTRALIZED A TTRIBUTION OF G ENERATIVE M ODELS
Anonymous authors
Paper under double-blind review
A BSTRACT
Growing applications of generative models have led to new threats such as malicious personation and digital copyright infringement.
One solution to these threats is model attribution, i.e., the identiﬁcation of user-end models where the contents under question are generated.
Existing studies showed empirical feasibility of attribution through a centralized classiﬁer trained on all existing user-end models.
However, this approach is not scalable in reality where the number of models ever grows.
Neither does it provide an attributability guarantee.
To this end, this paper studies decentralized attribution, which relies on binary classiﬁers associated with each user-end model.
Each binary classiﬁer is parameterized by a user-speciﬁc key and distinguishes its associated model distribution from the authentic data distribution.
We develop sufﬁcient conditions of the keys that guarantee an attributability lower bound.
Our method is validated on MNIST, CelebA, and FFHQ datasets.
We also examine the trade-off between generation quality and robustness of attribution against adversarial post-processes.
I NTRODUCTION
Recent advances in generative models (Goodfellow et al., 2014) have enabled the creation of synthetic contents that are indistinguishable even by naked eyes (Pathak et al., 2016; Zhu et al., 2017; Zhang et al., 2017;
Karras et al., 2017; Wang et al., 2018;
Brock et al., 2018;
Miyato et al., 2018; Choi et al., 2018; Karras et al., 2019a;b; Choi et al., 2019).
Such successes raised serious concerns (Kelly, 2019;
Breland, 2019) regarding emerging threats due to the applications of generative models.
This paper is concerned about two particular types of threats, namely, malicious personation (Satter, 2019) , and digital copyright infringement.
In the former, the attacker uses generative models to create and disseminate inappropriate or illegal contents; in the latter, the attacker steals the ownership of a copyrighted content (e.g., an art piece created through the assistance of a generative model) by making modiﬁcations to it.
These threats share the same solution, namely, model attribution, i.e., the identiﬁcation of user-end models where the contents under question are generated.
Existing studies showed empirical feasibility of attribution through a centralized classiﬁer trained on all existing user-end models (Yu et al., 2018).
However, this approach is not scalable in reality where the number of models ever grows.
Neither does it provide an attributability guarantee.
To this end, we study in this paper a decentralized attribution scheme: Instead of a centralized classiﬁer, we use a set of binary linear classiﬁers associated with each user-end model.
Each classiﬁer is parameterized by a user-speciﬁc key and distinguishes its associated model distribution from the authentic data distribution.
For correct at- tribution, we expect one-hot classiﬁcation outcomes for generated contents, and zero vectors for authentic data.
To achieve correct attribution, we study the sufﬁcient conditions of the user-speciﬁc keys that guarantee an attributability lower bound.
The resultant conditions are used to develop an algorithm for generating new keys.
Lastly, we assume that attackers can post-process generated contents to potentially deny the attribution, and study the tradeoff between generation quality and robustness of attribution against post-processes.
Problem formulation We assume that for a given dataset D ⊂ R d x , the registry generates userspeciﬁc keys , Φ :=
{ φ 1 , φ 2 , ... } where φ i ∈ R
d x and || φ i ||
= 1 .
|| · || is the l 2 norm.
A user-end generative model is denoted by G φ ( · ; θ ) : R d z → R d x where z and x are the latent and output variables, respectively, and θ are the model parameters.
When necessary, we will suppress θ and φ to reduce the notational burden.
The distribution of each key is accompanied by that of a public service that tells whether a query content belongs to G φ (labeled as 1 ) or not (labeled as − 1 ).
We model the underlying binary linear classiﬁer as f φ ( x ) = sign ( φ T x ) .
Note that linear models are necessary for the development of sufﬁcient conditions of attribution, and tests on benchmark datasets suggest that linear classiﬁers achieve good attribution performance.
The following quantities are central to our investigation: the distinguishability of G φ measures the accuracy of f φ ( x ) at classifying G φ against D :
[Equation]
Here P D is the authentic data distribution, and P G φ the model distribution.
The attributability measures the averaged multi-class classiﬁcation accuracy of each model distribution over the collection
G :
[Equation]
Distinguishability of G (attributability of G ) is achieved when D ( G ) ≥ 1 − δ
( A ( G ) ≥ 1 − δ ) for small δ > 0 .
Lastly, We denote by G ( · ; θ 0 ) (or shortened as G 0 )
the root model trained on D , and assume P G 0 = P D .
We will measure the (lack of) generation quality of G φ by the FID score (Heusel et al., 2017) and the l 2 norm of the mean output perturbation:
[Equation]
where P z is the latent distribution.
This paper investigates the following question: What are the sufﬁcient conditions of keys so that the resultant generative models can achieve distinguishability individually and attributability collectively, while maintaining their generation quality?
Contributions We claim the following contributions:
We develop sufﬁcient conditions of keys for distinguishability and attributability, which connect these metrics with the geometry of the data distribution, the angles between keys, and the generation quality.
The sufﬁcient conditions lead to simple design rules for the keys: keys should be data compliant,
i
e., φ T x < 0 for x ∼ P D , and orthogonal to each other.
We validate these rules using DCGAN (Radford et al., 2015) and StyleGAN (Karras et al., 2019a) on benchmark datasets including MNIST (LeCun & Cortes, 2010), CelebA (Liu et al., 2015)
and FFHQ (Karras et al., 2019a).
See Fig.
1 for a visualization of the attributable distributions perturbed from the authentic FFHQ dataset.
We empirically test the tradeoff between generation quality and robust attributability under random post-processes including image blurring, cropping, noising, JPEG conversion
and a combination of all.
S UFFICIENT C ONDITIONS FOR A TTRIBUTABILITY
From the deﬁnitions (Eq.
(1) and Eq.
(2)), achieving distinguishability is necessary for attributability.
In the following, we ﬁrst develop the sufﬁcient conditions for distinguishability through Proposition 1 and Theorem 1, and then those for attributability through Theorem 2.
Distinguishability through watermarking First, consider constructing a user-end model G φ by simply adding a perturbation ∆ x to the outputs of the root model G 0 .
Assuming that φ is datacompliant, this model can achieve distinguishability by solving the following problem with respect to ∆ x :
[Equation]
where ε > 0 represents a generation quality constraint.
We have the following proposition that reveals the connection between distinguishability, data geometry, and generation quality (proof in
Appendix A):
Proposition 1.
Let d max ( φ ) :
= max x ∼ P D | φ T x | .
If ε ≥ d max ( φ ) , then ∆ x ∗ = d max ( φ ) φ solves
Eq.
(4), and f φ ( x + ∆ x ∗ ) > 0 , ∀ x ∼ P D .
Watermarking through retraining user-end models The perturbation ∆ x ∗ can potentially be reverse engineered and removed when generative models are white-box to users (e.g., when models are downloaded by users).
Therefore, we propose to instead retrain the user-end models G φ using the perturbed dataset D γ,φ :=
{ G 0 ( z )+ γφ | z ∼ P z } with γ > 0 , so that the perturbation is realized through the model architecture and weights.
Speciﬁcally, the retraining ﬁne-tunes G 0 so that G φ ( z ) matches with G 0 ( z ) + γφ for z ∼ P z .
Since this matching will not be perfect, we use the following model to characterize the resultant G φ :
[Equation]
where the error (cid:15) ∼ N ( µ, Σ) .
In Sec.
3 we provide statistics of µ and Σ on the benchmark datasets, to show that the retraining captures the perturbations well (i.e., µ close to 0 and small variances in Σ ).
Proposition 1 no longer applies due to the existence of error.
As a result, we show in Theoremthat γ needs to be no smaller than d max ( φ ) in order for G φ to achieve distinguishability (proof in
Appendix B):
Theorem 1.
Let d max ( φ ) =
max x ∈D | φ T x | , σ 2 ( φ ) =
φ T Σ φ , δ ∈ [0 , 1] , and φ be a data-compliant key.
[Equation]
Remarks The computation of σ ( φ ) requires G φ , which in turn requires γ .
Therefore, an iterative search will be needed to determine γ that is small enough to limit the loss of generation quality, yet large enough for distinguishability (see Alg. 1).
Attributability We can now derive the sufﬁcient conditions for attributability of the generative models from a set of N keys (proof in Appendix C):
Theorem 2.
Let d min = min x ∈D
| φ T x | , d max = max x ∈D |
φ T x | , σ 2 ( φ ) =
φ T Σ φ , δ ∈ [0 , 1] .
Let
[Equation]
for keys φ and φ (cid:48) .
Then A ( G ) ≥ 1 −
Nδ , if D ( G ) ≥ 1 − δ for all G φ ∈ G , and
[Equation] [Equation]
for any pair of data-compliant keys φ and φ (cid:48) .
Remarks When σ ( φ (cid:48) ) is negligible for all φ (cid:48) and µ = 0 , a ( φ, φ (cid:48) ) is approximately d min ( φ (cid:48) )
/d max ( φ (cid:48) ) > 0 , in which case φ T φ (cid:48) ≤ 0 is sufﬁcient for attributability.
In Sec.
3 we empirically show that this approximation is plausible for the benchmark datasets.
E XPERIMENTS AND A NALYSIS
In this section we test Theorem 1, provide empirical support for the orthogonality of keys, and present experimental results on model attribution using MNIST, CelebA, and FFHQ.
Note that tests on the theorems require estimation of Σ , which is costly for models with high-dimensional outputs, and therefore are only performed on MNNIST and CelebA.
Key generation We generate keys by iteratively solving the following convex problem:
[Equation]
The orthogonality penalty is omitted for the ﬁrst key.
The solutions are normalized to unit l 2 norm before being inserted into the next problem.
We note that P D and P G 0 do not perfectly match in practice, and therefore we draw with equal chance from both distributions during the computation.
G 0 s are trained using the standard DCGAN architecture for MNIST and CelebA, and StyleGAN for
FFHQ.
Training details are deferred to Appendix D.
User-end generative models The training of G φ follows Alg.
1, where γ is iteratively tuned to balance generation quality and distinguishability.
For each γ , we collect a perturbed dataset D γ,φ and solve the following training problem:
[Equation]
starting from θ = θ 0 .
If the resultant model does not meet the distinguishability requirement due to the discrepancy between D γ,φ and G φ , the perturbation is updated as γ = αγ .
In experiments, we use a standard normal distribution for P z , and set δ =
Validation of Theorem
Here we validate the sufﬁcient condition for distinguishability.
Fig.
2a compares the LHS and RHS values of Eq.
(6) for 100 distinguishable user-end models.
The empirical distinguishability of these models are reported in Fig.
2e.
Calculation of the RHS of Eq.
(6) requires an estimation of µ and Σ .
To do this, we sample
[Equation]
) using 5000 samples of z ∼ P z , where G φ and γ are derived from Alg. 1. Σ and µ are then estimated for each φ .
Fig.
2c shows the statistics of µ and Σ for sampled user-end models for the benchmark datasets.
Results in Fig.
2 show that the sufﬁcient condition for distinguishability (Eq.
(6)) is satisﬁed for most of the sampled models through the training speciﬁed in Alg.
Lastly, we notice that the LHS values for MNIST are farther away from the equality line than those for CelebA.
This is because the MNIST data distribution resides at corners of the unit box.
Therefore perturbations of the distribution are more likely to exceed the bounds for pixel values.
Clamping of these invalid pixel values reduces the effective perturbation length.
Therefore to achieve distinguishability, Alg.seeks γ s larger than needed.
This issue is less observed in CelebA, where data points are rarely close to the boundaries.
input : φ , Goutput: G φ , γ 1 set γ = d max ( φ ) ; 2 collect D γ,φ ; 3 train G φ by solving Eq.
(10) using D γ,φ ; 4 compute empirical D ( G φ ) ; 5 if D ( G φ ) < 1 − δ then set γ = αγ ; goto step 2 ;
8 end
Validation of Theorem 2 Recall that from Theorem 2, we recognized that orthogonal keys are sufﬁcient.
To support this design rule, Fig.
2b presents the minimum RHS values of Eq.
(8) for 100 user-end models.
Speciﬁcally, for each φ i , we compute a ( φ i , φ j ) (Eq.
(7)) using φ j for j = 1 , ..., i − 1 and report min j a ( φ i , φ j ) , which sets a lower bound on the angle between φ i and all existing φ s. The resultant min
j a ( φ i , φ j ) are all positive for MNIST and close to zero for CelebA.
It should be noted that in the case of CelebA, an angle of ≥ 94 deg , instead of 90 deg , should be enforced between any pairs of keys.
However, since the conditions are sufﬁcient, orthogonal keys still empirically achieve high attributability (Fig. 2f), although improvements can be made by further increasing the angle between keys.
Also notice that Eq.
(9) does not enforce a hard constraint on orthogonality, leading to slightly acute angles ( 87 .
7 deg ) between keys for CelebA (Fig. 2h).
On the other hand, the positive values in Fig.
2b for MNIST suggests that further reducing the angles between keys is acceptable if one needs to increase the total capacity of attributable models.
However, doing so would require the derivation of new keys to rely on knowledge about all existing user-end models (for computing Eq.
[Table]
Empirical results on benchmark datasets Tab. 1 reports the metrics of interest measured on the 100 user-end models for each of MNIST and CelebA, and 20 models for FFHQ.
All models are trained to be distinguishable, and through Theorem 2, also achieve high attributability.
As a comparison, we demonstrate results where keys are 45 deg apart ( φ T φ = 0 .
71 ) using a separate set of 20 user-end models for each of MNIST and CelebA, and 5 models for FFHQ, in which case distinguishability no longer guarantees attributability.
Regarding generation quality, G φ s receive worse FID scores than G 0 due to the perturbations.
We visualize samples from user-end models and the corresponding keys in Fig.
Note that for human faces, FFHQ in particular, the perturbations create light shades around eyes and lips, which is an unexpected but reasonable result from Eq.
Robust training We now consider the scenario where outputs of the generative models are postprocessed (e.g., by adversaries) before being attributed.
When the post-processes are known, we can take counter measures through robust training, which intuitively, will lead to additional loss of generation quality.
To assess this tradeoff between robustness and generation quality, we train G φ against post-processes T : R d x → R d x from a distribution P T .
Due to the potential nonlinearity of
T and the lack of theoretical guarantee in this scenario, we resort to the following robust training problem:
[Equation]
where C is the hyper-parameter for generation quality.
Detailed analysis and comparison to select C are provided in
Appendix E. We consider ﬁve types of post-processes: blurring, crop- ping, noise, JPEG conversion and the combination of these four.
Examples of the post-processed images are shown in Fig.
Blurring uses Gaussian kernel widths uniformly drawn from 13 { 1 , 3 , 5 , 7 , 9 } .
Cropping crops images with uniformly drawn ratios between 80% and 100% , and scales the cropped images back to the original size using bilinear interpolation. Noise adds white noise with standard deviation uniformly drawn from [0 , 0 . 3] . JPEG applies JPEG compression.
Combination performs each attack with a 50% chance in the order of Blurring , Cropping ,
Noise and JPEG .
For differentiability, we use existing implementations of differentiable blurring (Riba et al.
and JPEG (Zhu et al.
For robust training, we apply the post-process to mini-batches with 50% probability.
We performed comprehensive tests using DCGAN (on MNIST and CelebA), PGAN (on CelebA), and CycleGAN (on Cityscapes).
Tab.
2 summarizes the average distinguishability, the attributability, || ∆ x || , and the FID scores with and without robust training of G φ .
Results are based onmodels for each architecture-dataset pair, where keys are kept orthogonal and data compliant.
From the results, defense against these post-processes can be achieved, except for Combination .
In addition, there is a clear tradeoff between robustness and generation quality.
To support this, Fig.compares samples with and without robust training from the tested models and datasets.
Lastly, it is worth noting that the training formulation in Eq.
(12) can also be applied to the training of non-robust user-end models in place of Eq.
However, the resultant model from Eq.
(12) cannot be characterized by Eq.
(5) with small µ and Σ , i.e., due to the nonlinearity of the training process, the user-end model distribution is deformed while it is perturbed.
This resulted in unsuccessful validation of the theorems, which led to the adoption of Eq.
(10) for theorem-consistent training.
Therefore, while the empirical tests here show feasibility of achieving robust attributability, counterparts to Theorem 1 and 2 in this context are yet to be developed.
D ISCUSSION
Generation Quality Control From Proposition 1 and Theorem 1, the inevitable loss of generation quality is directly related to the length of perturbation ( γ ), which is related to d max .
Fig.compares outputs from user-end models with different d max s. While it is possible to ﬁlter φ s based on their corresponding d max s for generation quality control, here we discuss a potential direction for prescribing a subspace of φ s within which quality can be controlled.
To start, we denote by
J ( x ) the Jacobian of G 0 with respect to its generator parameters θ 0 .
Our discussion is related to the matrix M =
E x ∼ P G
[ J ( x )] E x ∼ P G
[ J ( x ) T ] .
A spectral analysis of M reveals that the eigenvectors of M with large eigenvalues are more structured than those with small ones (Fig. 6(a)).
This ﬁnding is consistent with the deﬁnition of M : The largest eigenvectors of M represent the principal axes [Table] of all mean sensitivity vectors, where the mean is taken over the latent space.
For MNIST, these eigenvectors overlap with the digits; for CelebA, they are structured color patterns.
On the other hand, the smallest eigenvectors represent directions rarely covered by the sensitivity vectors, thus resembling random noise.
Based on this ﬁnding [Table] we test the hypothesis that keys more aligned with the eigenspace of the small eigenvalues will have smaller d max .
We test this hypothesis by computing the Pearson correlations between d max and φ T Mφ using 100 models for each of MNIST and
CelebA. The resultant correlations are 0.33 and 0.53, respectively.
In addition, we compare outputs from models using the largest and the smallest eigenvectors of M as the keys in Fig.
While a concrete human study is needed, the visual results suggest that using eigenvectors of M is a promising approach to segmenting the space of keys according to their induced generation quality.
Capacity of keys For real-world applications, we hope to maintain attributability for a large set of keys.
Our study so far suggests that the capacity of keys is constrained by the data compliance and orthogonality requirements.
While the empirical study showed the feasibility of computing keys through
Eq.
(9), ﬁnding the maximum number of feasible keys is a problem about optimal sphere packing on a segment of the unit sphere (Fig. 7).
To explain, the unit sphere represents the identiﬁability requirement
|| φ ||
= 1 .
The feasible segment of the unit sphere is determined by the data compliance and generation quality constraints.
And the spheres to be packed have radii following the sufﬁcient condition in Theorem 2.
Such optimal packing problems are known open challenges (Cohn et al.
(2017); Cohn (2016)).
For real-world applications where a capacity of attributable models is needed (which is the case for both malicious personation and copyright infringement settings), it is necessary to ﬁnd approximated solutions to the optimal packing problem.
R ELATED W ORK
Detection and attribution of model-generated contents This paper focused on the attribution of contents from generative models rather than the detection of hand-crafted manipulations (Agarwal & Farid (2017); Popescu & Farid (2005); O’brien & Farid (2012); Rao & Ni (2016); Huh et al.
Detection methods rely on ﬁngerprints intrinsic to generative models Odena et al.
Zhang et al. (2019b); Marra et al.
; Wang et al.
(2019), yet similar ﬁngerprints present for models trained on similar datasets Marra et al.
Thus ﬁngerprints cannot be used for attribution.
Skripniuk et al.
(2020) studied decentralized attribution.
Instead of using linear classiﬁers for attribution, they train a watermark encoder-decoder network that embeds (and reads) watermarks into (and from) the content, and compare the decoded watermark with user-speciﬁc ones.
Their method does not provide sufﬁcient conditions of the watermarks for attributability.
IP protection of digital contents and models Watermarks have conventionally been used for IP protection (Tirkel et al., 1993;
Van Schyndel et al., 1994; Bi et al., 2007; Hsieh et al., 2001; Pereira & Pun, 2000; Zhu et al., 2018; Zhang et al., 2019a) without considering the attribution guarantee.
Another approach to content IP protection is blockchain (Hasan & Salah, 2019).
However, this approach requires meta data to be transferred along with the contents, which may not be realistic in adversarial settings.
E.g., one can simply take a picture of a synthetic image to remove any meta data attached to the image ﬁle.
Aside from the protection of contents , mechanisms for protecting
IP of models have also been studied (Uchida et al., 2017;
Nagai et al., 2018; Le Merrer et al., 2019;
Adi et al., 2018; Zhang et al., 2018; Fan et al., 2019; Szyller et al., 2019; Zhang et al., 2020).
Model watermarking is usually done by adding watermarks into model weights (Uchida et al., 2017;
Nagai et al., 2018), by embedding unique input-output mapping into the model (Le Merrer et al., 2019; Adi et al., 2018; Zhang et al., 2018), or by introducing a passport mechanism so that model accuracy drops if the right passport is not inserted (Fan et al., 2019).
While closely related, existing work on model IP protection focused on the distinguishability of individual models, rather than the attributability of a model set.
Adversarial attacks Lastly, while the robust training process against post-processes is related to the adversarial attack setting (edy et al., 2014; Goodfellow et al., 2015; Madry et al., 2017), this study deﬁnes the robustness as the mean attribution accuracy under a distribution of post-processes rather than the worst accuracy.
C ONCLUSION
Motivated by emerging challenges with generative models, e.g., deepfake, this paper investigated the feasibility of decentralized attribution of such models.
The study is based on a protocol where the registry generates user-speciﬁc keys that guides the watermarking of user-end models to be distinguishable from the authentic data.
The outputs of user-end models will then be attributed by the registry through the binary classiﬁers parameterized by the keys.
We developed sufﬁcient conditions of the keys so that distinguishable user-end models achieve guaranteed attributability.
These conditions led to simple rules for designing the keys.
With concerns about adversarial postprocesses, we further showed that robust attribution can be achieved using the same design rules, and with additional loss of generation quality.
Lastly, we introduced two open challenges towards real-world applications of the proposed attribution scheme: the prescription of the key space with controlled generation quality, and the approximation of the capacity of keys.