Decentralized Attribution of Generative Models
Authors: Changhoon Kim
Yi Ren
Yezhou Yang
Venue: ICLR 2021 Poster (2021)
Keywords: GANs
Generative Model
Deepfake
Model Attribution

Abstract: Growing applications of generative models have led to new threats such as malicious personation and digital copyright infringement. 
One solution to these threats is model attribution, i.e., the identification of user-end models where the contents under question are generated.
Existing studies showed empirical feasibility of attribution through a centralized classifier trained on all existing user-end models. 
However, this approach is not scalable in a reality where the number of models ever grows. Neither does it provide an attributability guarantee.
To this end, this paper studies decentralized attribution, which relies on binary classifiers associated with each user-end model. 
Each binary classifier is parameterized by a user-specific key and distinguishes its associated model distribution from the authentic data distribution. 
We develop sufficient conditions of the keys that guarantee an attributability lower bound.
Our method is validated on MNIST, CelebA, and FFHQ datasets. We also examine the trade-off between generation quality and robustness of attribution against adversarial post-processes.

Published as a conference paper at ICLR 2021
DECENTRALIZED ATTRIBUTION OF GENERATIVE MODELS
Changhoon Kim Yi Ren Yezhou Yang'
School of Computing, Informatics, and Decision Systems Engineering' School for Engineering of Matter, Transport, and Energy² Arizona State University (kch, yiren, yz. yz.yang) (as.yangggsuueeeee..
ABSTRACT
Growing applications of generative models have led to new threats such as mali- cious personation and digital copyright infringement. One solution to these threats is model attribution, i.c., the identification user-end models where the contents under question generated from. Existing studies showed empirical feasibility attribution through a centralized classifier trained on all user-end models. How- ever, this approach is scalable in reality the number of models ever grows. Neither does it provide an attributability guarantee. To this end, this paper studies decentralized attribution, which relies on binary classifiers associated with each user-end model. Each binary classifier is parameterized by user-specific key and distinguishes its associated model distribution from the authentic data distribution. We develop sufficient conditions of the keys that guarantee an attributability lower bound. Our method is validated on MNIST, CelebA, and FFHQ datasets. We also examine the trade-off between generation quality and robustness of attribution against adversarial post-processes)
INTRODUCTION
Recent advances in generative models (Good-
fellow et al., 2014) have enabled the creation of synthetic contents that are indistinguish-
able even by naked eyes (Pathak et al.
2016
Zhu al. 2017
Zhang et al. 2017
Kar-
02
ras et al.
2017: Wang et al. 2018 Brock
al.) 2018 Miyato et al. 2018 Choi al.
2018
Karras et al. 2019a by Choi et 2019).
Such successes raised serious concerns regard- ing emerging threats due the applications of generative models (Kelly) 2019; Breland 2019). This paper is concerned about two particular types of threats, namely, malicious personation (Satter, 2019) and digital copy- right infringement. In the former, the attacker uses generative models to create and dissem- inate inappropriate or illegal contents; in the latter, the attacker steals the ownership of copyrighted content (e.g., an art piece created through the assistance of a generative model) by making modifications to it.
(a)
(b)
Figure 1: FFHQ dataset projected to the space spanned by two keys Φ1 and Φ2. We develop suf- ficient conditions for model attribution: Perturb- ing the authentic dataset along different keys with mutual angles larger than data-dependent thresh- old guarantees attributability of the perturbed dis- tributions. (a) threshold of deg suffices for benchmark datasets (MNIST, CelebA, FFHQ). (b) Smaller angles may not guarantee attributability.
We study model attribution, solution that may address both threats. Model attribution defined as the identification of user-end models where the contents under question are generated from. Existing
Equual contribution.
thub. com htt.:///.tt.............eeeeeeer
attribur innttiinn_eee lttrooooooooaaaaooaaaaeeeee gww.rat 1e

---

Published as a conference paper at ICLR 2021
studies demonstrated empirical feasibility of attribution through centralized classifier trained on all existing user-end models (Yu et al.] 2018). However, this approach is not scalable in reality where the number of models ever grows. Neither does it provide an attributability guarantee. To this end, we propose in this paper a decentralized attribution scheme: Instead of a centralized classifier, set of binary linear classifiers associated with each user-end model. Each classifier is parameterized by a user-specific key and distinguishes its associated model distribution from the authentic data distribution. For correct attribution, we expect one-hot classification outcomes for generated contents, and a zero vector for authentic data. To achieve correct attribution, we study the sufficient conditions of the user-specific keys that guarantee an attributability lower bound. The resultant conditions are used to develop an algorithm for computing the keys. Lastly, we assume that attackers can post-process generated contents to potentially deny the attribution, and study the tradeoff between generation quality and robustness of attribution against post-processes.
Problem formulation We assume that for given dataset D C RRd,, the registry generates user- specific keys, ① ;= {0i, Ø2, 22,... ...} where o; E Rd and || = 1. |I-I1 is the l2 norm. A user-end generative model is denoted by Go(:;)) variables, respectively, and 0 are the model parameters. When necessary, we will suppress 0 and 6 to reduce the notational burden. The dissemination of the user-end models is accompanied by a public service that tells whether query content belongs to G (labeled as 1) or not (labeled -1). We model the underlying binary linear classifier as f (z) = sign( z). Note that linear models are necessary for the development of sufficient conditions of attribution presented in this paper, although sufficient conditions for nonlinear classifiers worth exploring in the future.
Rdd + IR where = and æ are the latent and output
The following quantities are central to our investigation: (1) Distinguiishabiiity of G measures the accuracy of fa) at classifying Go against D:
1 2
D(G&) == -EE-Po Po Pob (1(((x)) = 1) ((f((z)) = -1]]-
(1)
Here Pp is the authentic data distribution, and Pc. the user-end distribution dependent on Φ. G is (1 distiiningueeeeeeee for some 8 (0, 1] when D(G) 2 1 1-8. 5. (2) Attributability measures
the averaged multi-class classification accuracy of each model distribution over the collection G :=
{G Gay]:
1 N Σ Ea-G. 1 x < 0 x 2, 0).
A(G)
G is (1 8) -tttrbbbeee when A(G) 2 - 8. (3) Lastly, We denote by G(-; 0o) (or shortened as Go) the root model trained on D, and assume Pa. Pp. We will measure the (lack of) generation quality of G& by the FID score (Heusel et al. 2017 and the l2 norm of the mean output perturbation:
Az(ø) = Es (Z; - G(2;00)).
(3)
where Pz is the latent distribution.
This paper investigates the following question: What are the sufficient conditions of keys so that the user-end generative models can achieve distinguishability individually and attributability col- lectively, while maintaining their generation quality?
Contributions
We claim the following contributions:
1. We develop sufficient conditions of keys for distinguishability and attributability, which connect these metrics with the geometry of the data distribution, the angles between keys, and the generation quality.
2. The sufficient conditions lead to simple design rules for the keys: keys should be (1) data compliant, i.e., o² x < for
Pp. and (2) orthogonal to each other. We validate these rules using DCGAN (Radford et al. 2015) and StyleGAN (Karras et al.] 2019a) on benchmark datasets including MNIST (LeCun 2015), and FFHQ (Karras et al. 2019a). See Fig. [ a visualization of the attributable distributions perturbed from the authentic FFHQ dataset.
Cortes) 2010), CelebA (Liu et al.)
3. We empirically test the tradcoff between generation quality and robust attributability under including image blurring, cropping, noising, JPEG conversion, and combination of all.
random
2

---

Published as a conference paper at ICLR 2021
2 SUFFICIENT CONDITIONS FOR AFTRIBUTABILITY
From the definitions (Eq. T and Eq. 2), achieving distinguishability is necessary for attributabilityy In the following, we first develop the sufficient conditions for distinguishability through Proposition and Theorem 1, and then those for attributability through Theorem
Distinguishability through watermarking First, consider constructing user-end model G by simply adding a perturbation Ar to the outputs of the root model Gp- Assuming that & is data- compliant, this model can achieve distinguishability by solving the following problem with respect to Ar:
min || S
Ea~Po
[max[1 o² (x O1)
where E >> 0 represents generation quality constraint. The following proposition reveals the con- nection between distinguishability, data geometry, and generation quality (proof in Appendix A);
Proposition Let dmac (6) == max 10 x|. e 2 d then Ax (1 + dimc (
solves Eq. (4), and folæ + Ar*) > O, A. Pp.
Watermarking through retraining user-end models The perturbation Ar" can potentially be reverse engineered and removed when generative models white-box users (e.g.. when models are downloaded by users). Therefore, we propose to instead retrain the user-end models G using the perturbed dataset o == {Go(z)+ 2 ~~ P2} with y > so that the perturbation realized through the model architecture and weights. Specifically, the retraining fine-tunes Go so that G (z(z) matches with Go(2) + yo for z P.. Since this matching will be perfect, we use the following model to characterize the resultant Gø:
Go(z) = Go(=) + + +
(5)
where the error e ~ (p, Σ). In Sec. 3 we provide statistics of μr and on the benchmark datasets, to show that the retraining captures the perturbations well (μ close to 0 and small variances in E).
Updating Proposition due to the existence of e leads to Theorem 1, where we show that y needs to be no smaller than dmux (Φ) in order for Go achieve distinguishability (proof in Appendix B):
Theorem 1. Let dmmar (Ø) = maX 1Φ x², d²) TTE 8 E [0, 1), and @ be a data-compliant key. ( 2 1 5/2(0 if
2 dymar (0) + ( log
OTT
(6)
02
Remarks The computation of (Ø) requires Go, which in turn requires y. Therefore, iterative search is needed to determine 7 that is small enough to limit the loss of generation quality, yet large enough for distinguishability (see Alg. M.
Attributability We can now derive the sufficient conditions for attributability of the generative models from a set of N keys (proof in Appendix C):
Theorem 2. Let dmin minzeD T xl,, diaa max 10 x/, 0² (0) ( dmac(()) ) + dmmin(d)) 20 μ.
EE 5 10, 1]. Let
(6:0))==11
a(d') log (+b) dmarl ( 6/
for keys & and ø. Then A(g) No, D(G) 2 1- for all Gø E G, and H' < a().
(8)
for any pair of data-compliant keys & and d.
Remarks When a0') is negligible for all 0' and μ 0. a(o,  is approximately dmin( ddmx((xx > 0, in which case o"d < 0 is sufficient for attributability. In Sec. we empirically show that this approximation is plausible for the benchmark datasets.
3

---

Published as a conference paper at ICLR 2021
(a)
(b)
(e)
RAIS
MINIST
miljpt 10880 03
1,000
1.30
0.96
MINST
CelebA
0,996
0.36
FETIO
RHS
0.994
0.34
MINIST
CelahA
0,990
00
320
40
60
80
100
60
80
100
(9)
Model
Model
MNIST
CelebA
(h)
15% S4
M MMMM
0.03
y
0.02
ColebA
FFWQ
99%
50%
20
Coo.
Σ
0.00
20
80
100
Model ID
Model ID
Figure 2: (a) Validation of Theorem 1: All points should be close to the diagonal line or to its right. (b) Support for orthogonal keys: Min. RHS value of Eq.(7) for keys are either positive (MNIST) or close to zero (CelebA). (c,d) Statistics of and Σ for two sample user-end models for MNIST and CelebA. Small i and small diag(E) suggest good match of Go to the perturbed data distributions. (e-h) Distinguishability, attributability, perturbation length, and orthogonality of 100 StyleGAN user-end models on FFHQ and 100 DCGAN user-end models on MNIST or CelebA, respectively.
EXPERIMENTS AND ANALYSIS
In this section we test Theorem provide empirical support for the orthogonality of keys, and present experimental results model attribution using MNIST, CelebA, and FFHQ. Note that tests on the theorems require estimation Σ. which is costly for models with high-dimensional outputs, and therefore are only performed MNIST and CelebA.
Key generation
We generate keys by iteratively solving the following convex problem:
Φ arg min Re-Po.Go [max 11 T5x
xr 0}]
max m 0.00.
The orthogonality penalty is omitted for the first key. The solutions are normalized to unit l2 norm before being inserted into the next problem. We note that Pp and Pc. do not perfectly match in practice, and therefore we draw with equal chance from both distributions during the computation. Gos are trained using standard DCGAN architecture for MNIST and CelebA, and StyleGAN for FFHQ. Training details are deferred to Appendix D.
The training G follows Alg. ! where y iteratively tuned to balance generation quality and distinguishability. For each collect a perturbed dataset D 7. and solve the following training problem:
User-end generative models
min (a [1]G((2;)) - =
(10)
starting from = 0g. If the resultant model does not meet the distinguishability requirement due to the discrepancy between Dy, and Ga, the perturbation is updated as 7 ay. In experiments, we use standard normal distribution for Ps. and set = 10 1 and a = 1.1.
Validation of Theorem Here we validate the sufficient condition for distinguishability. Fig. 2 compares the LHS and RHS values of Eq. (6) for 100 distinguishable user-end models. The em- pirical distinguishability of these models are reported Fig. 2 Calculation of the RHS of Eq. (6) requires estimations of and Σ. To do this, we sample
((2) Go(=; 0) G(2; (0) 7
(11)

---

Published as a conference paper at ICLR 2021
using 5000 samples of Ps, where G and are derived from Alg. Σ and μ are then estimated for each o. Fig. 2 and present histograms of the elements in μز and for two user-end models of the benchmark datasets. Results in Fig. 2 show that the sufficient condition for distinguishability (Eq. ) is satisfied for most of the sampled models through the training specified in Alg. D Lastly, notice that the LHS values for MNIST are farther away from the equality line than those for CelebA. This is because the MNIST data distribution resides at corners of the unit box. Therefore perturbations of the distribution are more likely to exceed the bounds for pixel values. Clamping of these invalid pixel values reduces the effective perturbation length. Therefore to achieve distin- guishability. Alg. [ seekks ys larger than needed. This issue is less observed CelebA, where data points are rarely close to the boundaries. Fig. present the values of ys of all user-end models.
Validation of Theorem 2 Recall that from Theorem 2, we rec- ognized that orthogonal keys sufficient. To support this de- sign rule, Fig. 2b presents the minimum RHS values of Eq. (8) for 100 user-end models. Specifically, for each Φi, we com- pute a( Φ,) (Eq. (7) using Φj for j 1...., 1 and re- port miny alor, U5), which sets an upper bound on the angle be- tween Φ; and all existing és. The resultant min, a(ds, 0j) are positive for MNIST and close to zero for CelebA. From this result, angle of 2 94 deg, instead of 90 90g, should be en- forced between any pairs of keys for CelebA. However, since the conditions are sufficient, orthogonal keys still empirically achieve high attributability (Di although improvements can be made by further increasing the angle between keys. Also no- tice that the current computation of keys (Bq. (9) does not en- force hard constraint on orthogonality, leading slightly acute angles (87.7 deg) between keys for CelebA (Fig. [2). On the other hand, the positive values in Fig. 2 for MNIST suggests that further reducing the angles be- tween keys is acceptable if one needs to increase the total capacity of attributable models. However, doing so would require the derivation of new keys to rely on knowledge about all existing user-end models (in order to compute Eq. (7).
Algorithm 1: Training of Gs
input : : Go
output: Gg. ?
set = :
collect Dy,
train G by solving Eq. 100
using Dy, y..
compute empirical ;
if ( < 1 then
set y = aY: goto step 2;
end
Table I: Empirical average of distinguishability (D))atrributabiliiyy (A(9)), || || and FID scores. DCGANM (DCGANcc for MNIST (CelebA). Std in parenthesis. FIDg: FID for G 4 means lower better and 1 means higher better.
GANs
Angle
D f
A(g) t
||
FID,
FID
Orthogonal
0.99
0.99
3.97 (0.29)
7.82 (0.15)
10.43 (0.77)
DCGANM
45 degree
0.99
0.13
3.85 (0.12)
11.02 (0.86)
Orthogonal
0.99
0.93
4.04 (0.32)
35,95 (0.12)
58.69 (5.21)
DCGAN
45 degree
0.99
0.15
4.57 (0.35)
59.81 (5.34)
Orthogonal
0.99
0.99
36.04 (23.35) 12.43(0.16)
35.23(14.67)
StyleGAN
45 degree
0.97
0.18
67.19 (35.53)
47.86(10.66)
Empirical results on benchmark datasets Tab. I reports the metrics of interest measured on the 100 user-end models for each of MNIST and CelebA, and 20 models for FFHQ. All models are trained to be distinguishable. And by utilizing Theorem 2, they also achieve high attributability. As a comparison, we demonstrate results where keys are 45 dog apart ToT = 0.71) using a separate set of 20 user-end models for each of MNIST and CelebA, and 5 models for FFHQ, in which case distinguishability no longer guarantees attributability. Regarding generation quality, G&S receive worse FID scores than Go due to the perturbations. We visualize samples from user-end models and the corresponding keys in Fig. 3 Note that for human faces, FFHQ in particular, the perturbations create light shades around eyes and lips, which is an unexpected but reasonable result.
Attribution robustness generation quality We now consider the scenario where outputs of the generative models are post-procett processed (e.g.. by adversaries) before being attributed. When the post-processes are known, can take counter measures through robust training, which intuitively
5

---

Published as a conference paper at ICLR 2021
(a) MNIST
(b) CelebA
(c) FFHQ
8
1
5
6
3
7
(
0
6
1
9
4
Figure 3: Visualization of sample keys (Ist row) and the corresponding user-end generated contents.
will lead additional loss of generation quality. To assess this tradeoff between robustness and generation quality, we train G against post-processes T: Rd= - Rd from a distribution Pr. Due the potential nonlinearity of T and the lack of theoretical guarantee in this scenario, we resort to the following robust training problem for deriving the user-end models:
min E [max(1 - So. (2: 0} + C[|Go(=) Go, Go, 9,
(12)
where C is the hyper-parameter for generation quality. Detailed analysis and comparison for lecting C are provided in Appendix E. We consider five types of post-processes: blurring, crop- ping, noise, JPEG conversion and the combination of these four. Examples of the post-processed images are shown in Fig. 5 Blurr surring uses Gaussian kernel widths uniformly drawn from (1, 5, 9}. Cropping crops images with uniformly drawn ratios between 80% and 100%, and scales the cropped images back to the original size using bilinear interpolation. No is se adds white noise with standard deviation uniformly drawn from [0, 0.3). JPEG applies JPEG compression. Combina i on performs each attack with 50% chance in the order of Blurring. Cropp' ng, Noise and JPEG. For differentiabilitty we use existing implementations of differentiable blur- ring (Riba et al. (2020)) and JPEG conversion (Zhu et al. (2018). For robust training, we apply the post-process to batche with 50% probability.
We performed comprehensive tests using DCGAN (on MNIST and CelebA), PGAN (on CelebA), and CycleGAN (on Cityscapes). Tab. summarizes the average distinguishability. the attributabil- ity, the perturbation length || and the FID score with and without robust training of Ga. Re- sults are based on 20 models for each architecture-datase pair, where keys are kept orthogonal and data compliant. From the results, defense against these post-processes can be achieved, except for Combi ion. Importantly, there is a clear tradeoff between robustness and generation quality. This can be seen from Fig. 5 which compares samples with7 and without robust training from the tested models and datasets.
Lastly, is worth noting that the training formulation in Eq. 12 can also be applied to the training of non-robust user-end models in place of Eq. (TO). However, the resultant model from Eq. (1) cannot be characterized by Eq. (5) with small / and Σ, i.e., due to the nonlinearity of the training process of Eq. (12] the user-end model distribution is deformed while it is perturbed. This resulted unsuccessful validation of the theorems, which led to the adoption of Eq. (10) for theorem-consistent training. Therefore, while the empirical results show feasibility of achieving robust attributabilitt using Eq. [12] counterparts to Theorems and 2 in this nonlinear setting are yet to be developed.

---

Published as a conference paper at ICLR 2021
Table 2: DCGANM: MNIST. DCGANc: CelebA. Dis.: Distinguishability before (Bfr) and after (Aft) robust training. Att.: Attributability. || and FID are after robust training. Std in paren- thesis. + means lower better and means higher is better. || and FID before robust train- D Aæ| 5.05. FID = 5.36. DCGANc: Aær 5.63, FID 53.91. PGAN:
ing:
|| = 9.29, FID 21.62. CycleGAN: || = 55.85, FID does not apply to CycleGAN.
Metric
Model
Blurring Aft
Cropping Aft
Noise Aft
JPEG
Combi.
Bfr
Bfr
Bfr
Bfr
Aft
Bfr
Aft
DCGANM
0.49
0.96
0.52
0.99
0.85
0.99
0.54
0.99
0.50
0.66
DCGANc
0.49
0.99
0.49
0.99
0.95
0.98
0.51
0.99
0.50
0.85
Dis. t
PGAN
0.50
0.98
0.51
0.99
0.97
0.99
0.96
0.99
0.50
0.76
CycleGAN
0.49
0.92
0.49
0.87
0.98
0.99
0.55
0.99
0.49
0.67
DCGANM
0.02
0.94
0.03
0.88
0.77
0.95
0,16
0.98
0.00
0.26
DCGANc
0.00
0.98
0.00
0.99
0.89
0.93
0.07
0.98
0.00
0.70
Att.
PGAN
0.13
0.99
0.07
0.99
0.97
0.99
0.99
0.99
0.06
0.98
CycleGAN
0.08
0.98
0.05
0.93
0.97
0.98
0.47
0.99
0.05
0.73
DCGANM
15.96(2.18)
9.17(0.65)
5.93(0.34)
6.48(0.94)
17.08(1.86)
DCGANc
11.83(0.65)
9.30(0.31)
4.75(0.17)
6.01(0.29)
13.69(0.59)
|| +
PGAN
18.49(2.04)
21.27(0.81)
10.20(0.81)
10.08(1.03)
24.82(2.33)
CycleGAN
68.03(3.62)
80.03(3.59)
55.47(1.60)
57.42(2.00)
83.94(4.66)
DCGANM
41.11(20.43)
21.58(2.44)
5.79(0.19)
6,50(1.70)
68.16(24.67)
FID
DCGANc
73.62(6.70)
98.86(9.51)
59.51(1.60)
60.35(2.57)
87.29(9.29)
PGAN
28.15(3.43)
47.94(5.71)
25.43(2.19)
22.86(2.06)
45.16(7.87)
DISCUSSION
Capacity of keys For real-world applications, we hope to maintain attributability for a large set of keys. Our study so far suggests that the capacity of keys is constrained by the data compliance and orthogonality requirements. While the empir- ical study showed the feasibility of computing keys through Eq. (9). finding the maximum number of feasible keys is problem about optimal sphere packing on a segment of the unit sphere (Fig. 4 To explain, the unit sphere represents the iden- tifiability requirement ||o|| = The feasible segment of the unit sphere is determined by the data compliance and gener- ation quality constraints. And the spheres to be packed have radii following the sufficient condition in Theorem Such optimal packing problems are known open challenges (Cohn) al. (2017; Cohn (016)). For real-world applications where a capacity of attributable models is needed (which is the case for both malicious personation and copyright infringement set- tings), it is necessary to find approximated solutions to this problem.
$ 3
Figure 4: Capacity of keys as a sphere packing problem: The fea- sible space (arc) is determined by the data compliance and genera- tion quality constraints, and the size of spheres by the minimal an- gle between keys.
Generation quality control From Proposition and Theorem 1, the inevitable loss of generation quality is directly related to the length of perturbation (7), which is related to dmax- Fig. 6 com- pares outputs from user-end models with different dmaxs. While it is possible filter ćs based on their corresponding d for generation quality control, here we discuss a potential direction for prescribing subspace of Φs within which quality can be controlled. To start, we denote by J(x) the Jacobian of Go with respect to its generator parameters fg. Our discussion is related to the ma- spectral analysis of M reveals that the eigenvectors of
trix Eg, [J [Jx=)]E [J (x))).. M with large eigenvalues are more structured than those with small ones (Fig. 7 aa)). This finding consistent with the definition of M: The largest eigenvectors of M represent the principal axes of all mean sensitivity vectors, where the mean is taken over the latent space. For MNIST, these eigenvectors overlap with the digits; for CelebA, they are structured color patterns. On the other hand, the smallest eigenvectors represent directions rarely covered by the sensitivity vectors, thus

---

Published as a conference paper at ICLR 2021
Bbn. Mid Top
704
104
104
7 704
0 4
7 0 4
(u) Non-robust
(b) Blurring
Cropping
7 704 4
7 0 4
704 104
70 4
7 704
(d) Noise
(6) JPEG
(f) Combination
Figure 5: Samples from user-end models with robust and non-robust training. For each subfigure
top: DCGAN MNIST and CelebA; bottom: PGAN (CelebA) and CycleGAN (Cityscapes). For each dataset top: samples from Go (after worst-case post-process in (b-f)); mid: samples from G (after robust training in (b-f)); btm (a): difference between non-robust Go and Go: btm (b-h) difference between robust and non-robust Go.
6
9
8
4
7
5
dmax"
(b) dmaa""
(c) d diear"
(0) dmax=
(f) d
Figure 6: MNIST, CelebA, and FFHQ examples from Gas with (a-c) small dmaa and (d-f) large dmaz- All models are distinguishable and attributable. (Zooming in on pdf file recommended.)
resembling random noise. Based on this finding, the cigenspace of the small eigenvalues will have smaller dmaxx- We test this hypothesis by com- puting the Pearson correlations between dmax and o M using 100 models for each of MNIST and CelebA. The resultant correlations are 0.33 and 0.53, respectively. In addition, we compare outputs from models using the largest and the smallest eigenvectors of While a concrete human study is needed, the visual results suggest that using eigenvectors of M is promising approach to segmenting the space of keys according to their induced generation quality.
the hypothesis that keys more aligned with
the keys in Fig. 77..
RELATED WORK
Detection and attribution of model-generated contents This paper focused on the attribution of contents from generative models rather than the detection of hand-crafted manipulations (Agarwal &
Farid ( & Farid (205)) ( brien & Fanid (2012); Rao & Ni((2016;; Huh et al. 2018)).
Rao Detection methods rely fingerprints intrinsic generative models (Odena al. (2016)
Zhhng

---

Published as a conference paper at ICLR 2021
A-1064.67
2-583.91
A-0.03
2m0.02
2
=22991.44
A-2518.4
1-0.24
2m0025
(a)
(b)
Figure 7: (a) Eigenvectors for the two largest and two smallest eigenvalues of M for DCGANs on MNIST (top) and CelebA (bottom). (b) Left column: Samples from Go; Rest: Gu Go where are the cigenvectors in (a).
et al. (2019b): Marra et al. (2019): Wang et al. (019)), yet similar fingerprints present for models trained on similar datasets (Marra et al. (2019). Thus fingerprints cannot be used for attribution. Skripniuk et al. (2020) studied decentralized attribution. Instead of using linear classifiers for attri- bution, they train watermark encoder-decoder network that embeds (and reads) watermarks into (and from) the content, and compare the decoded watermark with user-specific ones. Their method does not provide sufficient conditions of the watermarks for attributability.
Watermarks have conventionally been used for IP protection (Tirkel et al.] 1993 Van Schyndel et al. 1994 Bi et al. 2007 Hsieh et al. 200T] Pereira & Pun 2000 Zhu et al. 2018 Zhang et al.) 2 without considering the attribution guarantec. Another approach to content IP protection is blockchain (Hasan & Salah 2019). However, this approach requires meta data to be transferred along with the contents, which may not be realistic in adversarial settings. E.g., one can simply take a picture of a synthetic image to remove any meta data attached to the image file. Aside from the protection of contents, mechanisms for protecting IP of models have also been studied (Uchida et al. 2017 Nagai et al. [0188 Le Merrer et al.) 2019
IP protection of digital contents and models
Adi al. 2018 Zhang al.; 2018; Fan et al. 2019 Szyller al. 2019)] Zhang al.) 2020
Model watermarking is usually done by adding watermarks into model weights (Uchida et al. 2017
Nagai et al. 2018). by embedding unique input-output mapping into the model (Le Merrer al. 2019 Adi et al. 2018 Zhang et al.] 2018), or by introducing a passport mechanism so that model accuracy drops the right passport not inserted (Fan et af. 2019). While closely related, existing work model IP protection focused on the distinguishability of individual models, rather than the attributability of a model set.
CONCLUSION
Motivated by emerging challenges with generative models, e.g., deepfake, this paper investigated the feasibility of decentralized attribution of such models. The study is based on protocol where the registry generates user-specific keys that guides the watermarking of user-end models to be distinguishable from the authentic data. The outputs of user-end models will then be attributed by the registry through the binary classifiers parameterized by the keys. We developed sufficient conditions of the keys so that distinguishable user-end models achieve guaranteed attributability. These conditions led to simple rules for designing the keys. With concerns about adversarial post- processes, further showed that robust attribution be achieved using the same design rules, and with additional loss of generation quality. Lastly, we introduced two open challenges towards real-world applications of the proposed attribution scheme: the prescription of the key space with controlled generation quality, and the approximation of the capacity of keys.
ACKNOWLEDGEMENT
Support from NSF Robust Intelligence Program (1750082), ONR (N00014-18-1-27611, and Ama- zon AWS MLRA is gratefully acknowledged. We would like to express gratitude to Ni Trieu (ASU) for providing us invaluable advice, and Zhe Wang, Joshua Feinglass, Sheng Cheng, Yong- back Cho and Huiliang Shao for helpful comments.

---

Published as a conference paper at ICLR 2021
REFERENCES
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weak- ness into a strength: Watermarking deep neural networks by backdooring. In 27th {USENIX) Security Symposium ( Security 18), pp. 1615-1631, 2018.
Shruti Agarwal and Hany Farid. Photo forensics from jpeg dimples. In 2017 JEEE Workshop on Information Forensics and Security (WIFS), pp. 1-6. IEEE, 2017.
Ning Bi, Qiyu Sun, Daren Huang, Zhihua Yang. and Jiwu Huang. Robust image watermarking based on multiband wavelets and empirical mode decomposition. IEEE Transactions on Image Processing, 16(8):1956- 1966, 2007.
Ali Breland. The bizarre and terrifying case of the "deepfake" video that helped bring an african nation to the brink, motherjones, Mar 2019. URL https: /ww. mother joneeeerroonss...o com/
bongo/
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint /80i 1096, 2018.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Wou Woo Ha, Sunghun Kim, and Jaegul Choo. Star- gan: Unified generative adversarial networks for multi-domain maagetto-image translation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 8789-8797, 2018.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Wor Ha. Stargan v2: Diverse image synthesis for multiple domains. arXiv preprint arXiv: 1912.0161 2019.
Henry Cohn. A conceptual breakthrough in sphere packing. arXiv preprint 16 2016.
Henry Cohn, Abhinav Kumar, Stephen D Miller, Danylo Radchenko, and Maryna Viazovska. The sphere packing problem in dimension 24. Annals of Mathematics, pp- 1017-1033, 2017.
Lixin Fan, Kam Woh Ng. and Chee Seng Chan. Rethinking deep neural network ownership veri- fication: Embedding passports to defeat ambiguity attacks. In Advances in Neural Information Processing Systems, pp. 4716-4725, 2019.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Haya R Hasan and Khaled Salah. Combating deepfake videos using blockchain and smart contracts.
Ieee Access, 7:41596-41606, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. In Advances in Neural Information Processing Systems, pp. 6626-6637, 2017.
Ming-Shing Hsieh, Din-Chang Tseng, and Yong-Huai Huang. Hiding digital watermarks using multiresolution wavelet transform. IEEE Transactions on industrial electronics, 48(5):875-882, 2001.
Minyoung Huh, Andrew Liu, Andrew Owens, and Alexei A Efros. Fighting fake news: Image splice detection via learned self-consistency. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 101-117,200
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint 17X1v:1700.10196, 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4401-4410, 2019a.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyz- ing and improving the image quality of stylegan. arXiv preprint arXiv: 1912, 04958, 2019b.
10

---

Published as a conference paper at ICLR 2021
Makena
Kelly.
Congress
grapples
with
how to regulate deep-
fakes.
Congress
grapples
with
how
to regulate deepfakes. Jun
2019.
URL
wtt.g/e..cmm2000 https: www. /ttp://w..aaarrerroo////////////
deep fakes- regu ltt on- www.onnnfooooaaaaaooooooooooooo adam- waeelaamscshf----- aeronggess-artifaaannn oag.eeasassass------o----aaaaaaa
intel ignnee
Erwan Le Merrer, Patrick Perez, and Gilles Trédan. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications, pp. 1-12, 2019.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URLhttpp///ann.
lecun www. con/exdb/mnist)
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang.
Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. Do gans leave artificial fingerprints? In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 506-511. IEEE, 2019.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXh::1802.05957, 2018
Yuki Nagai, Yusuke Uchida, Shigeyuki Sakazawa, and Shin'ichi Satoh. Digital watermarking for deep neural networks. International Journal of Multimedia Information Retrieval, 7(1):3-16, 2018.
James F FObbrien and Hany Farid, Exposing photo manipulation with inconsistent reflections. ACM Trans. Graph., 31(1):4-1, 2012.
Augustus Odena, Vincent Dumoulin, and Chris Olah. Deconvolution and checkerboard arti- facts. Distill, 2016. doi: 10.23915/distil.0000 URL httt:://distil /ttp///2itill..pp////0////
pub/201
deconv checkerbaad
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2536-2544, 2016.
Shelby Percira and Thierry Pun. Robust template matching for affine resistant image watermarks. IEEE transactions image Processing, 96): (66):112311111 123-1129, 2000.
Alin C Popescu and Hany Farid. Exposing digital forgeries by detecting traces of resampling. IEEE Transactions on signal processing, 53(2):758-767, 2005.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, 2015. URL /RLL://p///a/iv.cr ort. / 1511.00 06434 cite 1xdv:1551.0044344ommmntt Under review as conference paper at ICLR 2016.
Yuan Rao and Jiangqun Ni. A deep learning approach to detection of splicing and copy-move forgeries in images. In 2016 IEEE International Workshop Information Forensics and Security (WIFS), pp. 1-6. IEEE, 2016.
E. Riba, D. Mishkin, D. Ponsa, E. Rublee, and G. Bradski. Korii: an open source differen- tiable computer vision library for pytorch, 2020. URL httpppp///aaxii.ooo///////////// 02190.pdF.
Raphael Satter. Experts: Spy used ai-generated face to connect with targets. Experts: Spy used Al-generated face to connect with targets, Jun 2019. URL httpss://aatt //tp:////ppwww//// com/
Vladislav Skripniuk, Ning Yu, Sahar Abdelnabi, and Mario Fritz. Black-box watermarking for generative adversarial networks. arXiv preprint arXi::000.008457, 2020.
Sebastian Szyller, Buse Gul Atli, Samuel Marchal, and N Asokan. Dawn: Dynamic adversarial watermarking of neural networks. arXiv preprint 100iv:9900000888 2019.

---

Published as a conference paper at ICLR 2021
Anatol Z Tirkel, GA Rankin, RM Van Schyndel, WJ Ho, NRA Mee, and Charles F Osbomne. Elec- tronic watermark. Digital Image Computing. Technology and Applications (DICTA '93), pp. 666- 673, 1993.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pp. 269-277, 2017.
Ron G Van Schyndel, Andrew Tirkel, and Charles F Osborme. A digital watermark. Proceedings of 1st International Conference Image Processing, volume 2, pp. 86-90. IEEE, 1994.
Sheng- Yu Wang. Oliver Wang, Andrew Owens, Richard Zhang, and Alexei A Efros. Detecting photoshopped faces by scripting photoshop. In Proceedings the IEEE International Conference Computer Vision, pp.
2019.
Ting-Chur Wang, Ming- Yu Liu, Jan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1798-8801 2018.
Ning Yu, Larry Davis, and Mario Fritz. Attributing fake images to gans: Analyzing fingerprints in generated images. arXiv preprint arXiv:1811.08/ 2018.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dim- itris N. Metaxas. Stackgan: Text to reato-rea image synthesis with stacked generative adver- sarial networks. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Mare Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking, In Proceed- ings of the 2018 on Asia Conference on Computer and Communications Security, pp. 159-172, 2018.
Jie Zhang, Dongdong Chen, Jing Liao, Han Fang, Weiming Zhang, Wenbo Zhou, Hao Cui, and Nenghai Yu. Model watermarking for image processing networks. In AAAI, pp. 12805-12812, 2020.
Kevin Alex Zhang, Alfredo Cuesta-Infante Lei Xu, and Kalyan Veeramachaneni. Steganogan: High capacity image steganography with gans, arXiv preprint arXiv:1901.0389) 2019a.
Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and simulating artifacts in gan fake images, arXiv preprint arXiv:I907.0633 06515, 2019b.
Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. Hidden: Hiding data with deep networks. In Proceedings of the European Conference Computer Vision (ECCV), pp. 657-672, 2018.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
12

---

Published as a conference paper at ICLR 2021
A PROOF OF PROPOSITION 1
Proposition 1. Let dmax (6) = max 10" x/. Ife 2 + (m) then Ax = (1 + dimac ( solves Eq. (4), and folx + Ar) -V Go-
Proof. Let o be a data-compliant key and let x be sampled from Pp. First, from the KKT conditions for Eq. (4) we can show that the solution Ax" proportional Φi:
Ar 0/P"
(13)
where pz 2 0 is the Lagrange multiplier. To minimize the objective, we seek μ such that
1 - (x + o= 1 - δ 1/1²" 00
(14)
for all x. Since x= à < 0 (data compliance), this requires 1/1²² = + Ammx(o), Therefore, when 8 dmax 6). Ax" (1 + (m)x solves Eq. (4). And folo Ax") 6² (1+ dmmx (b) = o² x + 1 (m) 0.
B PROOF OF THEOREM
Dds 8 [0, 1], and be a data data-compliant Theorem 1. Let dmox( = max 10 (2)
key. ((Gg) 2 8/2
7 2 20) log dmar( -TT
(15)
Proof. We first note that due to data compliance of keys, Epw [1 0)] 1. Therefore D(G,) 2 8/2 iff Eg-pa [1(o">x 0)] 2 5, i.e. Pr(o" 0) 2 1 5ạ for PG.-
We now seek lower bound for Pr(oT> x > 0). To do so, let and To be sampled from Pc. and Pc., respectively. Then we have
Φ Tτ (xo + 70 cc
(16)
- 3u + 1 d²c,
and
Pi(p"x> 0) Pr (0r -0" xo 70)-
(17)
Since dmax (p) 2 - T To, we have
Pr(p- x 0) Pr (o= dm -)) Pr
PP - 11) S dmnx (0) + + d)
(18)
The latter sign switching in equation 18 is granted by the symmetry of the distribution of o (e p)) which follows N/O0, o" EΦ). A sufficient condition for Pr(oTT x > 0) 2 1 - then
Pr Pr μ1) < dmax (0) + μ)) 2 1 5.
(19)
Recall the following tail bound of r ~ _/ 02) for y 2 0:
Pr(x < σу) > 1 exp(-1)²12)
(20)
Compare equation 20 with equation 19 the sufficient condition becomes
7 2 o (00) log
+ dmaa (1) - ST n.
(21)
82
n
13

---

Published as a conference paper at ICLR 2021
C PROOF OF THEOREM 2
Theorem 2. Let dmin m 10 x/. dmaa max D 10 x), (6) - | E [0.1]. A(G) 2
1 N6 if D(G) 2 ō for all Go E G and for any pair of data-compliant keys & and Ø
dmax(dd) ) + dmin -66
8²Ø' 0' s -1
(22)
o(d) hog (d2) + dimar ( D0
Proof. Let Φ and ø' be any pair of keys. Let and zo be sampled from Pc, and Pc.. respectively.
We first derive the sufficient conditions for Pr(Ø x 0) 2 ò. Since a an + 70 + e for E Gg. we have
8 x = TT (2g + +(+
(23)
= -
TT 20 + 10 0 + TTc
Then
Pr(p x <) = Pr (0 -0"
In o'=
(24)
2 Pr (0 (c: - 11) dmin (8)) - 70 8' o p)
where dd') min 10 and ( 1) NN0, 22 Using the same tail bound of normal distribution and Theorem 1, we have Pr(ø" x < 0) 2 if
- 2 ( log 82
- (m))(())) + TT
(25)
T 0 < -1+ dmue (ō') + dmin (d') 26
o(0') log (1)) + dmnax (1 ) 18T
Note that Pr(A 1, B,
1) 11 1 0) Pr(B =0 +
= 0, 0)
0) Pr(B = 0) for binary random variables A and B. With this, it is straight forward to show
that when Pr(d x < 0) > 1 for all ø' * 9, and Pr(ø x > 0) 2 for all Φ, then
Pr(p x 0, o < <009'' # の) 2 and A(G) 2 1 - N8.
D TRAINING DETAILS
D.1 METHOD
We trained user-end models based on the objective function (Eq,(10) in the main text). For datasets where the root models follow DCGAN and PGAN, the user-end models follow the same architec- ture. For the FFHQ dataset where StyleGAN is used, we introduce an additional shallow convolu- tional network as a residual part, which is added to the original StyleGAN output to match with perturbed datasets D.,0- In this case, the training using Eq.(10) is limited to the additional shallow network, while the StyleGAN weights are frozen. More specifically, denoting the combination of convolution, ReLU, and max-pooling by Cen--ReLU-Max. the shallow network consists of three Conv-ReLU-Mar blocks and one fully connected layer. All of the convolution layers have 4 x kernels, stride 2, and padding 1. And all of the max-pooling layers have x 3 kerness and stride
D.2 PARAMETERS
We adopt the Adam optimizer for training. Training hyper-parameters summarized in Table
14

---

Published as a conference paper at ICLR 2021
Table 3: plyper-parameters to train keys (Ø) and generators (G&).
GANs
Dataset
Batch Size
Learning Rate
B1
[2
Epochs
DCGAN
MNIST
16
0.001
0.9
0.99
10
DCGAN
CelebA
64
0.001
0.9
0.99
2
StyleGAN
FFHQ
0.001
0.9
0.99
D.3 TRAINING TIME
All experiments are conducted on V100 Tesla GPUs. Table summarizes the number of GPUs used and the training time for the non-robust models (Eq-(10) in the main text) and robust models (Eq. in the main text). Recall that we chose (1.) for training the non-robust user-end models for consistency with the theorems, although (12) can be used achieve attributability in practice, as is shown in the robust attribution study. Therefore, the non-robust training takes longer to due the iteration of y in Alg.
Table 4: Training time (in minute) of one key (Eq,(9) in main text) and one generator (Eq,(10) in main text). DCGANM: DCGAN for MNIST, DCGANc: DCGAN for CelebA.
GANs
GPUs
Key
Non-robust
Blurring Cropping
Noise
JPEG
Combination
DCGANM
1.77
14
4.12
3.96
4.19
5.71
5.12
DCGANc
5.31
15
10.33
9.56
10.35
10.25
10.76
PGAN
2
50.89
141.07
140.05
131.90
133.46
132.46
135.07
CycleGAN
20.88
16.04
16.26
15.43
15.71
15.98
16.41
StyleGAN
54.23
3.12
E ABLATION STUDY
Here we conduct ablation study on the hyper-parameter C for the robust training formulation (12). Training with larger C' focuses more on generation quality, thus sacrificing distinguisha- bility and attributability. These effects are reported in Table and Table 6 Due to limited time, the results here are averaged over five models for each C and data-model pairs.
15

---

Published as a conference paper at ICLR 2021
Table 5: Distinguishability (top), attributability (btm) before (Bfr) and after (Aft) robust training.
DCGANM: DCGAN for MNIST, DCGANc: DCGAN for CelebA.
Model
C
Blurring
Cropping
Noise Bfr
JPEG
Comb combination
Bfr
Aft
Bfr
Aft
Aft
Bfr
Aft
Bfr
Aft
DCGANM
10
0.49
0.97
0.51
0.99
0.84
0.99
0.53
0.99
0.50
0.63
DCGANM
100
0.49
0.61
0.51
0.98
0.76
0.98
0.53
0.99
0.50
0.52
DCGANM
1K
0.49
0.50
0.51
0.81
0.69
0.91
0.53
0.97
0.50
0.51
DCGANc
10
0.49
0.99
0.49
0.99
0.96
0.99
0.50
0.99
0..99
0.85
DCGAN
100
0.50
0.96
0.49
0.99
0.92
0.93
0.50
0.99
0.49
0.61
DCGANc
IK
0.50
0.62
0.49
0.97
0.88
0.91
0.50
0.99
0.49
0.51
PGAN
100
0.50
0.98
0.50
0.99
0.96
0.99
0,96
0.99
0.50
0.81
PGAN
IK
0.50
0.89
0.49
0.95
0.94
0.95
0.88
0.99
0.50
0.60
PGAN
10K
0.50
0.61
0.50
0.76
0.89
0.90
0.76
0.98
0.50
0.51
CycleGAN
IK
0.49
0.92
0.50
0.87
0.98
0.99
0.55
0.99
0.49
0.62
CycleGAN
10K
0.49
0.70
0.50
0.66
0.94
0.96
0.52
0.98
0.50
0.51
DCGANM
10
0.02
0.94
0.03
0.88
0.77
0.95
0.16
0.98
0.00
0.26
DCGANM
100
0.00
0.87
0.00
0.85
0.73
0.90
0.10
0.95
0.00
0.13
DCGANM
1K
0.00
0.75
0.00
0.80
0.63
0.80
0.10
0.91
0.00
0.05
DCGAN
10
0.00
0.98
0.00
0.99
0.89
0.93
0.07
0.98
0.00
0.70
DCGAN
100
0.00
0.95
0.00
0.93
0.82
0.85
0.02
0.93
0.00
0.61
DCGAN
IK
0.00
0.90
0.00
0.89
0.77
0.81
0.00
0.88
0.00
0.43
PGAN
100
0.26
1.00
0.21
1.00
0.99
0.99
0.99
0.99
0.00
0.99
PGAN
1K
0.21
0.99
0.00
0.99
0.97
0.98
0.98
0.99
0.00
0.54
PGAN
10K
0.00
0.51
0.00
0.90
0.90
0.92
0.83
0.99
0.00
0.22
CycleGAN
IK
0.00
0.99
0.00
0.97
0.97
0.99
0.45
0.99
0.00
0.77
CycleGAN
1OK
0.00
0.87
0.00
0.77
0.95
0.96
0.30
0.99
0.00
0.31
Table |Arl| (top) and FID score (btm). Standard deviations in parenthesis. DCGANM: DCGAN
for MNIST, DCGANc: DCGAN for CelebA, Combi. :: Combination attack. Lower is better.
Model
C
Baseline
Blurring
Cropping
Noise
JPEG
Combi.
DCGANM
10
5.05(0.09)
15.96(2.18)
9.17(0.65)
5.93(0.34)
6.48(0.94)
17.08(1.86)
DCGANM
100
4.09(0.53)
12.95(4.47)
7.62(1.55)
4.57(0.78)
4.70(1.02)
12.70(3.37)
DCGANM
1K
3.88(0.60)
7.17(2.10)
7.43(1.37)
4.22(0.77)
5.12(1.94)
7.56(1.41)
DCGAN
10
5.63(0.11)
11.83(0.65)
9.30(0.31)
4.75(0.17)
6.01(0.29)
13.69(0.59)
DCGAN
100
3.08(0.27)
10.00(1.61)
7.80(0.58)
3.20(0.45)
4.26(0.59)
11.65(1.48)
DCGAN
IK
2.55(0.36)
7.68(1.53)
7.13(0.47)
2.65(0.24))
3.39(0.58)
9.23(1.22)
PGAN
100
9.29(0.95)
18.49(2.04)
21.27(0.81)
10.20(0.81)
10.08(1.03)
24.82(2.33)
PGAN
IK 6.52(1.85)
14.79(4.15)
18.88(1.96)
6.40(1.48)
7.09(1.62)
22.09(2.12)
PGAN
10K 5.04(1.63)
10.19(2.87)
18.23(0.94)
5.13(1.14)
5.67(1.62)
17.26(1.39)
CycleGAN
IK 55.85(3.67)
68.03(3.62)
80.03(3.59)
55.47(1.60)
57.42(2.00)
83.94(4.66)
CycleGAN
10K 49.66(5.01)
58.64(3.70)
66.05(3.47)
53.14(0.44)
54.52(2.30)
66.24(5.29)
DCGANM
10
5.36(0.12)
41.11(20.43)
21.58(2.44)
1.79(0.19)
6.50(1.70)
18.16(24.67)
DCGANM
100
5.32(0.11)
23.83(14.29)
18.39(3.70)
5,41(0.18)
5.46(0.11)
36.05(16.20)
DCGANM
1K
5.23(0.12)
10.85(4.28)
18.08(1.77)
5.37(0.14)
5.30(0.96)
21.86(4.166
DCGAN
10
53.91(2.20)
73.62(6.70)
98.86(9.51)
59.51(1.60)
60.35(2.57)
87.29(9.29)
DCGAN
100
45.02(3.37)
73.12(11.03)
85.50(12.25)
47.60(2.57)
50.48(4.58)
78.11(12.95)
DCGAN
1K
40.85(3.41)
55.63(7.97)
72.11(13.81)
40.87(3.03)
45.46(5.03)
57.13(7.20)
PGAN
100
21.62(1.73)
28.15(3.43)
47.94(5.71)
25.43(2.19)
22.86(2.06)
45.16(7.87)
PGAN
IK
19.05(3.14)
25.19(5.26)
43.48(12.24)
19.20(2.96)
19.05(2.82)
35.07(8.72)
PGAN
10K
16.75(1.87)
18.96(2.65)
37.01(8.74)
16.94(1.89)
17.39(2.33)
26.63(4.44)
16


Reviews:
[{"review_id": "rVsuemtEn3f", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a decentralized attribution to the generative model trained on the same dataset. The goal is to distinguish the user-end generative models, and thus facilitates the IP-protection. The idea is to use orthogonal keys to distinguish the generated samples from authentic data. Furthermore, this paper provided theoretic insights into the proposed method. Experimental results on MNIST and CelebA datasets backup the claims. \n\nStrengths:\n1.\tThe author tempts to distinguish the generative models trained on the sample dataset. This problem is interesting and inspiring to the researchers in the related field.\n\nWeaknesses:\n1.\tThis paper is not well-organized and many parts are misleading. For example, above Eq.3, the author assumes P_{G_{0}} = P_{D}. Does the author take the samples generated by the root generator as the authentic dataset? However, in Section 2 above Eq. 4, the author claims that the authentic data does not belong to any generator. \n2.\tIn Eq.4, the key-dependent generator is obtained via adding perturbation to the output of the root model. This setting may be troublesome as :1. These generators are not actually trained. This is different from the problem which this paper tempt to solve. 2. No adversarial loss to guarantee the perturbed data being similar to the authentic data. 2. How to distinguish the samples from different generators.\n3.\tSince Eq.4 is closely related to adversarial attack, the authors are supposed to discuss their connections in the related works.\n4.\tThe name of ‘decentralized attribution’ is misleading. Decentralized models are something like federated learning, where a ‘center’ model grasps information from ‘decentralized models’. However, the presented work is not related to such decentralization.\n5.\tTypos: regarding the adversarial generative models ->regarding to the adversarial generative models; along the keys->along with the keys.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "N5TheiSNZD", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nGiven the rebuttal, I am willing to raise my score to a 6 due to the added StyleGAN, PGAN, and other experiments and improved paper layout / clarity. The added experiments are welcome addition to the paper and demonstrate this technique. The lip and eyestaining are interesting results and I do hope this direction gets explored in the future.\n\n\nSummary\n==============================================================\n\nThis paper tries to create a decentralized method of attributing user-end generative model. The authors explore the conditions needed for these keys to be attributable and an example on how to design the keys for this condition.\n\nScore:\n===============================================================\n\n\nWhile this work is clearly worthwhile and useful as more generative models are deployed, the chosen toy examples may be a little too simplistic and low resolution to test this technique. Are these keys still attributable as these datasets continue to grow larger. I would raise more score if the technique was shown to work well on datasets more complex than MNIST and Celeb-A. Would the results holds on higher resolution datasets of greater variety? \n\nI would like to see if the keys work for StyleGAN based models that are a very accessible way of generating fake profile pictures, for example https://apnews.com/article/bc2f19097a4c4fffaa00de6770b8a60d\n\nI am unconvinced that this paper should be accepted. Given more experiments on more relevant / diverse datasets and improvements to figures in the paper, I could be persuaded to change my review.\nPros: \n=================================================================\n- The paper itself is well written\n- The demonstrates the keys can be made robust to common end user augmentations (cropping, compression, etc...)\n\n\nCons + Improvements:\n================================================================\n- The technique is only tested on datasets of CelebA and MNIST . Would like to see results on FFHQ or other ones that are used more often by prospective actors. \n- The paper only tests on one architecture DCGAN. I would like to see results on StyleGAN/StyleGANv2 architecture since its much more helpful for those models to attributable due to their widespread use. It also opens up some interesting scientific questions about which latent space the keys should be embedded in or if the multiscale training impedes this attribution regime. \n- The low resolution of the datasets \n- Figures & tables can be difficult to read\n\nMisc fixes / Questions:\n================================================================\n- The furthest left column for Table 1 and 2 are useless since you only test on one architecture. Consider a multicell label for readability.\n- Table II should have more labels into which values of the table are distinguishability and which are FID.\n- Typo: higer Table II\n- Unlabeled graph next to Algorithm I has axises that are difficult to read.\n- Notations for Figure 2 (c) are difficult to read (too small).\n- Are the red lines in Figure 2 (b) suppose to be best fit lines?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2Rcpx_0gk1Y", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nFake content produced by generative models is of great concerns. This paper investigates attribution techniques to identify models that generated the content. The key theoretic result is the derivation of the sufficient conditions for decentralized attribution and the design of keys following these conditions. Thee paper shows that decentralized attribution can be achieved when keys are orthogonal to each other, and belonging to a subspace determined by the data distribution. Results are validated on two datasets, MNIST and CelebA. \n\nStrengths\n\nThe paper derives the sufficient conditions for decentralized attribution of models generated the content. The idea is to add a uniform and bounded perturbation. The theory is on the data geometry-dependent threshold. There is a tradeoff between distinguishability and generation quality.\n\nWeaknesses\n\nThe decentralized attribution problem is not clearly motivated. What are the use cases? Why centralized attribution does not work for these use cases? \n\nThe attack model is not clear from the paper. Specifically, who is the attacker and what is the capability? It seems that the attacker has access to the original generative model. It is not clear why the user-end models need to be retrained. Is this trained by the attacker?\n\nThe application settings are not clear.  There are many questions on the motivating example \"A company develops a GAN model for image post-processing. A third-party organization (the registry) assigns keys to the company, who is then required to embed a watermark to the GAN models according to the keys for users to download. With the keys, the registry can trace the GAN generated images back to the user-end models.\" What kind of postprocessing? Are these identify preserving changes or generating new faces? Why is there a need for the watermark to depend on user keys?\n\nIn robust training, the paper discusses five types of post-processes: blurring, cropping, noise, JPEG conversion and the combination of these four. Are these necessary constraints on what the GAN models can do in order for the attribution techniques to work?\n\nThe derivation is on the sufficient conditions. What is the necessary conditions?\n\nMinor comments\n\nFigure 1 is not referenced anywhere in the paper. \n\nFigure 2 caption has no information on what the two rows are. I suppose one is on MNIST and the other is on CelebA. \n\nDecision\n\nThe paper seems to present interesting results on decentralized attribution of generative models. However, the paper poorly motivates the problem, use cases, capabilities and limitations of the approach. I can not recommend accept at this point. It is very possible I do not have the context to appreciate the paper. I hope the rebuttal process can make things clear. \n\n\n=====POST-REBUTTAL COMMENTS======== \n\nI applaud the authors for the very detailed response and the efforts in the updated draft. Many of my questions were clarified.  However, there are still important questions remaining.\n\n1. In addressing my Q4, \"We do believe that a separate discussion is needed on whether the registry or the user-end devices should retrain the models, given that users can potentially be the attackers in the malicious personation case. This discussion will entail an exploration of the tradeoff between training efficiency and training security of generative models, which is beyond the scope of this paper.\" This raises the feasibility question of decentralized approach proposed in this paper.\n\n2. In addressing my Q6, \"We note that just like problem settings for adversarial attacks, there is a potential mismatch between our expectation of the attackers' capability and their actual capability. Therefore, we cannot assess whether these constraints are necessary, but we acknowledge that they are not sufficient.\" This needs more investigation.\n\n3. Additionally, in addressing R3, \"Lastly, we agree that it is definitely interesting to understand which latent space the keys should be embedded. So far on FFHQ, attributability is achieved for 20 keys derived through the proposed method. More keys on the way (see wall-clock costs for computing keys and models in the supplementary). All of the keys share the same structure (eye shadows and lip stains), which may suggest a limit space for attributable keys. We polished the discussion on approximating the capacity of keys through an optimal packing problem, which is an open challenge.\" \nIf the number of keys are limited, this seriously impacts the contribution of this paper.\n\nOverall, I am more positive. I am willing to raise my score to 6. However, the paper is still somewhat borderline.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a decentralized attribution method to distinguish the generative models trained on the sample dataset. The key theoretic result is the derivation of the sufficient conditions for decentralized attribution and the design of keys following these conditions. Results are validated on two datasets with several generative models. The work is very interesting.
R1: Overall, I am more positive. I am willing to raise my score to 6. However, the paper is still somewhat borderline.
R3: Given the rebuttal, I am willing to raise my score to a 6 due to the added StyleGAN, PGAN, and other experiments and improved paper layout / clarity. The added experiments are welcome addition to the paper and demonstrate this technique. The lip and eyestaining are interesting results and I do hope this direction gets explored in the future.

Decision: Accept (Poster)