SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We apply the framework of [35] to understand our learning of  $S$  and the reason why  $S$  would possess P1. Fig. 2(a) illustrates easy and hard samples in our problem: patterns of face identities can be observed in some samples (easy samples), while other samples (hard samples) exhibit diverse appearance. Similar to [35], Fig. 2(b) shows that these easy and hard face samples tend to have high and low likelihood under  $T$ . Fig. 2(c) shows the learning of  $S$  on these easy and hard samples at different epochs. Consistent with the "DNNs Learn Patterns First" finding in [35],  $S$  learns general identity patterns first to fit the easy samples. Therefore,  $P_{S}$  of easy samples improve at a faster pace in the training, and many of them achieve high  $P_{S}$ . As easy samples tend to have high  $P_{T}$ , we observe P1 in  $S$ . For the hard samples (which tend to have low  $P_{T}$ ), it is uncommon for  $S$  to achieve high likelihood on them as they do not fit easily to the pattern learned by  $S$ .
In this section, we present extensive experiment results and ablation studies: (i) We show that our proposed T-ACGAN can lead to better surrogate models compared to alternative approaches (Sec. 6.2). (ii) We show that our proposed approach LOKT can significantly outperform the existing SOTA label-only MI attack (Sec. 6.3). (iii) We present additional results (Sec. 6.4) to demonstrate the efficacy of LOKT against SOTA MI defense methods. We further show that LOKT compares favorably in terms of query budget compared to existing SOTA. Additional experiments/analysis provided in Supplementary.
To ensure a fair comparison, we adopt the exact experimental setup used in BREPMI [6]. In what follows, we provide details of the experimental setup.
Dataset. We use three datasets, namely CelebA [40], Facescrub [41], and Pubfig83 [42]. We further study Label-Only MI attacks under distribution shift using FFHQ dataset [43] which contains images that vary in terms of background, ethnicity, and age. Following [2, 6], we divide each dataset (CelebA/ Facescrub/ Pubfig83) into two non-overlapping sets: private set  $\mathcal{D}_{priv}$  for training the target model  $T$ , and public set  $\mathcal{D}_{pub}$  for training GAN/T-ACGAN. More details on datasets and attacked identities can be found in Supplementary.
Table 1: Details of target model  $T$ . To showcase the effectiveness of our proposed method, we conduct a comprehensive set of 30 experiments, covering 10 different setups.
Dpriv	T
Architecture	# classes
CelebA	FaceNet64 [36]
IR152 [37]
VGG16 [38]	1,000
BiDO-HSIC [39]
MID [9]
Facescrub	FaceNet64 [36]	200
Pubfig83	FaceNet64	50
Target Models. Following [2, 6], we use 3 target models  $T$  including VGG16 [38], IR152 [37], and FaceNet64 [36]. All target models are provided in [2, 6].
$\dagger$  Fig. 1(e) are  $P_T$  and  $P_S$  of  $x_{f} = G(z,y)$  from our T-ACGAN. More details in Supp.
Face Embeddings
(a) Easy & Hard samples
Easy
(b)  $T$  behaviour under Easy & Hard sample
(c) Training Dynamics of Easy & Hard samples
Figure 2: We apply the framework of [35] to analyze learning dynamics of  $S$  to reason why  $S$  possesses property P1, and therefore could be an effective proxy for  $T$  under MI. We analyze generated samples  $x_{f}$  from our T-ACGAN for 3 identities (IDs 20, 16, 36). Note that  $x_{f}$  analysis is relevant as generated samples are used in MI attacks. (a): We analyze face embeddings of  $x_{f}$  extracted from publicly available SOTA face recognition model here. Different clusters and different distances from cluster centroids can be observed, suggesting patterns of face identities in some samples (easy samples) while diverse appearance in other samples (hard samples). We use distances from centroids to identify easy samples  $x_{f}^{e}$  and hard samples  $x_{f}^{h}$  (easy samples are indicated using transparent blue circle for each ID in the visualization). Visualization of  $x_{f}^{e}$  and  $x_{f}^{h}$  in image space further demonstrates identity patterns in  $x_{f}^{e}$  and diverse appearance in  $x_{f}^{h}$ . (b): Similar to [35], we observe that  $x_{f}^{e}$  and  $x_{f}^{h}$  tend to have high and low likelihood under  $T(P_{T})$  resp (training data). (c): We track likelihood under  $S(P_{S})$  for  $x_{f}^{e}$  and  $x_{f}^{h}$  during the training of  $S$ . As training progresses,  $P_{S}$  of  $x_{f}^{e}$  and  $x_{f}^{h}$  improve, and samples move up vertically (note that  $P_{T}$  of samples do not change). Consistent with the "DNNs Learn Patterns First" finding in [35],  $S$  learns general identity patterns first to fit the easy samples. Therefore,  $P_{S}$  of  $x_{f}^{e}$  improve at a faster pace in the training, and many of them achieve high  $P_{S}$  at epoch = 200. As  $x_{f}^{e}$  tend to have high  $P_{T}$ , we observe property P1 in  $S$ . For  $x_{f}^{h}$  (many of them tend to have low  $P_{T}$ ), it is uncommon for  $S$  to achieve high likelihood on them as they do not fit easily to the pattern learned by  $S$ . See Supplementary for additional details and analysis. Best viewed in color.
Additionally, we use the following methods/ models for evaluating the attack performance under SOTA MI defense methods:  $\bullet$  BiDO-HSIC [39] $\ddagger$ .  $\bullet$  MID [9] $\S$ . The details are included in Table 1.
Evaluation Metrics. Following [6, 11, 2], we use the following metrics to quantitatively evaluate the performance of MI attacks. Further, we also conduct user studies to assess the quality of reconstructed data (Sec. 6.5).
- Attack Accuracy (Attack acc.): Following [1, 2, 6], we utilize an evaluation model,  $E$ , which employs a distinct architecture and is trained on  $\mathcal{D}_{pri}^{\mathbb{I}}$ .  $E$  serves as a proxy for human inspection [1]. Higher attack accuracy indicates superior performance.
- KNN Distance (KNN dt.): The KNN distance indicates the shortest distance between the reconstructed image of a specific identity and its private images. Specifically, the shortest distance is computed using the  $l_{2}$  distance in the feature space, using the evaluation model's penultimate layer.
$\ddagger$ https://github.com/AlanPeng0897/Defend_MI
$\S$  https://github.com/Jiachen-T-Wang/mi-defense
$\S$  Following previous work,  $E$  can also be trained on  $\mathcal{D}_{pri}$  and samples from additional identities. This could improve generalization of  $E$  for accurate evaluation.
A smaller KNN distance signifies that the reconstructed images are more closely aligned with the private images of the target identity.
6.2 Training surrogate model with different algorithms
In this section, we demonstrate that our proposed T-ACGAN can lead to better surrogate models for MI. We describe a set of alternative approaches that can be used to train surrogate models using  $\mathcal{D}_{pub}$  and compare the performance of these approaches with our proposed method. Specifically, we consider a set of five algorithms, which can be broadly classified into three categories, for learning the surrogate model  $S$ :
- Directly use the public dataset  $\mathcal{D}_{pub}$ . We present two methods to train  $S$ :  $\bullet$  Direct I. We train  $S$  using the public dataset labelled with target model, i.e.  $\tilde{\mathcal{D}}_{pub}$  with samples  $(x_p, \tilde{y})$ ,  $x_p \in \mathcal{D}_{pub}$ ,  $\tilde{y} = T(x_p)$ ; see Sec. 4.1.  $\bullet$  Direct II. We apply data augmentation to  $\tilde{\mathcal{D}}_{pub}$  of Direct I to reduce the class imbalance in Direct I, followed by training  $S$  using the newly more balanced dataset.
- Training an ACGAN. We provide two versions:  $\bullet$  ACGAN I. We train an ACGAN model on  $\tilde{\mathcal{D}}_{pub}$  used in Direct I.  $\bullet$  ACGAN II. We train an ACGAN model on augmented  $\tilde{\mathcal{D}}_{pub}$  used in Direct II. As  $C\circ D$  in ACGAN serves as a classifier, we use  $C\circ D$  for MI attacks.
- Training proposed T-ACGAN. We use our proposed method described in Section 4.3 to train T-ACGAN. Similar to ACGAN I and II, we use  $C \circ D$  after training T-ACGAN for the attack.
For this comparison, we utilize the following settings:  $T =$  FaceNet64,  $\mathcal{D}_{priv} =$  CelebA,  $\mathcal{D}_{pub} =$  CelebA. Both ACGAN and T-ACGAN adopt the SNResnet architecture [34, 44]. To ensure a fair comparison, we use the same architecture as  $C \circ D$  in ACGAN and T-ACGAN for the surrogate model  $S$  in Direct I and Direct II. Detailed architecture specifications can be found in the Supplementary. After training the models, we employ the widely-used KEDMI [2] as the white-box attack on the trained surrogate models. Table 2 presents the results. The effectiveness of T-ACGAN in training surrogate models for MI attacks can be observed.
Table 2: We compare different approaches to train surrogate model for MI attacks. We utilize the following settings:  $T =$  FaceNet64,  ${\mathcal{D}}_{priv} =$  CelebA,  ${\mathcal{D}}_{pub} =$  CelebA,and employ the KEDMI[2] for MI attacks.
Algorithm	Attack acc. ↑	KNN dt. ↓
Direct I	5.87 ± 1.65	1936.12
Direct II	9.60 ± 2.22	1890.16
ACGAN I	6.47 ± 2.15	1771.26
ACGAN II	7.87 ± 3.10	1785.20
T-ACGAN	42.07 ± 3.46	1473.99
6.3 Comparison against SOTA label-only MI attack
Standard MI attack setup. In this section, we present the results obtained from the standard attack setup on three datasets: CelebA, Facescrub, and Pubfig83, as detailed in Table 3. We evaluate three designs of surrogate: (i) We directly use  $C \circ D$  from our T-ACGAN as the surrogate model. The architecture of T-ACGAN can be found in the supplementary material. (ii) We utilize the synthetic data generated by  $G$  of our T-ACGAN and label it using the target classifier  $T$  to train another surrogate model, denoted as  $S =$  Densenet-161 [45]. (iii) We employ the same data as in (ii) to train an ensemble of surrogate models, denoted as  $S_{en}$ , using different architectures including Densenet-121, Densenet-161, and Densenet-169.
We compare our results with the state-of-the-art (SOTA) label-only MI attack BREPMI [6]. To conduct our attacks, we utilize white-box PLGMI [7] on the surrogate models. Since PLGMI performs attacks using a conditional GAN trained with white-box access of the target classifier, we replace it with our T-ACGAN, which becomes available for use after training the surrogate models.
Our proposed method LOKT demonstrates a significant improvement in Attack accuracy and KNN distance compared to the SOTA label-only MI attack BREPMI [6]. Our top 1 attack accuracies are better than BREPMI from  $17.2\%$  to  $29.87\%$  across all setups when we utilize the ensemble  $S_{en}$ .
Fig. 1 (f) presents a visual comparison of various methods under the setup  $\mathcal{D}_{priv} = \mathrm{CelebA}$ ,  $\mathcal{D}_{pub} = \mathrm{CelebA}$ . More results are available in the Supplementary. Results clearly indicate that LOKT produces images that are closer to the ground truth (private data) compared to BREPMI [6]. This outcome provides strong evidence of the effectiveness of our approach in generating realistic images that closely resemble private data, which is critical for conducting successful MI attacks.
MI attacks under large distribution shift. Table 3 compares the MI attack results in the large distribution shift setup, where we use  $\mathcal{D}_{pub} = \mathrm{FFHQ}$ ,  $\mathcal{D}_{priv} = \mathrm{CelebA}/$  Facescrub/ Pubfig83, and
Table 3: We conduct comprehensive experiments to compare our proposed method LOKT and existing SOTA BREPMI [6] across standard MI attack benchmarks. Specifically, we evaluate the performance of our three proposed designs of surrogate, namely  $C \circ D$ ,  $S$ , and  $S_{en}$ , while BREPMI performs black-box search on  $T$  directly. We highlight the best results in each setup in bold.
Setup	Attack	Attack acc. ↑	KNN dt. ↓	Setup	Attack	Attack acc. ↑	KNN dt. ↓
T = FaceNet64	BREPMI	73.93 ± 4.98	1284.41	T = FaceNet64	BREPMI	55.60 ± 4.34	1012.83
Dpriv = CelebA	C ∘ D	81.00 ± 4.79	1298.63	Dpriv = Pubfig83	C ∘ D	74.80 ± 5.93	924.58
Dpub = CelebA	LOKT S	92.80 ± 2.59	1207.25	Dpub = Pubfig83	LOKT S	61.60 ± 3.58	993.44
Sen	93.93 ± 2.78	1181.72	Sen	80.00 ± 3.16	883.52
T = IR152	BREPMI	71.47 ± 5.32	1277.23	T = FaceNet64	BREPMI	72.80 ± 3.90	971.51
Dpriv = CelebA	C ∘ D	72.07 ± 4.03	1358.94	Dpriv = Pubfig83	C ∘ D	85.60 ± 2.61	914.15
Dpub = CelebA	LOKT S	89.80 ± 2.33	1220.00	Dpub = FFHQ	LOKT S	88.40 ± 2.97	920.99
Sen	92.13 ± 2.06	1206.78	Sen	94.40 ± 3.85	862.24
T = VGG16	BREPMI	57.40 ± 4.92	1376.94	T = FaceNet64	BREPMI	40.20 ± 6.60	1236.4
Dpriv = CelebA	C ∘ D	71.33 ± 4.39	1364.47	Dpriv = Facescrub	C ∘ D	45.70 ± 4.00	1296.29
Dpub = CelebA	LOKT S	85.60 ± 3.03	1252.09	Dpub = Facescrub	LOKT S	53.20 ± 5.29	1280.70
Sen	87.27 ± 1.97	1246.71	Sen	58.60 ± 4.86	1225.13
T = FaceNet64	BREPMI	43.00 ± 5.14	1470.55	T = FaceNet64	BREPMI	37.30 ± 3.99	1456.59
Dpriv = CelebA	C ∘ D	43.27 ± 3.53	1516.18	Dpriv = Facescrub	C ∘ D	44.50 ± 5.98	1403.73
Dpub = FFHQ	LOKT S	59.13 ± 2.77	1437.86	Dpub = FFHQ	LOKT S	47.20 ± 4.39	1404.85
Sen	62.07 ± 3.89	1428.04	Sen	53.70 ± 4.57	1338.67
Table 4: We report Label-only MI Attack results under SOTA defense models namely BiDO [39] and MID [9]. We use  $\mathcal{D}_{priv} = \mathrm{CelebA}$ ,  $\mathcal{D}_{pub} = \mathrm{CelebA}$ . We highlight the best results in bold.
Setup	Attack	Attack acc. ↑	KNN dt. ↓
T = BiDO [39]	BREPMI[6]	37.40 ± 3.66	1500.45
Dpriv = CelebA	C ∘ D	45.73 ± 5.94	1493.48
Dpub = CelebA	LOKT S	58.53 ± 4.87	1427.22
Sen	60.73 ± 3.07	1395.93
T = MID [9]	BREPMI[6]	39.20 ± 4.19	1458.61
Dpriv = CelebA	C ∘ D	44.13 ± 3.54	1475.73
Dpub = CelebA	LOKT S	55.33 ± 4.40	1393.76
Sen	60.33 ± 4.76	1374.34
Table 5: The comparison of the number of queries (in millions) used by LOKT and BREPMI [6]. The attacks using  $S$  and  $S_{en}$  consume additional 500k queries comparing to  $C \circ D$  to label the synthetic images to train  $S$  and  $S_{en}$ . Our results show that we use fewer number of queries than BREPMI in all setups.
T	LOKT C∘D	LOKT S/Se n	BREPMI
FaceNet64	12.16	12.66	17.98
IR152	12.16	12.66	18.06
VGG16	12.16	12.66	18.12
BiDO-HSIC	12.16	12.66	18.39
MID	12.16	12.66	18.25
$T =$  FaceNet64. The attack results of BREPMI drop significantly (by  $30.93\%$  for CelebA and  $2.9\%$  for Facescrub), while the results for Pubfig83 notably increase, which can be attributed to the small size of the Pubfig83 dataset [6]. Our proposed method outperforms BREPMI, with the top 1 attack accuracies increase from  $16.40\%$  to  $21.60\%$  for all setups. Moreover, the KNN distance indicates that our reconstructed images are closer to the private data than those reconstructed by BREPMI.
6.4 Additional results
MI attack results using MI defense model. We investigate the attacks on the MI defense model (see Table 4). Specifically, we utilize the SOTA defense model BiDO-HSIC [39] and MID [9]. Our results indicate that BiDO-HSIC successfully reduce the effectiveness of the white-box SOTA attack, PLGMI, by  $9.57\%$  (See the result in the Supplementary). In the label-only setup, the performance of BREPMI becomes relatively low with attack accuracy of only  $37.40\%$  for BiDO-HSIC [39] and  $39.20\%$  for MID [9]. In contrast, our approach achieves a much higher attack accuracy of  $60.73\%$  and  $60.33\%$ , almost doubling the performance of BREPMI. These results demonstrate that our approach is effective in conducting MI attacks on MI defense models, even in scenarios where the adversary has limited information about the target classifier.
High resolution. We conduct the experiment with high resolution images which has not been addressed yet for label-only setup [6]. In particular, we train a new target classifier  $T =$  Resnet-152 using CelebA setup with the image size  $= 128 \times 128$ . For fair comparison between BREPMI and our proposed method, T-ACGAN has the same GAN architectures used by BREPMI. The details of the architecture can be found in the Supplementary.
The results are shown in Table 6. LOKT outperforms BREPMI, with top 1 accuracy surpassing BREPMI by  $20.27\%$ . Our inverted images are closer to private training samples than BREPMI
Table 6: We conduct the experiment with higher resolution images. We use  $T =$  Resnet-152,  ${\mathcal{D}}_{priv}$  =CelebA,  ${\mathcal{D}}_{pub} =$  CelebA, image size  $= {128} \times  {128}$  . The natural accuracy of  $T$  is  ${86.07}\%$  . We highlight the best results in bold.
Setup	Attack	Attack acc. ↑	KNN dt. ↓
T = IR152	BREPMI[6]	50.33 ± 4.71	1389.09
Dpriv = CelebA	C ∘ D	66.87 ± 3.93	1356.53
Dpub = CelebA	LOKT	S	66.80 ± 3.83	1341.04
Seon	70.60 ± 4.43	1320.16
Table 7: User study results. Our human study reveals that users distinctly favor our approach, with  $64.30\%$  user preference for images reconstructed using our proposed approach, LOKT, compared to BREPMI's lower  $35.70\%$  user preference.
Method	User Preference (↑)
BREPMI	35.70%
LOKT	64.30%
(smaller KNN distance). We believe our study can provide new insight on the effectiveness of SOTA label-only attack at a higher resolution of  $128 \times 128$ , paving the way to future label-only model inversion attacks at resolutions beyond  $128 \times 128$ .
Query budget. In this experiment, we compare query budget between our proposed method and BREPMI [2]. In the BREPMI, queries to the target classifier  $T$  are required to identify the initial points for attacking and estimate the gradients during the attack. In our method, queries to  $T$  are required to label the synthetic data during the training of T-ACGAN to obtain  $C \circ D$ , and additional 500k queries to label generated images of T-ACGAN to train  $S$  and the ensemble  $S_{en}$ . For comparison, as shown in Table 5, we use  $\mathcal{D}_{priv} = \text{CelebA}$  and  $\mathcal{D}_{pub} = \text{CelebA}$ . The results show that our proposed method requires 30% fewer queries compared to BREPMI.
6.5 User study
User study setup. In this section, we go beyond objective metrics and consider subjective evaluation of MI attacks. In particular, we conduct a human study to understand the efficacy of our proposed method, LOKT, compared to BREPMI. We follow the setup by [10] for human study and use Amazon Mechanical Turk (MTurk) for experiments. The user interface is provided in the Supplementary. In this study, users are shown 5 real images of a person (identity) as reference. Then users are required to compare the 5 real images with two inverted images: one from our method (LOKT), the other from BREPMI. We use  $D_{priv} = \text{CelebA}$ ,  $D_{pub} = \text{CelebA}$  and  $T = \text{FaceNet64}$ . Following [10], we randomly selected 50 identities with 10 unique users evaluating each task accounting to 1000 comparison pairs.
User study results. We report the user study results in Table 7. Our human study reveals that users distinctly favor our approach, with  $64.30\%$  user preference for images reconstructed using our proposed approach, in contrast to BREPMI's lower  $35.70\%$  user preference. These subjective evaluations further show the efficacy of our proposed method, LOKT, in the challenging label-only MI setup.
Conclusion. Instead of performing a black-box search approach as in existing SOTA, we propose a new label-only MI approach (LOKT) by transferring decision knowledge from the target model to surrogate models and performing white-box attacks on the surrogate models. To obtain the surrogate models, we propose a new T-ACGAN to leverage generative modeling and the target model for effective knowledge transfer. Using findings of general learning dynamics of DNNs, we conduct analysis to support that our surrogate models are effective proxies for the target model under MI. We perform extensive experiments and ablation to support our claims and demonstrate significant improvement over existing SOTA.
Broader Impacts. Understanding model inversion attacks holds significance as AI models continue to see widespread deployment across various applications. By studying and understanding the approaches and methodologies for model inversion, researchers can develop good practices in deploying AI models and robust defense mechanisms for different applications esp. those involving sensitive training data. It is important to emphasize that the objective of model inversion research is to raise awareness of potential privacy threats and bolster our collective defenses.
Limitations. While our experiments are extensive compared to previous works, practical applications involve different types of private training datasets such as healthcare data. Nevertheless, our assumptions are general, and we believe our findings can be applied to a broader range of applications.
Acknowledgements. This research is supported by the National Research Foundation, Singapore under its AI Singapore Programmes (AISG Award No.: AISG2-TC-2022-007) and SUTD project PIE-SGP-AI-2018-01. This research work is also supported by the Agency for Science, Technology and Research (A*STAR) under its MTC Programmatic Funds (Grant No. M23L7b0021). This material is based on the research/work support in part by the Changi General Hospital and Singapore University of Technology and Design, under the HealthTech Innovation Fund (HTIF Award No. CGH-SUTD-2021-004). We thank anonymous reviewers for their insightful feedback.
[32] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
[33] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
[34] Takeru Miyato and Masanori Koyama. cGANs with projection discriminator. In International Conference on Learning Representations, 2018.
[35] Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization in deep networks. In ICML, 2017.
[36] Yu Cheng, Jian Zhao, Zhecan Wang, Yan Xu, Karlekar Jayashree, Shengmei Shen, and Jiashi Feng. Know you at one glance: A compact vector representation for low-shot learning. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1924-1932, 2017.
[37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[38] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[39] Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han. Bilateral dependency optimization: Defending against model-inversion attacks. In KDD, 2022.
[40] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730-3738, 2015.
[41] Hong-Wei Ng and Stefan Winkler. A data-driven approach to cleaning large face datasets. In 2014 IEEE international conference on image processing (ICIP), pages 343–347. IEEE, 2014.
[42] Nicolas Pinto, Zak Stone, Todd Zickler, and David Cox. Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook. In CVPR 2011 WORKSHOPS, pages 35-42. IEEE, 2011.
[43] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019.
[44] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations, 2018.
[45] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700-4708, 2017.