Quality Evaluation of GANs Using Cross Local Intrinsic Dimensionality
Authors: Sukarna Barua, Xingjun Ma, Sarah Monazam Erfani, Michael Houle, James Bailey
Keywords: Generative Adversarial Networks, Evaluation Metric, Local Intrinsic Dimensionality
TL;DR: We propose a new metric for evaluating GAN models.
Abstract: Generative Adversarial Networks (GANs) are an elegant mechanism for data generation.  However, a key challenge when using GANs is how to best measure their ability to generate realistic data. In this paper, we demonstrate that an intrinsic dimensional characterization of the data space learned by a GAN model leads to an effective evaluation metric for GAN quality.  In particular, we propose a new evaluation measure, CrossLID, that assesses the local intrinsic dimensionality (LID) of input data with respect to neighborhoods within GAN-generated samples.  In experiments on 3 benchmark image datasets, we compare our proposed measure to several state-of-the-art evaluation metrics. Our experiments show that CrossLID is strongly correlated with sample quality, is sensitive to mode collapse, is robust to small-scale noise and image transformations, and can be applied in a model-free  manner.  Furthermore, we show how CrossLID can be used within the GAN training process to improve generation quality.