Package: pytorch-gan-metrics
Pytorch Implementation of Common GAN metricsInstallpip install pytorch-gan-metricstorch>=1.8.2torchvision>=0.9.2Quick Startfrompytorch_gan_metricsimportget_inception_score,get_fidimages=...# [N, 3, H, W] normalized to [0, 1]IS,IS_std=get_inception_score(images)# Inception ScoreFID=get_fid(images,'path/to/statistics.npz')# Frechet Inception Distancepath/to/statistics.npzis compatiable withofficialFID implementation.NotesThe FID implementation is inspired frompytorch-fid.This repository is developed for personal research. If you think this package can also benefit your life, please feel free to open issues.FeaturesCurrently, this package supports following metrics:Inception Score(IS)Fréchet Inception Distance(FID)The computation procedure of IS and FID are integrated to avoid multiple forward propagations.Support reading images on the fly to avoid out of memory especially for large scale images.Support computation on GPU to speed up some cpu operations such asnp.covandscipy.linalg.sqrtm.Reproducing Results of Official Implementations on CIFAR-10Train ISTest ISTrain(50k) vs Test(10k)FIDOfficial11.24±0.2010.98±0.223.1508pytorch-gan-metrics11.26±0.1410.96±0.353.1518pytorch-gan-metricsuse_torch=True11.26±0.1510.96±0.193.1509The results are slightly different from official implementations due to the framework difference between PyTorch and TensorFlow.DocumentationPrepare Statistics (for FID)Downloadprecalculated statistics orCalculate statistics for your custom dataset using command line toolpython-mpytorch_gan_metrics.calc_fid_stats\--pathpath/to/images\--statspath/to/statistics.npzSeecalc_fid_stats.pyfor details.Inception FeaturesWhen getting IS or FID, theInceptionV3will be loaded intotorch.device('cuda:0')if GPU is availabel; Otherwise,torch.device('cpu')will be used.Changedeviceargument inget_*functions to set torch device.Usingtorch.Tensoras imagesPrepare images in typetorch.float32with shape[N, 3, H, W]and normalized to[0,1].frompytorch_gan_metricsimport(get_inception_score,get_fid,get_inception_score_and_fid)images=...# [N, 3, H, W]assert0<=images.min()andimages.max()<=1# Inception ScoreIS,IS_std=get_inception_score(images)# Frechet Inception DistanceFID=get_fid(images,'path/to/statistics.npz')# Inception Score & Frechet Inception Distance(IS,IS_std),FID=get_inception_score_and_fid(images,'path/to/statistics.npz')Using PyTorch DataLoader to Provide ImagesUsepytorch_gan_metrics.ImageDatasetto collect images on your storage or use your customtorch.utils.data.Dataset.frompytorch_gan_metricsimportImageDatasetdataset=ImageDataset(path_to_dir,exts=['png','jpg'])loader=DataLoader(dataset,batch_size=50,num_workers=4)It is possible to wrap a generative model in a dataset to support generating images on the fly. Remember to setnum_workers=0to avoid copying models across multiprocess.classGeneratorDataset(Dataset):def__init__(self,G,z_dim):self.G=Gself.z_dim=z_dimdef__len__(self):return50000def__getitem__(self,index):returnself.G(torch.randn(1,self.z_dim).cuda())[0]dataset=GeneratorDataset(G,z=128)loader=DataLoader(dataset,batch_size=50,num_workers=0)Calculate metricsfrompytorch_gan_metricsimport(get_inception_score,get_fid,get_inception_score_and_fid)# Inception ScoreIS,IS_std=get_inception_score(loader)# Frechet Inception DistanceFID=get_fid(loader,'path/to/statistics.npz')# Inception Score & Frechet Inception Distance(IS,IS_std),FID=get_inception_score_and_fid(loader,'path/to/statistics.npz')Load Images from a DirectoryCalculate metrics for images in a directory and its subfolders.frompytorch_gan_metricsimport(get_inception_score_from_directory,get_fid_from_directory,get_inception_score_and_fid_from_directory)IS,IS_std=get_inception_score_from_directory('path/to/images')FID=get_fid_from_directory('path/to/images','path/to/statistics.npz')(IS,IS_std),FID=get_inception_score_and_fid_from_directory('path/to/images','path/to/statistics.npz')Accelerating Matrix Computation by PyTorchSetuse_torch=Truewhen calling functionsget_*such asget_inception_score,get_fid, etc.WARNINGwhenuse_torch=Trueis used, the FID might benandue to the unstable implementation of matrix sqrt.This option is recommended to be used when evaluating generative models on a server which is equipped with high efficiency GPUs while the cpu frequency is low.Tested Versionspython 3.9 + torch 1.8.2 + CUDA 10.2python 3.9 + torch 1.11.0 + CUDA 10.2python 3.9 + torch 1.12.1 + CUDA 10.2LicenseThis implementation is licensed under the Apache License 2.0.This implementation is derived frompytorch-fid, licensed under the Apache License 2.0.FID was introduced by Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler and Sepp Hochreiter in "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium", seehttps://arxiv.org/abs/1706.08500The original implementation of FID is by the Institute of Bioinformatics, JKU Linz, licensed under the Apache License 2.0.
Seehttps://github.com/bioinf-jku/TTUR.