Package: jingan
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio===0.7.2 -fhttps://download.pytorch.org/whl/torch_stable.htmlpip install click requests tqdm pyspng ninja imageio-ffmpeg==0.4.3conda install  psutil -yconda install scipy -yStyleGAN2-ADA — Official PyTorch implementationTraining Generative Adversarial Networks with Limited DataTero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Ailahttps://arxiv.org/abs/2006.06676Abstract:Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes. The approach does not require changes to loss functions or network architectures, and is applicable both when training from scratch and when fine-tuning an existing GAN on another dataset. We demonstrate, on several datasets, that good results are now possible using only a few thousand training images, often matching StyleGAN2 results with an order of magnitude fewer images. We expect this to open up new application domains for GANs. We also find that the widely used CIFAR-10 is, in fact, a limited data benchmark, and improve the record FID from 5.59 to 2.42.For business inquiries, please visit our website and submit the form:NVIDIA Research LicensingRelease notesThis repository is a faithful reimplementation ofStyleGAN2-ADAin PyTorch, focusing on correctness, performance, and compatibility.CorrectnessFull support for all primary training configurations.Extensive verification of image quality, training curves, and quality metrics against the TensorFlow version.Results are expected to match in all cases, excluding the effects of pseudo-random numbers and floating-point arithmetic.PerformanceTraining is typically 5%–30% faster compared to the TensorFlow version on NVIDIA Tesla V100 GPUs.Inference is up to 35% faster in high resolutions, but it may be slightly slower in low resolutions.GPU memory usage is comparable to the TensorFlow version.Faster startup time when training new networks (<50s), and also when using pre-trained networks (<4s).New command line options for tweaking the training performance.CompatibilityCompatible with old network pickles created using the TensorFlow version.New ZIP/PNG based dataset format for maximal interoperability with existing 3rd party tools.TFRecords datasets are no longer supported — they need to be converted to the new format.New JSON-based format for logs, metrics, and training curves.Training curves are also exported in the old TFEvents format if TensorBoard is installed.Command line syntax is mostly unchanged, with a few exceptions (e.g.,dataset_tool.py).Comparison methods are not supported (--cmethod,--dcap,--cfg=cifarbaseline,--aug=adarv)Truncation is now disabled by default.Data repositoryPathDescriptionstylegan2-ada-pytorchMain directory hosted on Amazon S3├ada-paper.pdfPaper PDF├imagesCurated example images produced using the pre-trained models├videosCurated example interpolation videos└pretrainedPre-trained models├  ffhq.pklFFHQ at 1024x1024, trained using original StyleGAN2├  metfaces.pklMetFaces at 1024x1024, transfer learning from FFHQ using ADA├  afhqcat.pklAFHQ Cat at 512x512, trained from scratch using ADA├  afhqdog.pklAFHQ Dog at 512x512, trained from scratch using ADA├  afhqwild.pklAFHQ Wild at 512x512, trained from scratch using ADA├  cifar10.pklClass-conditional CIFAR-10 at 32x32├  brecahad.pklBreCaHAD at 512x512, trained from scratch using ADA├paper-fig7c-training-set-sweepsModels used in Fig.7c (sweep over training set size)├paper-fig11a-small-datasetsModels used in Fig.11a (small datasets & transfer learning)├paper-fig11b-cifar10Models used in Fig.11b (CIFAR-10)├transfer-learning-source-netsModels used as starting point for transfer learning└metricsFeature detectors used by the quality metricsRequirementsLinux and Windows are supported, but we recommend Linux for performance and compatibility reasons.1–8 high-end NVIDIA GPUs with at least 12 GB of memory. We have done all testing and development using NVIDIA DGX-1 with 8 Tesla V100 GPUs.64-bit Python 3.7 and PyTorch 1.7.1. Seehttps://pytorch.org/for PyTorch install instructions.CUDA toolkit 11.0 or later.  Use at least version 11.1 if running on RTX 3090.  (Why is a separate CUDA toolkit installation required?  See comments in#2.)Python libraries:pip install click requests tqdm pyspng ninja imageio-ffmpeg==0.4.3.  We use the Anaconda3 2020.11 distribution which installs most of these by default.Docker users: use theprovided Dockerfileto build an image with the required library dependencies.The code relies heavily on custom PyTorch extensions that are compiled on the fly using NVCC. On Windows, the compilation requires Microsoft Visual Studio. We recommend installingVisual Studio Community Editionand adding it intoPATHusing"C:\Program Files (x86)\Microsoft Visual Studio\<VERSION>\Community\VC\Auxiliary\Build\vcvars64.bat".Getting startedPre-trained networks are stored as*.pklfiles that can be referenced using local filenames or URLs:# Generate curated MetFaces images without truncation (Fig.10 left)
python generate.py --outdir=out --trunc=1 --seeds=85,265,297,849 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metfaces.pkl

# Generate uncurated MetFaces images with truncation (Fig.12 upper left)
python generate.py --outdir=out --trunc=0.7 --seeds=600-605 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metfaces.pkl

# Generate class conditional CIFAR-10 images (Fig.17 left, Car)
python generate.py --outdir=out --seeds=0-35 --class=1 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/cifar10.pkl

# Style mixing example
python style_mixing.py --outdir=out --rows=85,100,75,458,1500 --cols=55,821,1789,293 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metfaces.pklOutputs from the above commands are placed underout/*.png, controlled by--outdir. Downloaded network pickles are cached under$HOME/.cache/dnnlib, which can be overridden by setting theDNNLIB_CACHE_DIRenvironment variable. The default PyTorch extension build directory is$HOME/.cache/torch_extensions, which can be overridden by settingTORCH_EXTENSIONS_DIR.Docker: You can run the above curated image example using Docker as follows:docker build --tag sg2ada:latest .
./docker_run.sh python3 generate.py --outdir=out --trunc=1 --seeds=85,265,297,849 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metfaces.pklNote: The Docker image requires NVIDIA driver releaser455.23or later.Legacy networks: The above commands can load most of the network pickles created using the previous TensorFlow versions of StyleGAN2 and StyleGAN2-ADA. However, for future compatibility, we recommend converting such legacy pickles into the new format used by the PyTorch version:python legacy.py \
    --source=https://nvlabs-fi-cdn.nvidia.com/stylegan2/networks/stylegan2-cat-config-f.pkl \
    --dest=stylegan2-cat-config-f.pklProjecting images to latent spaceTo find the matching latent vector for a given image file, run:python projector.py --outdir=out --target=~/mytargetimg.png \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/ffhq.pklFor optimal results, the target image should be cropped and aligned similar to theFFHQ dataset. The above command saves the projection targetout/target.png, resultout/proj.png, latent vectorout/projected_w.npz, and progression videoout/proj.mp4. You can render the resulting latent vector by specifying--projected_wforgenerate.py:python generate.py --outdir=out --projected_w=out/projected_w.npz \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/ffhq.pklUsing networks from PythonYou can use pre-trained networks in your own Python code as follows:with open('ffhq.pkl', 'rb') as f:
    G = pickle.load(f)['G_ema'].cuda()  # torch.nn.Module
z = torch.randn([1, G.z_dim]).cuda()    # latent codes
c = None                                # class labels (not used in this example)
img = G(z, c)                           # NCHW, float32, dynamic range [-1, +1]The above code requirestorch_utilsanddnnlibto be accessible viaPYTHONPATH. It does not need source code for the networks themselves — their class definitions are loaded from the pickle viatorch_utils.persistence.The pickle contains three networks.'G'and'D'are instantaneous snapshots taken during training, and'G_ema'represents a moving average of the generator weights over several training steps. The networks are regular instances oftorch.nn.Module, with all of their parameters and buffers placed on the CPU at import and gradient computation disabled by default.The generator consists of two submodules,G.mappingandG.synthesis, that can be executed separately. They also support various additional options:w = G.mapping(z, c, truncation_psi=0.5, truncation_cutoff=8)
img = G.synthesis(w, noise_mode='const', force_fp32=True)Please refer togenerate.py,style_mixing.py, andprojector.pyfor further examples.Preparing datasetsDatasets are stored as uncompressed ZIP archives containing uncompressed PNG files and a metadata filedataset.jsonfor labels.Custom datasets can be created from a folder containing images; seepython dataset_tool.py --helpfor more information. Alternatively, the folder can also be used directly as a dataset, without running it throughdataset_tool.pyfirst, but doing so may lead to suboptimal performance.Legacy TFRecords datasets are not supported — see below for instructions on how to convert them.FFHQ:Step 1: Download theFlickr-Faces-HQ datasetas TFRecords.Step 2: Extract images from TFRecords usingdataset_tool.pyfrom theTensorFlow version of StyleGAN2-ADA:# Using dataset_tool.py from TensorFlow version at
# https://github.com/NVlabs/stylegan2-ada/
python ../stylegan2-ada/dataset_tool.py unpack \
    --tfrecord_dir=~/ffhq-dataset/tfrecords/ffhq --output_dir=/tmp/ffhq-unpackedStep 3: Create ZIP archive usingdataset_tool.pyfrom this repository:# Original 1024x1024 resolution.
python dataset_tool.py --source=/tmp/ffhq-unpacked --dest=~/datasets/ffhq.zip

# Scaled down 256x256 resolution.
python dataset_tool.py --source=/tmp/ffhq-unpacked --dest=~/datasets/ffhq256x256.zip \
    --width=256 --height=256MetFaces: Download theMetFaces datasetand create ZIP archive:python dataset_tool.py --source=~/downloads/metfaces/images --dest=~/datasets/metfaces.zipAFHQ: Download theAFHQ datasetand create ZIP archive:python dataset_tool.py --source=~/downloads/afhq/train/cat --dest=~/datasets/afhqcat.zip
python dataset_tool.py --source=~/downloads/afhq/train/dog --dest=~/datasets/afhqdog.zip
python dataset_tool.py --source=~/downloads/afhq/train/wild --dest=~/datasets/afhqwild.zipCIFAR-10: Download theCIFAR-10 python versionand convert to ZIP archive:python dataset_tool.py --source=~/downloads/cifar-10-python.tar.gz --dest=~/datasets/cifar10.zipLSUN: Download the desired categories from theLSUN project pageand convert to ZIP archive:python dataset_tool.py --source=~/downloads/lsun/raw/cat_lmdb --dest=~/datasets/lsuncat200k.zip \
    --transform=center-crop --width=256 --height=256 --max_images=200000

python dataset_tool.py --source=~/downloads/lsun/raw/car_lmdb --dest=~/datasets/lsuncar200k.zip \
    --transform=center-crop-wide --width=512 --height=384 --max_images=200000BreCaHAD:Step 1: Download theBreCaHAD dataset.Step 2: Extract 512x512 resolution crops usingdataset_tool.pyfrom theTensorFlow version of StyleGAN2-ADA:# Using dataset_tool.py from TensorFlow version at
# https://github.com/NVlabs/stylegan2-ada/
python dataset_tool.py extract_brecahad_crops --cropsize=512 \
    --output_dir=/tmp/brecahad-crops --brecahad_dir=~/downloads/brecahad/imagesStep 3: Create ZIP archive usingdataset_tool.pyfrom this repository:python dataset_tool.py --source=/tmp/brecahad-crops --dest=~/datasets/brecahad.zipTraining new networksIn its most basic form, training new networks boils down to:python train.py --outdir=~/training-runs --data=~/mydataset.zip --gpus=1 --dry-run
python train.py --outdir=~/training-runs --data=~/mydataset.zip --gpus=1The first command is optional; it validates the arguments, prints out the training configuration, and exits. The second command kicks off the actual training.In this example, the results are saved to a newly created directory~/training-runs/<ID>-mydataset-auto1, controlled by--outdir. The training exports network pickles (network-snapshot-<INT>.pkl) and example images (fakes<INT>.png) at regular intervals (controlled by--snap). For each pickle, it also evaluates FID (controlled by--metrics) and logs the resulting scores inmetric-fid50k_full.jsonl(as well as TFEvents if TensorBoard is installed).The name of the output directory reflects the training configuration. For example,00000-mydataset-auto1indicates that thebase configurationwasauto1, meaning that the hyperparameters were selected automatically for training on one GPU. The base configuration is controlled by--cfg:Base configDescriptionauto(default)Automatically select reasonable defaults based on resolution and GPU count. Serves as a good starting point for new datasets but does not necessarily lead to optimal results.stylegan2Reproduce results for StyleGAN2 config F at 1024x1024 using 1, 2, 4, or 8 GPUs.paper256Reproduce results for FFHQ and LSUN Cat at 256x256 using 1, 2, 4, or 8 GPUs.paper512Reproduce results for BreCaHAD and AFHQ at 512x512 using 1, 2, 4, or 8 GPUs.paper1024Reproduce results for MetFaces at 1024x1024 using 1, 2, 4, or 8 GPUs.cifarReproduce results for CIFAR-10 (tuned configuration) using 1 or 2 GPUs.The training configuration can be further customized with additional command line options:--aug=noaugdisables ADA.--cond=1enables class-conditional training (requires a dataset with labels).--mirror=1amplifies the dataset with x-flips. Often beneficial, even with ADA.--resume=ffhq1024 --snap=10performs transfer learning from FFHQ trained at 1024x1024.--resume=~/training-runs/<NAME>/network-snapshot-<INT>.pklresumes a previous training run.--gamma=10overrides R1 gamma. We recommend trying a couple of different values for each new dataset.--aug=ada --target=0.7adjusts ADA target value (default: 0.6).--augpipe=blitenables pixel blitting but disables all other augmentations.--augpipe=bgcfncenables all available augmentations (blit, geom, color, filter, noise, cutout).Please refer topython train.py --helpfor the full list.Expected training timeThe total training time depends heavily on resolution, number of GPUs, dataset, desired quality, and hyperparameters. The following table lists expected wallclock times to reach different points in the training, measured in thousands of real images shown to the discriminator ("kimg"):ResolutionGPUs1000 kimg25000 kimgsec/kimgGPU memCPU mem128x12814h 05m4d 06h12.8–13.77.2 GB3.9 GB128x12822h 06m2d 04h6.5–6.87.4 GB7.9 GB128x12841h 20m1d 09h4.1–4.64.2 GB16.3 GB128x12881h 13m1d 06h3.9–4.92.6 GB31.9 GB256x25616h 36m6d 21h21.6–24.25.0 GB4.5 GB256x25623h 27m3d 14h11.2–11.85.2 GB9.0 GB256x25641h 45m1d 20h5.6–5.95.2 GB17.8 GB256x25681h 24m1d 11h4.4–5.53.2 GB34.7 GB512x512121h 03m21d 22h72.5–74.97.6 GB5.0 GB512x512210h 59m11d 10h37.7–40.07.8 GB9.8 GB512x51245h 29m5d 17h18.7–19.17.9 GB17.7 GB512x51282h 48m2d 22h9.5–9.77.8 GB38.2 GB1024x102411d 20h46d 03h154.3–161.68.1 GB5.3 GB1024x1024223h 09m24d 02h80.6–86.28.6 GB11.9 GB1024x1024411h 36m12d 02h40.1–40.88.4 GB21.9 GB1024x102485h 54m6d 03h20.2–20.68.3 GB44.7 GBThe above measurements were done using NVIDIA Tesla V100 GPUs with default settings (--cfg=auto --aug=ada --metrics=fid50k_full). "sec/kimg" shows the expected range of variation in raw training performance, as reported inlog.txt. "GPU mem" and "CPU mem" show the highest observed memory consumption, excluding the peak at the beginning caused bytorch.backends.cudnn.benchmark.In typical cases, 25000 kimg or more is needed to reach convergence, but the results are already quite reasonable around 5000 kimg. 1000 kimg is often enough for transfer learning, which tends to converge significantly faster. The following figure shows example convergence curves for different datasets as a function of wallclock time, using the same settings as above:Note:--cfg=autoserves as a reasonable first guess for the hyperparameters but it does not necessarily lead to optimal results for a given dataset. For example,--cfg=stylegan2yields considerably better FID  for FFHQ-140k at 1024x1024 than illustrated above. We recommend trying out at least a few different values of--gammafor each new dataset.Quality metricsBy default,train.pyautomatically computes FID for each network pickle exported during training. We recommend inspectingmetric-fid50k_full.jsonl(or TensorBoard) at regular intervals to monitor the training progress. When desired, the automatic computation can be disabled with--metrics=noneto speed up the training slightly (3%–9%).Additional quality metrics can also be computed after the training:# Previous training run: look up options automatically, save result to JSONL file.
python calc_metrics.py --metrics=pr50k3_full \
    --network=~/training-runs/00000-ffhq10k-res64-auto1/network-snapshot-000000.pkl

# Pre-trained network pickle: specify dataset explicitly, print result to stdout.
python calc_metrics.py --metrics=fid50k_full --data=~/datasets/ffhq.zip --mirror=1 \
    --network=https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/ffhq.pklThe first example looks up the training configuration and performs the same operation as if--metrics=pr50k3_fullhad been specified during training. The second example downloads a pre-trained network pickle, in which case the values of--mirrorand--datamust be specified explicitly.Note that many of the metrics have a significant one-off cost when calculating them for the first time for a new dataset (up to 30min). Also note that the evaluation is done using a different random seed each time, so the results will vary if the same metric is computed multiple times.We employ the following metrics in the ADA paper. Execution time and GPU memory usage is reported for one NVIDIA Tesla V100 GPU at 1024x1024 resolution:MetricTimeGPU memDescriptionfid50k_full13 min1.8 GBFréchet inception distance[1]against the full datasetkid50k_full13 min1.8 GBKernel inception distance[2]against the full datasetpr50k3_full13 min4.1 GBPrecision and recall[3]againt the full datasetis50k13 min1.8 GBInception score[4]for CIFAR-10In addition, the following metrics from theStyleGANandStyleGAN2papers are also supported:MetricTimeGPU memDescriptionfid50k13 min1.8 GBFréchet inception distance against 50k real imageskid50k13 min1.8 GBKernel inception distance against 50k real imagespr50k313 min4.1 GBPrecision and recall against 50k real imagesppl2_wend36 min2.4 GBPerceptual path length[5]in W, endpoints, full imageppl_zfull36 min2.4 GBPerceptual path length in Z, full paths, cropped imageppl_wfull36 min2.4 GBPerceptual path length in W, full paths, cropped imageppl_zend36 min2.4 GBPerceptual path length in Z, endpoints, cropped imageppl_wend36 min2.4 GBPerceptual path length in W, endpoints, cropped imageReferences:GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, Heusel et al. 2017Demystifying MMD GANs, Bińkowski et al. 2018Improved Precision and Recall Metric for Assessing Generative Models, Kynkäänniemi et al. 2019Improved Techniques for Training GANs, Salimans et al. 2016A Style-Based Generator Architecture for Generative Adversarial Networks, Karras et al. 2018LicenseCopyright © 2021, NVIDIA Corporation. All rights reserved.This work is made available under theNvidia Source Code License.Citation@inproceedings{Karras2020ada,
  title     = {Training Generative Adversarial Networks with Limited Data},
  author    = {Tero Karras and Miika Aittala and Janne Hellsten and Samuli Laine and Jaakko Lehtinen and Timo Aila},
  booktitle = {Proc. NeurIPS},
  year      = {2020}
}DevelopmentThis is a research reference implementation and is treated as a one-time code drop. As such, we do not accept outside code contributions in the form of pull requests.AcknowledgementsWe thank David Luebke for helpful comments; Tero Kuosmanen and Sabu Nadarajan for their support with compute infrastructure; and Edgar Schönfeld for guidance on setting up unconditional BigGAN.