Package: ex2mcmc
Ex2MCMC: Local-Global MCMC kernels: the bost of both worlds (NeurIPS 2022)[Paper][ArXiv]Authors: Sergey Samsonov, Evgeny Lagutin, Marylou Gabrié, Alain Durmus, Alexey Naumov, Eric Moulines.Abstract:In the present paper we study an Explore-Exploit Markov chain Monte Carlo strategy (Ex2MCMC) that combines local and global samplers showing that it enjoys the advantages of both approaches. We prove V-uniform geometric ergodicity of Ex2MCMC without requiring a uniform adaptation of the global sampler to the target distribution. We also compute explicit bounds on the mixing rate of the Explore-Exploit strategy under realistic conditions. Moreover, we also analyze an adaptive version of the strategy (FlEx2MCMC) where a normalizing flow is trained while sampling to serve as a proposal for global moves. We illustrate the efficiency of Ex2MCMC and its adaptive version on classical sampling benchmarks as well as in sampling high-dimensional distributions defined by Generative Adversarial Networks seen as Energy Based Models.Ex2MCMC: Local-Global MCMC kernels: the bost of both worlds (NeurIPS 2022) [Paper]Single chain mixingSampling from GAN as Energy-Based Models with MCMCAlgorithmsInstallationDevelopement installationDownload checkpoints and statsUsageDemonstration on SNGANExperiments with synthetic distributions:Experiments with GANs on MNIST datasetExperiments with GANs on CIFAR10 datasetSampling and FID computationResultsFID and Inception Score (CIFAR10)Sampling trajectories (CIFAR10)Energy landscape approximation (MNIST)CitationSingle chain mixingSampling from GAN as Energy-Based Models with MCMCAlgorithmsEx2MCMC:FlEx2MCMC:InstallationCreate environment:condacreate-nex2mcmcpython=3.8
condaactivateex2mcmcpipinstallex2mcmc
pipinstallgit+https://github.com/kwotsin/mimicry.gitDevelopement installationInstall poetry (if absent):curl-sSLhttps://install.python-poetry.org|python3-
poetryconfigvirtualenvs.createfalseInstall the project:poetryinstall--withdev
poetryaddgit+https://github.com/kwotsin/mimicry.git@a7fda06c4aff1e6af8dc4c4a35ed6636e434c766Download checkpoints and statsCIFAR10 checkpoints:GANStepsPath, GPath, DDCGAN NS100knetG_100000_steps.pthnetD_100000_steps.pthSNGAN, Hinge100knetG.pthnetD.pthMNIST checkpoints:GANPathVanillavanilla_gan.pthWGAN CPwgan.pthYou also can run script to download checkpoints:chmod+xget_ckpts.sh
./get_ckpts.shDownload statistics for FID cimputation for CIFAR10 dataset:mkdir-pstats&gdown1jjgB_iuvmoVAXPRvVTI_hBfuIz7mQgOg-Ostats/fid_stats_cifar10.npzUsageDemonstration on SNGANTry with colab:Experiments with synthetic distributions:FlEx2MCMC  vs     NUTS:ExperimentPathColabToyish Gaussianexperiments/exp_synthetic/toyish_gaussian.ipynbTBDGaussian mixtureexperiments/exp_synthetic/gaussian_mixture.ipynbFlEx for Mixture of Gaussiansexperiments/exp_synthetic/flex_mog.ipynbFlEx for banana-shaped distributionexperiments/exp_synthetic/flex_banana.ipynbFlEx for Neal's funnel distributionexperiments/exp_synthetic/flex_funnel.ipynbTo reproduce the experimets on banana-shaped and funnel distributions:pythonexperiments/exp_synthetic/banana_funnel_metrics.py--distribution{banana,funnel}--devicecuda:0Experiments with GANs on MNIST datasetexperiments/exp_mnist/JSGAN_samples.ipynbexperiments/exp_mnist/WGAN_samples.ipynbExperiments with GANs on CIFAR10 datasetexperiments/exp_cifar10_demo/DCGAN_samples.ipynbexperiments/exp_cifar10_demo/SNGAN_samples.ipynbSampling and FID computationpythonexperiments/exp_cifar10_fid/run.pyconfigs/mcmc_configs/{ula,mala,isir,ex2mcmc,flex2mcmc}.ymlconfigs/mmc_dcgan.ymlTo run a full experiment:chmod+xexperiments/exp_cifar10_fid/run.sh&./experiments/exp_cifar10_fid/run.shResultsFID and Inception Score (CIFAR10)GANMCMCStepsInception ScoreFIDDCGANnone06.328.4DCGANi-SIR1k6.9622.7DCGANMALA1k6.9523.4DCGANEx2MCMC (our)1k7.5619.0DCGANFlEx2MCMC (our)1k7.9219.2DCGANFlEx2MCMC (our)1807.6217.1Sampling trajectories (CIFAR10)Generation trajectories for DCGAN.ULA:MALA:i-SIR:Ex2MCMC:FlEx2MCMC:Energy landscape approximation (MNIST)Projection of GAN samples onto the energy landsape when trained on MNIST dataset:Citation@article{samsonov2022local,title={Local-Global MCMC kernels: the best of both worlds},author={Samsonov, Sergey and Lagutin, Evgeny and Gabri{\'e}, Marylou and Durmus, Alain and Naumov, Alexey and Moulines, Eric},journal={Advances in Neural Information Processing Systems},volume={35},pages={5178--5193},year={2022}}