Package: pydpm
A python library focuses on constructing Deep Probabilistic Models (DPMs).
Our developed Pydpm not only provides efficient distribution sampling functions on GPU, but also has included the implementations of existing popular DPMs.Documentation|Paper [Arxiv]|Tutorials|Benchmarks|Examples|News:fire:A new version that does not depend on Pycuda has been released.:fire:An abundance of professional learning materials onDeep Generative Modelsfrom the Ermom's group at Stanford University. (CS236 - Fall 2021):fire:A tutorial of DPMs has been uploaded by Prof. Wilker Aziz (University of Amsterdam).InstallThe current version of PyDPM can be installed under either Windows or Linux system with PyPI.$ pip install pydpmFor Windows system, we recommed to install Visual Studio 2019 as the compiler equipped with CUDA 11.5 toolkit;
For Linux system, we recommed to install the latest version of CUDA toolkit.The enviroment for testing has been released for easily reproducing our results.$ conda env create -f enviroment.yamlOverviewThe overview of the framework of PyDPM library can be roughly split into four sectors, specifically Sampler, Model, Evaluation, and Example modules, which have been illustrated as follows:Sampler module includes both parts of the basic Distribution Sampler and the sophisticate Model Sampler, which can effectively accomplish the sampling
requirements of these DPMs constructed on either CPU or GPU;Model module contains a wide variety of classical and popular DPMs, which can be directly called as APIs in Python;Evaluation module provides a DataLoader sub-module to process data samples in various forms, such as images, text, graphs etc., and also a Metric sub-module to comprehensively evaluate these DPMs after training;Example module, for each DPM included in the Model module, we provides a corresponding code demo equipped with a detailed explanation in the official docs.The workflow of applying PyDPM for downstream tasks, which can be splited into four steps as follows:Device deployment of pyDPM can be choose as a platform with either CPU or GPU;Mechasnisms of model training or testing includes either or both of Gibbs sampling and back propagation,  implemented by pyDPM.sampler and pyTorch respecitveily;Model categories in pyDPM mainly include Bayesian Probabilistic Model, Deep-Learning Probabilistic Models, and Hybrid Probabilistic Models;Applications of DPMs has included Nature Language Processing (NLP), Graph Neural Network (GNN), and Recommendation System (RS) etc.Model ListThe Model module in pyDPM has included a wide variety of popular DPMs, which can be roughly split into several categories, including Bayesian Probabilistic Model, Deep-Learning Probabilistic Models, and Hybrid Probabilistic Models.Bayesian Probabilistic ModelsProbabilistic Model NameAbbreviationPaper LinkLatent Dirichlet AllocationLDABlei et al., 2003Poisson Factor AnalysisPFAZhou et al., 2012Poisson Gamma Belief NetworkPGBNZhou et al., 2015Convolutional Poisson Factor AnalysisCPFAWang et al., 2019Convolutional Poisson Gamma Belief NetworkCPGBNWang et al., 2019Factor AnalysisFAGaussian Mixed ModelGMMPoisson Gamma Dynamical SystemsPGDSZhou et al., 2016Deep Poisson Gamma Dynamical SystemsDPGDSGuo et al., 2018Dirichlet Belief NetworksDirBNZhao et al., 2018Deep Poisson Factor AnalysisDPFAGan et al., 2015Word Embeddings Deep Topic ModelWEDTMZhao et al., 2018Multimodal Poisson Gamma Belief NetworkMPGBNWang et al., 2018Graph Poisson Gamma Belief NetworkGPGBNWang et al., 2020Deep-Learning Probabilistic ModelsProbabilistic Model NameAbbreviationPaper LinkRestricted Boltzmann MachinesRBMHinton et al., 2010Variational AutoencoderVAEKingma et al., 2014Generative Adversarial NetworkGANGoodfellow et al., 2014Density estimation using Real NVPRealNVP (2d)Dinh et al., 2017Denoising Diffusion Probabilistic ModelsDDPMHo et al., 2020Density estimation using Real NVPRealNVP (image)Dinh et al., 2018Conditional Variational AutoencoderCVAESohn et al., 2015Deep Convolutional Generative Adversarial NetworksDCGANRadford et al., 2016Wasserstein Generative Adversarial NetworksWGANArjovsky et al., 2017Information Maximizing Generative Adversarial NetsInfoGANXi Chen et al., 2016Hybrid Probabilistic ModelsProbabilistic Model NameAbbreviationPaper LinkWeibull Hybrid Autoencoding InferenceWHAIZhang et al., 2018Weibull Graph Attention AutoencoderWGAAEWang et al., 2020Recurrent Gamma Belief NetworkrGBNGuo et al., 2020Multimodal Weibull Variational AutoencoderMWVAEWang et al., 2020Sawtooth Embedding Topic ModelSawETMDuan et al., 2021TopicNetTopicNetDuan et al., 2021Deep Coupling Embedding Topic Modeldc-ETMLi et al., 2022Topic Taxonomy Mining with Hyperbolic EmbeddingHyperMinerXu et al., 2022Knowledge Graph Embedding Topic ModelKG-ETMWang et al., 2022Variational Edge Parition ModelVEPMHe et al., 2022Generative Text Convolutional Neural NetworkGTCNNWang et al., 2022Deep Proabilistic Models planned to be built:fire:Welcome to introduce classical or novel Deep Proabilistic Models for us.Probabilistic Model NameAbbreviationPaper LinkNouveau Variational AutoencoderNVAEVahdat et al., 2020flow-based Variational Autoencoderf-VAESu et al., 2018Score-Based Generative ModelsSGMBortoli et al., 2022Poisson Flow Generative ModelsPFGMXu et al., 2022Stable DiffusionLDMRombach et al., 2022Denoising Diffusion Implicit ModelsDDIMSong et al., 2022Vector Quantized DiffusionVQ-DiffusionTang et al., 2023Vector Quantized Variational AutoencoderVQ-VAEAaron van den Oord et al., 2017Conditional Generative Adversarial NetscGANMirza et al., 2014Information Maximizing Variational AutoencodersInfoVAEzhao et al.,2017Generative FlowGlowKingama et al., 2018Structured Denoising Diffusion Models in Discrete State-SpacesDP3MAustin et al., 2021UsageExample: a few code lines to quickly construct and evaluate a 3-layer Bayesian model namedPGBNon GPU.frompydpm.modelimportPGBNfrompydpm.metricimportACC# create the model and deploy it on gpu or cpumodel=PGBN([128,64,32],device='gpu')model.initial(train_data)train_local_params=model.train(train_data,iter_all=100)train_local_params=model.test(train_data,iter_all=100)test_local_params=model.test(test_data,iter_all=100)# evaluate the model with classification accuracy# the demo accuracy can achieve 0.8549results=ACC(train_local_params.Theta[0],test_local_params.Theta[0],train_label,test_label,'SVM')# save the model after trainingmodel.save()Example: a few code lines to quickly deploy distribution sampler of Pydpm on GPU.frompydpm.samplerimportBasic_Samplersampler=Basic_Sampler('gpu')a=sampler.gamma(np.ones(100)*5,1,times=10)b=sampler.gamma(np.ones([100,100])*5,1,times=10)CompareCompare the distribution sampling efficiency of PyDPM with numpy:Compare the distribution sampling efficiency of PyDPM with tensorflow and torch:Compare the distribution sampling efficiency of PyDPM with CuPy and PyCUDA(used by pydpm v1.0):ContactLicense: Apache License Version 2.0Contact:  Chaojie Wangxd_silly@163.com, Wei Zhao13279389260@163.com, Xinyang Liulxy771258012@163.com, Bufeng Ge20009100138@stu.xidian.edu.cn, Jiawen Wuwjw19960807@163.comCopyright (c), 2020, Chaojie Wang, Wei Zhao, Xinyang Liu, Jiawen Wu, Jie Ren, Yewen Li, Wenchao Chen, Hao Zhang, Bo Chen and Mingyuan Zhou