Package: bayestorch
BayesTorchWelcome tobayestorch, a lightweight Bayesian deep learning library for fast prototyping based onPyTorch. It provides the basic building blocks for the following
Bayesian inference algorithms:Bayes by Backprop (BBB)Markov chain Monte Carlo (MCMC)Stein variational gradient descent (SVGD)💡 Key featuresLow-code definition of Bayesian (or partially Bayesian) modelsSupport for custom neural network layersSupport for custom prior/posterior distributionsSupport for layer/parameter-wise prior/posterior distributionsSupport for composite prior/posterior distributionsHighly modular object-oriented designUser-friendly and easily extensible APIsDetailed API documentation🛠️️ InstallationUsing PipFirst of all, installPython 3.6 or later. Open a terminal and run:pip install bayestorchFrom sourceFirst of all, installPython 3.6 or later.
Clone or download and extract the repository, navigate to<path-to-repository>, open a
terminal and run:pip install -e .▶️ QuickstartHere are a few code snippets showcasing some key features of the library.
For complete training loops, please refer toexamples/mnistandexamples/regression.Bayesian model trainable via Bayes by Backpropfromtorch.nnimportLinearfrombayestorch.distributionsimport(get_mixture_log_scale_normal,get_softplus_inv_scale_normal,)frombayestorch.nnimportVariationalPosteriorModule# Define modelmodel=Linear(5,1)# Define log scale normal mixture prior over the model parametersprior_builder,prior_kwargs=get_mixture_log_scale_normal(model.parameters(),weights=[0.75,0.25],locs=(0.0,0.0),log_scales=(-1.0,-6.0))# Define inverse softplus scale normal posterior over the model parametersposterior_builder,posterior_kwargs=get_softplus_inv_scale_normal(model.parameters(),loc=0.0,softplus_inv_scale=-7.0,requires_grad=True,)# Define Bayesian model trainable via Bayes by Backpropmodel=VariationalPosteriorModule(model,prior_builder,prior_kwargs,posterior_builder,posterior_kwargs)Partially Bayesian model trainable via Bayes by Backpropfromtorch.nnimportLinearfrombayestorch.distributionsimport(get_mixture_log_scale_normal,get_softplus_inv_scale_normal,)frombayestorch.nnimportVariationalPosteriorModule# Define modelmodel=Linear(5,1)# Define log scale normal mixture prior over `model.weight`prior_builder,prior_kwargs=get_mixture_log_scale_normal([model.weight],weights=[0.75,0.25],locs=(0.0,0.0),log_scales=(-1.0,-6.0))# Define inverse softplus scale normal posterior over `model.weight`posterior_builder,posterior_kwargs=get_softplus_inv_scale_normal([model.weight],loc=0.0,softplus_inv_scale=-7.0,requires_grad=True,)# Define partially Bayesian model trainable via Bayes by Backpropmodel=VariationalPosteriorModule(model,prior_builder,prior_kwargs,posterior_builder,posterior_kwargs,[model.weight],)Composite priorfromtorch.distributionsimportIndependentfromtorch.nnimportLinearfrombayestorch.distributionsimport(CatDistribution,get_laplace,get_normal,get_softplus_inv_scale_normal,)frombayestorch.nnimportVariationalPosteriorModule# Define modelmodel=Linear(5,1)# Define normal prior over `model.weight`weight_prior_builder,weight_prior_kwargs=get_normal([model.weight],loc=0.0,scale=1.0,prefix="weight_",)# Define Laplace prior over `model.bias`bias_prior_builder,bias_prior_kwargs=get_laplace([model.bias],loc=0.0,scale=1.0,prefix="bias_",)# Define composite prior over the model parametersprior_builder=(lambda**kwargs:CatDistribution([Independent(weight_prior_builder(**kwargs),1),Independent(bias_prior_builder(**kwargs),1),]))prior_kwargs={**weight_prior_kwargs,**bias_prior_kwargs}# Define inverse softplus scale normal posterior over the model parametersposterior_builder,posterior_kwargs=get_softplus_inv_scale_normal(model.parameters(),loc=0.0,softplus_inv_scale=-7.0,requires_grad=True,)# Define Bayesian model trainable via Bayes by Backpropmodel=VariationalPosteriorModule(model,prior_builder,prior_kwargs,posterior_builder,posterior_kwargs,)📧 Contactluca.dellalib@gmail.com