Package: tinyda
tinyDAMultilevel Delayed Acceptance MCMC sampler with finite-length subchain sampling and adaptive error modelling. This is intended as a simple, lightweight implementation, with minimal dependencies, i.e. nothing beyond the SciPy stack and ArviZ. It is fully imperative and easy to use!For instructions, have a look at thedocumentation, theexamplesor theusage section below.InstallationtinyDA can be installed from PyPI:pip install tinydaDependenciesNumPySciPyArviZtqdmRay(multiprocessing, optional)FeaturesSamplersMetropolis-HastingsDelayed Acceptance (Christen & Fox, 2005)Multilevel Delayed Acceptance (Lykkegaard et al. 2022)ProposalsRandom Walk Metropolis Hastings (RWMH) - Metropolis et al. (1953), Hastings (1970)preconditioned Crank-Nicolson (pCN) - Cotter et al. (2013)Adaptive Metropolis (AM) - Haario et al. (2001)Operator-weighted pCN - Law (2014)Metropolis Adjusted Langevin Algorithm (MALA) - Roberts & Tweedie (1996)DREAM(Z) - Vrugt (2016)Multiple-Try Metropolis (MTM) - Liu et al. (2000)Adaptive Error ModelsState independent - Cui et al. (2018)State dependent - Cui et al. (2018)DiagnosticsConvert a tinyDA chain to an ArviZ InferenceData object for near-unlimited diagnostics!UsageDocumentation is available atRead the Docs. A few illustrative examples are available as Jupyter Notebooks in the root directory. Below is a short summary of the core features.DistributionsThe prior and likelihood can be defined using standardscipy.statsclasses:importtinyDAastdafromscipy.statsimportmultivariate_normal# set the prior mean and covariance.mean_prior=np.zeros(n_dim)cov_prior=np.eye(n_dim)# set the covariance of the likelihood.cov_likelihood=sigma**2*np.eye(data.shape[0])# initialise the prior distribution and likelihood.my_prior=multivariate_normal(mean_prior,cov_prior)my_loglike=tda.GaussianLogLike(data,cov_likelihood)If using a Gaussian likelihood, we recommend using thetinyDAimplementation, since it is unnormalised and plays along well withtda.AdaptiveLogLikeused for the Adaptive Error Model. Home-brew distributions can easily be defined, and must have a.rvs()method for drawing random samples and alogpdf(x)method for computing the log-likelihood, as per theSciPyimplementation.tinyDA.PosteriorThe heart of the TinyDA sampler is thetinyDA.Posterior, which is responsible for:Calling the model with some parameters (a proposal) and collecting the model output.Evaluating the prior density of the parameters, and the likelihood of the data, given the parameters.Constructingtda.Linkinstances that hold information for each sample.ThetinyDA.Posteriortakes as input the prior, the likelihood, and a forward model. Therefore, a forward model must be defined. This model can be either a functionmodel_output = my_function(parameters)or a class instance with a.__call__(self, parameters)method. The function or__call__method must return either just the model output or a tuple of(model_output, qoi). In this example, we define a class that performs simple linear regression on whatever inputsxwe have.classMyLinearModel:def__init__(self,x):self.x=xdef__call__(self,parameters):# the model output is a simple linear regressionmodel_output=parameters[0]+parameters[1]*self.x# no quantity of interest beyond the parameters.qoi=None# return both.returnmodel_output,qoimy_model=MyLinearModel(x)my_posterior=tda.Posterior(my_prior,my_loglike,my_model)ProposalsA proposal is simply initialised with its parameters:# set the covariance of the proposal distribution.am_cov=np.eye(n_dim)# set the number of iterations before starting adaptation.am_t0=1000# set some adaptive metropolis tuning parameters.am_sd=1am_epsilon=1e-6# initialise the proposal.my_proposal=tda.AdaptiveMetropolis(C0=am_cov,t0=am_t0,sd=am_sd,epsilon=am_epsilon)SamplingAfter defining a proposal, a coarse posteriormy_posterior_coarse, and a fine posteriormy_posterior_fine, the Delayed Acceptance sampler can be run usingtinyDA.sample():my_chains=tda.sample([my_posterior_coarse,my_posterior_fine],my_proposal,iterations=12000,n_chains=2,subsampling_rate=10)If using a hirarchy with more than two models, a Multilevel Delayed Acceptance sampler can be run by supplying a list of posteriors in ascending order and a correponsing list of subsampling rates:my_chains=tda.sample([my_posterior_level0,my_posterior_level1,my_posterior_level2,my_posterior_level3],my_proposal,iterations=12000,n_chains=2,subsampling_rate=[10,5,5])PostprocessingThe entire sampling history is now stored inmy_chainsin the form of a dictionary with tinyDA.Link instances. You can convert the output oftinyDA.sample()to an ArviZ InferenceData object withidata=tda.to_inference_data(my_chains,burnin=2000)If you want to have a look at the coarse samples, you can pass an additional argument:idata=tda.to_inference_data(my_chains,level='coarse',burnin=20000)Theidataobject can then be used with the ArviZ diagnostics suite to e.g. get MCMC statistics, plot the traces and so on.ContributingIf you feel that tinyDA is missing some features, or that something could be improved, please do not hesitate to create a fork and submit a PR! If you want to help improve the package, please have a look at theissuesand consider if something seems doable to you.If you would like to contribute, please consider the following:It's called tinyDA because it's small. The list of dependencies should be keptshort. Great things can be achieved using NumPy!tinyDA has loads of nice features, but it's somewhat lacking in terms of CI. Any kind of CI, tests and improvements to the software infrastructure would be greatly appreciated!The development of tinyDA is sponsored bydigiLab.TODOParallel multi-chain samplingMore user-friendly diagnosticsMultilevel Delayed AcceptanceMALA proposalTestsVariance ReductionWrapper for framework-agnostic adaptive coarse modelEmbedded spaces for hierachical models