Package: millipede-regression
millipede: A library for Bayesian variable selection..    ..
                           millipede      )  (
                      _ _ _ _ _ _ _ _ _ _(.--.)
                     {_{_{_{_{_{_{_{_{_{_( '_')
                     /\/\/\/\/\/\/\/\/\/\ `---millipede is aPyTorch-based library for Bayesian variable selection in generalized
linear models that can be run on both CPU and GPU and that
can handle datasets with numbers of data points and covariates in the tens of thousands or more.What is Bayesian variable selection?Bayesian variable selection is a model-based approach for identifying parsimonious explanations of observed data.
In the context of generalized linear models withPcovariates{X_1, ..., X_P}and responsesY,
Bayesian variable selection can be used to identifysparsesubsets of covariates (i.e. far fewer thanP)
that are sufficient for explaining the observed responses in terms of a linear function of the covariates.In more detail, Bayesian variable selection is formulated as a model selection problem in which we consider
the space of2^Pmodels in which some covariates are included and the rest are excluded.
For example, for continuous-valued responses one particular model might take the formY = beta_3 X_3 + beta_9 X_9with (non-zero) coefficientsbeta_3andbeta_9.
A priori we assume that models with fewer included covariates are more likely than those with more included covariates.
The set of parsimonious models best supported by the data then emerges from the posterior distribution over the space of models.What's especially appealing about Bayesian variable selection is that it provides an interpretable score
called the PIP (posterior inclusion probability) for each covariateX_p.
The PIP is a true probability and so it satisfies0 <= PIP <= 1by definition.
Covariates with large PIPs are good candidates for being explanatory of the responseY.Being able to compute PIPs is particularly useful for high-dimensional datasets with largeP.
For example, we might want to select a small number of covariates to include in a predictive model (i.e. feature selection).
Alternatively, in settings where it is implausible to subject allPcovariates to
some expensive downstream analysis (e.g. a laboratory experiment),
Bayesian variable selection can be used to select a small number of covariates for further analysis.Requirementsmillipede requires Python 3.8 or later and the following Python packages:PyTorch,pandas, andpolyagamma.Note that if you wish to run millipede on a GPU you need to install PyTorch with CUDA support.
In particular if you run the following command from your terminal it should report True:python -c 'import torch; print(torch.cuda.is_available())'Installation instructionsInstall directly from GitHub:pip install git+https://github.com/BasisResearch/millipede.gitInstall from source:git clone git@github.com:BasisResearch/millipede.git
cd millipede
pip install .Basic usageUsing millipede is easy:# import millipedefrommillipedeimportNormalLikelihoodVariableSelector# create a VariableSelector object appropriate to your datatypeselector=NormalLikelihoodVariableSelector(dataframe,# pass in the data'Response',# indicate the column of responsesS=1,# specify the expected number of covariates to include a priori)# run the MCMC algorithm to compute posterior inclusion probabilities# and other posterior quantities of interestselector.run(T=1000,T_burnin=500)# inspect the resultsprint(selector.summary)See the Jupyter notebooks in thenotebooksdirectory for detailed example usage.Supported data typesThe covariatesXare essentially arbitrary and can be continuous-valued, binary-valued, a mixture of the two, etc.
Currently the responseYcan be any of the following:Response typeSelector classcontinuous-valuedNormalLikelihoodVariableSelectorbinary-valuedBernoulliLikelihoodVariableSelectorbounded countsBinomialLikelihoodVariableSelectorunbounded countsNegativeBinomialLikelihoodVariableSelectorScalabilityRoughly speaking, the cost of the MCMC algorithms implemented in millipede is proportional
toN x P, whereNis the total number of data points andPis the total number of covariates.
For anapproximateguide to hardware requirements please consult the following table:RegimeExpectationsN x P < 10^7Use a CPU10^7 < N x P < 10^8Use a GPU10^8 < N x P < 10^10Use a GPU with the subset_size argument10^10 < N x PYou may be out of luckDocumentationRead the docshere.FAQHow many MCMC iterations do I need for good results?It's hard to say. Generally speaking, difficult regimes with highly-correlated covariates or a large number of
covariates are expected to require more iterations. Similarly, datasets with count-based responses are expected to require
more iterations than those with continuous-valued responses (because the underlying inference problem is more difficult).
The best way to determine if you need more MCMC iterations is to run millipede twice with different random number seeds.
If the results for both runs are not similar, you probably want to increase the number of iterations.
As a general rule of thumb, it's probably good to aim for at least10^4-10^5samples if doing so is feasible.
Also, you probably want at least 1000 burn-in iterations.Contact informationMartin Jankowiak:martin@basis.aiReferencesJankowiak, M., 2022.Bayesian Variable Selection in a Million Dimensions. arXiv preprint arXiv:2208.01180.Zanella, G. and Roberts, G., 2019.Scalable importance tempering and Bayesian variable selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81(3), pp.489-517.Citing millipedeIf you use millipede please consider citing:@article{jankowiak2022bayesian,
      title={Bayesian Variable Selection in a Million Dimensions},
      author={Martin Jankowiak},
      journal={arXiv preprint arXiv:{2208.01180},
      year={2022},
      eprint={2208.01180},
      archivePrefix={arXiv},
      primaryClass={stat.ME}
}