Package: botorch
BoTorch is a library for Bayesian Optimization built on PyTorch.BoTorch is currently in beta and under active development!Why BoTorch ?BoTorchProvides a modular and easily extensible interface for composing Bayesian
optimization primitives, including probabilistic models, acquisition functions,
and optimizers.Harnesses the power of PyTorch, including auto-differentiation, native support
for highly parallelized modern hardware (e.g. GPUs) using device-agnostic code,
and a dynamic computation graph.Supports Monte Carlo-based acquisition functions via thereparameterization trick, which makes it
straightforward to implement new ideas without having to impose restrictive
assumptions about the underlying model.Enables seamless integration with deep and/or convolutional architectures in PyTorch.Has first-class support for state-of-the art probabilistic models inGPyTorch, including support for multi-task Gaussian
Processes (GPs) deep kernel learning, deep GPs, and approximate inference.Target AudienceThe primary audience for hands-on use of BoTorch are researchers and
sophisticated practitioners in Bayesian Optimization and AI.
We recommend using BoTorch as a low-level API for implementing new algorithms
forAx. Ax has been designed to be an easy-to-use platform
for end-users, which at the same time is flexible enough for Bayesian
Optimization researchers to plug into for handling of feature transformations,
(meta-)data management, storage, etc.
We recommend that end-users who are not actively doing research on Bayesian
Optimization simply use Ax.InstallationInstallation RequirementsPython >= 3.9PyTorch >= 1.13.1gpytorch == 1.11linear_operator == 0.5.1pyro-ppl >= 1.8.4scipymultiple-dispatchPrerequisite only for MacOS users with Intel processors:Before installing BoTorch, we recommend first manually installing PyTorch, a required dependency of
BoTorch. Installing it according to thePyTorch installation instructionsensures that it is properly linked against MKL, a library that optimizes mathematical computation for Intel processors.
This will result in up to an order-of-magnitude speed-up for Bayesian optimization, as at the moment,
installing PyTorch from pip does not link against MKL.The PyTorch installation instructions currently recommend:InstallAnaconda. Note that there are different installers for Intel and M1 Macs.Install PyTorch following thePyTorch installation instructions.
Currently, this suggests runningconda install pytorch torchvision -c pytorch.If you want to customize your installation, please follow thePyTorch installation instructionsto build from source.Option 1: Installing the latest releaseThe latest release of BoTorch is easily installed either viaAnaconda(recommended) or pip.To install BoTorch from Anaconda, runcondainstallbotorch-cpytorch-cgpytorch-cconda-forgeThe above command installs BoTorch and any needed dependencies.-c pytorch -c gpytorch -c conda-forgemeans that the most preferred source to install from is the PyTorch channel, the next most preferred is the GPyTorch channel,
and the least preferred is conda-forge.Alternatively, to install withpip, dopipinstallbotorchNote: Make sure thepipbeing used is actually the one from the newly created Conda environment. If you're using a Unix-based OS, you can usewhich pipto check.Option 2: Installing from latest main branchIf you would like to try our bleeding edge features (and don't mind potentially
running into the occasional bug here or there), you can install the latest
development version directly from GitHub. If you want to also install the
currentgpytorchandlinear_operatordevelopment versions, you will need
to ensure that theALLOW_LATEST_GPYTORCH_LINOPenvironment variable is set:pipinstall--upgradegit+https://github.com/cornellius-gp/linear_operator.git
pipinstall--upgradegit+https://github.com/cornellius-gp/gpytorch.gitexportALLOW_LATEST_GPYTORCH_LINOP=truepipinstall--upgradegit+https://github.com/pytorch/botorch.gitOption 3: Editable/dev installIf you want tocontributeto BoTorch, you will want to install editably so that you can change files and have the
changes reflected in your local install.If you want to install the currentgpytorchandlinear_operatordevelopment versions, as in Option 2, do that
before proceeding.Option 3a: Bare-bones editable installgitclonehttps://github.com/pytorch/botorch.gitcdbotorch
pipinstall-e.Option 3b: Editable install with development and tutorials dependenciesgitclonehttps://github.com/pytorch/botorch.gitcdbotorchexportALLOW_BOTORCH_LATEST=truepipinstall-e".[dev, tutorials]"dev: Specifies tools necessary for development
(testing, linting, docs building; seeContributingbelow).tutorials: Also installs all packages necessary for running the tutorial notebooks.You can also install either the dev or tutorials dependencies without installing both, e.g. by changing the last command topip install -e ".[dev]".Getting StartedHere's a quick run down of the main components of a Bayesian optimization loop.
For more details see ourDocumentationand theTutorials.Fit a Gaussian Process model to dataimporttorchfrombotorch.modelsimportSingleTaskGPfrombotorch.fitimportfit_gpytorch_mllfromgpytorch.mllsimportExactMarginalLogLikelihood# Double precision is highly recommended for GPs.# See https://github.com/pytorch/botorch/discussions/1444train_X=torch.rand(10,2,dtype=torch.double)Y=1-(train_X-0.5).norm(dim=-1,keepdim=True)# explicit output dimensionY+=0.1*torch.rand_like(Y)train_Y=(Y-Y.mean())/Y.std()gp=SingleTaskGP(train_X,train_Y)mll=ExactMarginalLogLikelihood(gp.likelihood,gp)fit_gpytorch_mll(mll)Construct an acquisition functionfrombotorch.acquisitionimportUpperConfidenceBoundUCB=UpperConfidenceBound(gp,beta=0.1)Optimize the acquisition functionfrombotorch.optimimportoptimize_acqfbounds=torch.stack([torch.zeros(2),torch.ones(2)])candidate,acq_value=optimize_acqf(UCB,bounds=bounds,q=1,num_restarts=5,raw_samples=20,)Citing BoTorchIf you use BoTorch, please cite the following paper:M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Advances in Neural Information Processing Systems 33, 2020.@inproceedings{balandat2020botorch,
  title={{BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization}},
  author={Balandat, Maximilian and Karrer, Brian and Jiang, Daniel R. and Daulton, Samuel and Letham, Benjamin and Wilson, Andrew Gordon and Bakshy, Eytan},
  booktitle = {Advances in Neural Information Processing Systems 33},
  year={2020},
  url = {http://arxiv.org/abs/1910.06403}
}Seeherefor an incomplete selection of peer-reviewed papers that build off of BoTorch.ContributingSee theCONTRIBUTINGfile for how to help out.LicenseBoTorch is MIT licensed, as found in theLICENSEfile.