Package: defaults
Extensible and Fault-Tolerant Hyperparameter ManagementHParams is a thoughtful approach to configuration management for machine learning projects. It
enables you to externalize your hyperparameters into a configuration file. In doing so, you can
reproduce experiments, iterate quickly, and reduce errors.Features:Approachable and easy-to-use APIBattle-tested over three yearsFast with little to no runtime overhead (< 3e-05 seconds) per configured functionRobust to most use cases with 100% test coverage and 75 testsLightweight with only one dependencyLogo byChloe Yeo, Corporate Sponsorship byWellSaid LabsInstallationMake sure you have Python 3. You can then installhparamsusingpip:pipinstallhparamsInstall the latest code via:pipinstallgit+https://github.com/PetrochukM/HParams.gitOops 🐛With HParams, you will avoid common but needless hyperparameter mistakes. It will throw a warning
or error if:A hyperparameter is overwritten.A hyperparameter is declared but not set.A hyperparameter is set but not declared.A hyperparameter type is incorrect.Finally, HParams is built with developer experience in mind. HParams includes 13 errors and 6 warnings
to help catch and resolve issues quickly.ExamplesAdd HParams to your project by following one of these common use cases:Configure Training 🤗Configure your training run, like so:# main.pyfromhparamsimportconfigurable,add_config,HParams,HParamfromtypingimportUnion@configurabledeftrain(batch_size:int=HParam()):passclassModel():@configurabledef__init__(self,hidden_size=HParam(int),dropout=HParam(float)):passadd_config({'main':{'train':HParams(batch_size=32),'Model.__init__':HParams(hidden_size=1024,dropout=0.25),}})HParams supports optional configuration typechecking to help you find bugs! 🐛Set DefaultsConfigure PyTorch and Tensorflow defaults to match via:fromtorch.nnimportBatchNorm1dfromhparamsimportconfigurable,add_config,HParams# NOTE: `momentum=0.01` to match Tensorflow defaultsBatchNorm1d.__init__=configurable(BatchNorm1d.__init__)add_config({'torch.nn.BatchNorm1d.__init__':HParams(momentum=0.01)})Configure your random seed globally, like so:# config.pyimportrandomfromhparamsimportconfigurable,add_config,HParamsrandom.seed=configurable(random.seed)add_config({'random.seed':HParams(a=123)})# main.pyimportconfigimportrandomrandom.seed()CLIExperiment with hyperparameters through your command line, for example:foo@bar:~$file.py--torch.optim.adam.Adam.__init__'HParams(lr=0.1,betas=(0.999,0.99))'importsysfromtorch.optimimportAdamfromhparamsimportconfigurable,add_config,parse_hparam_argsAdam.__init__=configurable(Adam.__init__)parsed=parse_hparam_args(sys.argv[1:])# Parse command line argumentsadd_config(parsed)Hyperparameter optimizationHyperparameter optimization is easy to-do, check this out:importitertoolsfromtorch.optimimportAdamfromhparamsimportconfigurable,add_config,HParamsAdam.__init__=configurable(Adam.__init__)deftrain():# Train the model and return the loss.passforbetasinitertools.product([0.999,0.99,0.9],[0.999,0.99,0.9]):add_config({Adam.__init__:HParams(betas=betas)})# Grid search over the `betas`train()Track HyperparametersEasily track your hyperparameters using tools likeComet.fromcomet_mlimportExperimentfromhparamsimportget_configexperiment=Experiment()experiment.log_parameters(get_config())Multiprocessing: Partial SupportExport a Pythonfunctools.partialto use in another process, like so:fromhparamsimportconfigurable,HParam@configurabledeffunc(hparam=HParam()):passpartial=func.get_configured_partial()With this approach, you don't have to transfer the global state to the new process. To transfer the
global state, you'll want to useget_configandadd_config.Docs 📖The complete documentation for HParams is availablehere.Learn more about related projects to HParamshere.ContributingWe've released HParams because a lack of hyperparameter management solutions. We hope that
other people can benefit from the project. We are thankful for any contributions from the
community.Contributing GuideRead ourcontributing guideto
learn about our development process, how to propose bugfixes and improvements, and how to build and
test your changes to HParams.AuthorsMichael Petrochuk— DeveloperChloe Yeo— Logo DesignCitingIf you find HParams useful for an academic publication, then please use the following BibTeX to
cite it:@misc{hparams,
author ={Petrochuk, Michael},
title ={HParams: Hyperparameter management solution},
year ={2019},
publisher ={GitHub},
journal ={GitHub repository},
howpublished ={\url{https://github.com/PetrochukM/HParams}},}