Package: configmypy
ConfigMyPyConfigure My Python: Easy configuration of Python projects.QuickstartJust clone the repository and install it, here in editable mode:gitclonehttps://github.com/JeanKossaifi/configmypycdconfigmypy
python-mpipinstall-e.Then, assuming you have a configuration fileconfig.yamlin your folder, get your configuration usingfromconfigmypyimportConfigPipeline,YamlConfig,ArgparseConfigpipe=ConfigPipeline([YamlConfig('./config.yaml'),ArgparseConfig()])config=pipe.read_conf()Then just read your parameters from config:arg = config.arg
arg2 = config['arg2']
...The script will read config.yaml, as well as optional command-line arguments, which will overwrite the config.RationaleHow many times did you start having a large Python project, let's say to run some Machine Learning models, where you wanted to change quickly the parameters.
Then you want to track them. In you write a small logger. Then you write a small yaml config file to more easily manage the quickly growing list of parameters.
Then you realize you sometimes need to change some of these parameters on the fly.
In comes argparse, and you manually override an increasingly long list of paramters with a code becoming a long list ofifargs.parameterisnotNone:config['parameter']=args.parameter...and there goes your sanity... Along with any hope of properly managing and tracking your experiments.To address this frustration, I wrote this minimal package that makes this easy.Reading from a.yamlconfig fileImagine you have aconfig.yamlfile that looks like this:default:opt:optimizer:'adam'lr:0.1data:dataset:'ns'batch_size:12test:opt:optimizer:'SGD'You can read that usingconfigmypy.YamlConfig:reader = YamlConfig('./config.yaml', config_name='default')config, _ = reader.read_conf()Bunch configurationsAll our configurations return a Bunch, not a dict. A Bunch is simply a dictionary that exposes its parameters as attributes so you can access them, equivalently, asconfig['param']orconfig.param.Configuration PipelineThe real power ofconfigmypycomes from the ConfigPipeline: you can have several steps called sequentially.
A typical usecase would be:Read a default (.yaml) configOptionally overwrite any of those parameters using argparse (command-line arguments)If the user specified another configuration, update the parameters read so far with that new configThis would look like this withconfigmypy:pipe=ConfigPipeline([YamlConfig('./config.yaml',config_name='default'),ArgparseConfig(config_file=None,config_name=None),YamlConfig()])config=pipe.read_conf()This will:first read thedefaultsection inconfig.yamlupdate any parameters with those passed by the user, including additional parametersconfig_fileandconfig_namewhich are automatically passed to the next stepIf the user specified aconfig_file, that will be read by the next step and used to update the configuration.You can check the configuration by callingpipe.log():>>>pipe.log()####################################    CONFIGURATION    ####################################Steps:------(1)YamlConfigwithconfig_file=./config.yaml,config_name=default(2)ArgparseConfigwithadditionalconfig={'config_file':None,'config_name':None}(3)YamlConfigwithconfig_file=config.yaml,config_name=test-------------------------------Configuration:--------------opt.optimizer=SGDopt.lr=0.1data.dataset=nsdata.batch_size=24###############################Questions or issuesThis is very much a project in development that I wrote for myself and decided to open-source so myself and others could easily reuse it for multiple projects, while knowing it is actually tested!If you have any questions or find any bugs, please open an issue, or better yet, a pull-request!