Package: effortless-config
Effortless ConfigGlobally scoped configuration with argparse integration.Installationpip install effortless-configRationaleWhen building machine learning models, I often find myself with a file namedconfig.pythat has a bunch of global variables that I reference throughout the codebase.As the work progresses, I end up withgroupsof specific configuration settings that correspond to specific experiments.I want to be able to select configuration group on the command line when I start an experiment.And I want to be able to override certain settings within that experiment, from the command line.Basic usageFirst, define a class that extendseffortless_config.Configand specify your default configuration values inside it:fromeffortless_configimportConfigclassconfig(Config):# lowercase "c" optionalSOME_INTEGER_SETTING=10FLOAT_SETTING=0.5A_BOOLEAN=FalseMY_STRING_SETTING='foo'Then you can refer to these settings elsewhere in your code base:defmain():print(f'SOME_INTEGER_SETTING is{config.SOME_INTEGER_SETTING}')print(f'FLOAT_SETTING is{config.FLOAT_SETTING}')print(f'A_BOOLEAN is{config.A_BOOLEAN}')print(f'MY_STRING_SETTING is{config.MY_STRING_SETTING}')The functionconfig.parse_args()creates anargparseparser from your configuration. If we put the two previous code snippets in a file calledbasic_example.pyand finish it off withif__name__=='__main__':config.parse_args()main()...we can see the available settings with-h/--help:$pythonbasic_example.py--helpusage: basic_example.py [-h] [--some-integer-setting SOME_INTEGER_SETTING][--float-setting FLOAT_SETTING] [--a-boolean {true,false}][--my-string-setting MY_STRING_SETTING]optional arguments:-h, --help            show this help message and exit--some-integer-setting SOME_INTEGER_SETTING--float-setting FLOAT_SETTING--a-boolean {true,false}--my-string-setting MY_STRING_SETTINGWe can then override configuration settings from the command line:$pythonbasic_example.pySOME_INTEGER_SETTING is 10FLOAT_SETTING is 0.5A_BOOLEAN is FalseMY_STRING_SETTING is foo$pythonbasic_example.py--some-integer-setting1000SOME_INTEGER_SETTING is 1000FLOAT_SETTING is 0.5A_BOOLEAN is FalseMY_STRING_SETTING is fooList and dict type settings can be overridden by passing json strings from the command line, e.g.$pythonlists_and_dicts.py--my-list='[1, 2, 3]'--my-dict='{"foo": 10, "bar": 20}'UsinggroupsConfigurationgroupsare families of settings that you want to tie together. For example, you might have some machine learning problem for which you have a large model and a small model, and you want to easily switch between the two models without having to specify all parameters on the command line every time.To specify groups, use thesettings(...)function:fromeffortless_configimportConfig,settingclassconfig(Config):groups=['large','small']NUM_LAYERS=setting(default=5,large=10,small=3)NUM_UNITS=setting(default=128,large=512,small=32)USE_SKIP_CONNECTIONS=setting(default=True,small=False)LEARNING_RATE=0.1OPTIMIZER='adam'setting(...)has the signature:defsetting(default:T,**kwargs:T)->T...whereTisUnion[int, float, str, bool, list, dict, NoneType]andkwargsis a map from group names to values. Specifying parameters by value is shorthand for asettingwith no groups, i.e.SOME_KEY = 'value'is equivalent toSOME_KEY = setting(default='value').When using groups you must first define the group names using theconfig.groupslist.Then in your code you can use these settings like in the basic exampledefmain():print(f'NUM_LAYERS is{config.NUM_LAYERS}')print(f'NUM_UNITS is{config.NUM_UNITS}')print(f'USE_SKIP_CONNECTIONS is{config.USE_SKIP_CONNECTIONS}')print(f'LEARNING_RATE is{config.LEARNING_RATE}')print(f'OPTIMIZER is{config.OPTIMIZER}')if__name__=='__main__':config.parse_args()main()Now we see an additional--configurationoption when we ask for--help:$pythongroup_example.py--helpusage: group_example.py [-h] [--configuration {default,large,small}][--num-layers NUM_LAYERS] [--num-units NUM_UNITS][--use-skip-connections {true,false}][--learning-rate LEARNING_RATE] [--optimizer OPTIMIZER]optional arguments:-h, --help            show this help message and exit--configuration {default,large,small}, -c {default,large,small}--num-layers NUM_LAYERS--num-units NUM_UNITS--use-skip-connections {true,false}--learning-rate LEARNING_RATE--optimizer OPTIMIZERThe--configurationoption specifies the configuration group, and defaults todefaultif omitted.For example:$pythongroup_example.pyNUM_LAYERS is 5NUM_UNITS is 128USE_SKIP_CONNECTIONS is TrueLEARNING_RATE is 0.1OPTIMIZER is adam$pythongroup_example.py--configurationlargeNUM_LAYERS is 10NUM_UNITS is 512USE_SKIP_CONNECTIONS is TrueLEARNING_RATE is 0.1OPTIMIZER is adam$pythongroup_example.py--configurationsmallNUM_LAYERS is 3NUM_UNITS is 32USE_SKIP_CONNECTIONS is FalseLEARNING_RATE is 0.1OPTIMIZER is adamWe can also override individual settings in conjunction with groups. Individual settings take precedence over the group setting:$pythongroup_example.py--configurationlarge--num-units768NUM_LAYERS is 10NUM_UNITS is 768USE_SKIP_CONNECTIONS is TrueLEARNING_RATE is 0.1OPTIMIZER is adamTestingWhen writing tests, you can use theconfig.overridecontext manager to override individual settings:importpytestfrom.configimportconfigdeftest_with_context_manager():withconfig.override(FLOAT_SETTING=0.8,A_BOOLEAN=True):assertconfig.FLOAT_SETTING*config.SOME_INTEGER_SETTING==8assertconfig.A_BOOLEANisTrueTheconfig.overridemethod can also be used without context management in conjunction withconfig.reset_to_defaults:deftest_with_manual_reset():config.override(FLOAT_SETTING=0.8,A_BOOLEAN=True)assertconfig.FLOAT_SETTING*config.SOME_INTEGER_SETTING==8assertconfig.A_BOOLEANisTrueconfig.reset_to_defaults()Btw, this has nothing to do with Chef's Effortless Config project, I forgot to google the name before I put it on pypi and now I guess I'm stuck with it.