Package: pythonic-config
ConfigThis package allows you to configure functions explicitly and safely. You will be able to create an
intuitive type-checked configuration file that directly sets function arguments, globally.This is a lightweight package with only two widely used dependencies and only a couple hundred line
of code.ContentsConfig-ContentsInstallUsage 🤗Configuring a functionWriting a configuration fileConfiguring via the command lineLogging the configurationAdvanced: Sharing configurations between processesAdvanced: Ensuring the configuration is usedAdvanced: Find unused configurationsInstallMake sure you have Python 3, then you can installpythonic-configusingpip:pipinstallpythonic-configInstall the latest code via:pipinstallgit+https://github.com/PetrochukM/Config.gitUsage 🤗Configuring a functionAny function can be configured, and then used anywhere, see below:importconfigascf# Define functiondefdo_something_cool(how_many_times:int):pass# Configure functioncf.add({do_something_cool:cf.Args(how_many_times=5)})# Use the configured function anywhere! 🎉do_something_cool(how_many_times=cf.get())This approach is simple but powerful. Now, each configuration can be directly attributed to a
documented function argument.Furthermore,configincorporatestypeguard💂‍♀️ so every configuration is type checked at runtime.Writing a configuration fileThe simple example above can be extended to create a configuration file, for example:importconfigascfimportdataimporttraincf.add({data.get_data:cf.Args(train_data_path="url_lists/all_train.txt",val_data_path="url_lists/all_val.txt"),data.dataset_reader:cf.Args(type_="cnn_dm",source_max_tokens=1022,target_max_tokens=54,),train.make_model:cf.Args(type_="bart"),train.Trainer.make_optimizer:cf.Args(type_="huggingface_adamw",lr=3e-5,correct_bias=True)train.Trainer.__init__:cf.Args(num_epochs=3,learning_rate_scheduler="polynomial_decay",grad_norm=1.0,)})With this approach, this configuration file will make it clear which (hyper)parameters are set and
where. This improves overall readability of the configuration file.🐍 Last but not least, the configuration file is written in Python, you can use variables, lambdas,
etc to further modularize.Configuring via the command lineIn case you want to change one variable at a time, this package supports configuration from the
command line, for example:python example.py --sorted='Args(reverse=True)'importsysimportconfigascfcf.add(cf.parse_cli_args(sys.argv[1:]))Logging the configurationLastly, it's useful to track the configuration file by logging it. This package supports that
viaconfig.log. In the example below, we log the configuration toComet.fromcomet_mlimportExperimentimportconfigascfexperiment=Experiment()experiment.log_parameters(cf.log())Advanced: Sharing configurations between processesIn multiprocessing, it may be useful to share the configuration file between processes. In this
case, the configuration can be exported to another process and then subsequently imported, see
below:frommultiprocessingimportProcessimportconfigascfdefhandler(configs:cf.Config):cf.add(configs)if__name__=="__main__":process=Process(target=handler,args=(cf.export(),))process.start()process.join()Advanced: Ensuring the configuration is usedIn a large code base, it might be hard to tell if the configuration has been set for every function
call. In this case, we've exposedconfig.tracewhich can double check every function call
against the configuration, see below:importsysimportconfigascfdefconfigured(a=111):passsys.settrace(cf.trace)cf.add({configured:cf.Args(a=1)})configured()# `cf.trace` issues a WARNING!configured(a=cf.get())We also have another option for faster tracing withconfig.enable_fast_trace. Instead of a system
wide trace, this traces the configured functions by modifying their code and inserting a trace
function at the beginning of the function definition. This has a MUCH lower overhead; however, it is
still in beta due to the number of edge cases.Advanced: Find unused configurationsIn a large code base, you may have a lot of configurations, some of which are no longer being used.purgecan be run on a process exit, and it'll warn you if configurations were not used.importatexitimportconfigascfatexit.register(cf.purge)