Package: scriptconfig
Read the docshttps://scriptconfig.readthedocs.ioGitlab (main)https://gitlab.kitware.com/utils/scriptconfigGithub (mirror)https://github.com/Kitware/scriptconfigPypihttps://pypi.org/project/scriptconfigThe main webpage for this project is:https://gitlab.kitware.com/utils/scriptconfigThe goal ofscriptconfigis to make it easy to be able to define a CLI bysimply defining a dictionary. This enables you to write simple configs and
update from CLI, kwargs, and/or json.Thescriptconfigmodule provides a simple way to make configurable scripts
using a combination of config files, command line arguments, and simple Python
keyword arguments. A script config object is defined by creating a subclass ofConfigwith a__default__dict class attribute. An instance of a customConfigobject will behave similar a dictionary, but with a few
conveniences.InstallationThescriptconfig.  package can be installed via pip:pipinstallscriptconfigExample ScriptScriptconfig is used to define a flat configuration dictionary with values that
can be specified via Python keyword arguments, command line parameters, or a
yaml config file. Consider the following script that prints its config, opens a
file, computes its hash, and then prints it to stdout.importscriptconfigasscfgimporthashlibimportubeltasubclassFileHashConfig(scfg.Config):"""
    The docstring will be the description in the CLI help
    """__default__={'fpath':scfg.Value(None,position=1,help='a path to a file to hash'),'hasher':scfg.Value('sha1',choices=['sha1','sha512'],help='a name of a hashlib hasher'),}defmain(**kwargs):config=FileHashConfig(data=kwargs,cmdline=True)print('config ={!r}'.format(config))fpath=config['fpath']hasher=getattr(hashlib,config['hasher'])()withopen(fpath,'rb')asfile:hasher.update(file.read())hashstr=hasher.hexdigest()print('The{hasher}hash of{fpath}is{hashstr}'.format(hashstr=hashstr,**config))if__name__=='__main__':main()If this script is in a modulehash_demo.py(e.g. in the examples folder of
this repo), it can be invoked in these following ways.Purely from the command line:# Get helppythonhash_demo.py--help# Using key-val pairspythonhash_demo.py--fpath=$HOME/.bashrc--hasher=sha1# Using a positional arguments and other defaultspythonhash_demo.py$HOME/.bashrcFrom the command line using a yaml config:# Write out a config fileecho'{"fpath": "hashconfig.json", "hasher": "sha512"}'>hashconfig.json# Use the special `--config` cli arg provided by scriptconfigpythonhash_demo.py--config=hashconfig.json# You can also mix and match, this overrides the hasher in the config with sha1pythonhash_demo.py--config=hashconfig.json--hasher=sha1Lastly you can call it from good ol’ Python.importhash_demohash_demo.main(fpath=hash_demo.__file__,hasher='sha512')Example Script (New Syntax)NEW in 0.6.2: there is now a more concise syntax available using a scriptconfig.DataConfig.
The equivalent version of the above code is:importscriptconfigasscfgimporthashlibclassFileHashConfig(scfg.DataConfig):"""
    The docstring will be the description in the CLI help
    """fpath=scfg.Value(None,position=1,help='a path to a file to hash')hasher=scfg.Value('sha1',choices=['sha1','sha512'],help='a name of a hashlib hasher')defmain(**kwargs):config=FileHashConfig.cli(data=kwargs)print('config ={!r}'.format(config))fpath=config['fpath']hasher=getattr(hashlib,config['hasher'])()withopen(fpath,'rb')asfile:hasher.update(file.read())hashstr=hasher.hexdigest()print('The{hasher}hash of{fpath}is{hashstr}'.format(hashstr=hashstr,**config))if__name__=='__main__':main()This can be invoked from the examples folder similarly to the above script
(replacehash_data.pywithhash_data_datconfig.py.)Project Design GoalsWrite Python programs that can be invoked either through the commandline
or via Python itself.Drop in replacement for any dictionary-based configuration system.Intuitive parsing (currently working on this), ideally improve on
argparse if possible. This means being able to easily specify simple
lists, numbers, strings, and paths.To get started lets consider some example usage:>>>importscriptconfigasscfg>>># In its simplest incarnation, the config class specifies default values.>>># For each configuration parameter.>>>classExampleConfig(scfg.Config):>>>__default__={>>>'num':1,>>>'mode':'bar',>>>'ignore':['baz','biz'],>>>}>>># Creating an instance, starts using the defaults>>>config=ExampleConfig()>>># Typically you will want to update default from a dict or file.  By>>># specifying cmdline=True you denote that it is ok for the contents of>>># `sys.argv` to override config values. Here we pass a dict to `load`.>>>kwargs={'num':2}>>>config.load(kwargs,cmdline=False)>>>assertconfig['num']==2>>># The `load` method can also be passed a json/yaml file/path.>>>config_fpath='/tmp/foo'>>>open(config_fpath,'w').write('{"num": 3}')>>>config.load(config_fpath,cmdline=False)>>>assertconfig['num']==3>>># It is possible to load only from CLI by setting cmdline=True>>># or by setting it to a custom sys.argv>>>config.load(cmdline=['--num=4'])>>>assertconfig['num']==4>>># Note that using `config.load(cmdline=True)` will just use the>>># contents of sys.argvNotice in the above example the keys in your default dictionary are command
line arguments and values are their defaults.  You can augment default values
by wrapping them inscriptconfig.Valueobjects to encapsulate information
like help documentation or type information.>>>importscriptconfigasscfg>>>classExampleConfig(scfg.Config):>>>__default__={>>>'num':scfg.Value(1,help='a number'),>>>'mode':scfg.Value('bar',help='mode1 help'),>>>'mode2':scfg.Value('bar',type=str,help='mode2 help'),>>>'ignore':scfg.Value(['baz','biz'],help='list of ignore vals'),>>>}>>>config=ExampleConfig()>>># smartcast can handle lists as long as there are no spaces>>>config.load(cmdline=['--ignore=spam,eggs'])>>>assertconfig['ignore']==['spam','eggs']>>># Note that the Value type can influence how data is parsed>>>config.load(cmdline=['--mode=spam,eggs','--mode2=spam,eggs'])The above examples are even more concise with the new DataConfig syntax.>>>importscriptconfigasscfg>>># In its simplest incarnation, the config class specifies default values.>>># For each configuration parameter.>>>classExampleConfig(scfg.DataConfig):>>>num=1>>>mode='bar'>>>ignore=['baz','biz']>>># Creating an instance, starts using the defaults>>>config=ExampleConfig()>>>assertconfig['num']==1>>># Or pass in known data. (load as shown in the original example still works)>>>kwargs={'num':2}>>>config=ExampleConfig.cli(default=kwargs,cmdline=False)>>>assertconfig['num']==2>>># The `load` method can also be passed a json/yaml file/path.>>>config_fpath='/tmp/foo'>>>open(config_fpath,'w').write('{"mode": "foo"}')>>>config.load(config_fpath,cmdline=False)>>>assertconfig['num']==2>>>assertconfig['mode']=="foo">>># It is possbile to load only from CLI by setting cmdline=True>>># or by setting it to a custom sys.argv>>>config=ExampleConfig.cli(argv=['--num=4'])>>>assertconfig['num']==4>>># Note that using `config.load(cmdline=True)` will just use the>>># contents of sys.argvFeaturesSerializes to jsonDict-like interface. By default aConfigobject operates independent of config files or the command line.Can create command line interfacesCan directly create an independent argparse objectCan use special command line loading usingself.load(cmdline=True). This extends the basic argparse interface with:Can specify options as either--optionvalueor--option=valueDefault config options allow for “smartcasting” values like lists and pathsAutomatically add--config,--dumps, and--dumpCLI options
when reading cmdline viaload.Inheritence unions configs.GotchasCLI Values with commas:When usingscriptconfigto generate a command line interface, it uses a
function calledsmartcastto try to determine input type when it is not
explicitly given. If you’ve ever used a program that tries to be “smart” you’ll
know this can end up with some weird behavior. The case where that happens here
is when you pass a value that contains commas on the command line. If you don’t
specify the default value as ascriptconfig.Valuewith a specifiedtype, if will interpret your input as a list of values. In the future we
may change the behavior ofsmartcast, or prevent it from being used as a
default.Boolean flags and positional arguments:scriptconfigalways provides a key/value way to express arguments. However, it also
recognizes that sometimes you want to just type--flagand not--flag=1.
We allow for this forValueswithisflag=1, but this causes a
corner-case ambituity with positional arguments. For the following example:classMyConfig(scfg.DataConfig):arg1=scfg.Value(None,position=1)flag1=scfg.Value(False,isflag=True,position=1)For--flag1We cannot determine if you wanted{'arg1': 1, 'flag1': False}or{'arg1': None, 'flag1': True}.This is fixable by either using strict key/value arguments, expressing all
positional arguments before using flag arguments, or using the `` – ``
construct and putting all positional arguments at the end. In the future we may
raise an AmbiguityError when specifying arguments like this, but for now we
leave the behavior undefined.FAQQuestion: How do I override the default values for a scriptconfig object using json file?Answer:  This depends if you want to pass the path to that json file via the command line or if you have that file in memory already.  There are ways to do either. In the first case you can pass--config=<path-to-your-file>(assuming you have set thecmdline=Truekeyword arg when creating your config object e.g.:config = MyConfig(cmdline=True). In the second case when you create an instance of the scriptconfig object pass thedefault=<yourdict>when creating the object: e.g.config = MyConfig(default=json.load(open(fpath,'r'))).  But the special--config--dumpand--dumpsCLI arg is baked into script config to make this easier.Related SoftwareI’ve never been completely happy with existing config / argument parser
software. I prefer to not use decorators, so click and to some extend hydra are
no-gos. Fire is nice when you want a really quick CLI, but is not so nice if
you ever go to deploy the program in the real world.The builtin argparse in Python is pretty good, but I with it was easier to do
things like allowing arguments to be flags or key/value pairs. This library
uses argparse under the hood because of its stable and standard backend, but
that does mean we inherit some of its quirks.The configargparse library - like this one - augments argparse with the ability
to read defaults from config files, but it has some major usage limitations due
to its implementation and there are better options (like jsonargparse). It also
does not support the use case of calling the CLI as a Python function very
well.The jsonargparse library is newer than this one, and looks very compelling.  I
feel like the definition of CLIs in this library are complementary and I’m
considering adding support in this library for jsonargparse because it solves
the problem of nested configurations and I would like to inherit from that.
Keep an eye out for this feature in future work.Hydra -https://hydra.cc/docs/intro/OmegaConf -https://omegaconf.readthedocs.io/en/latest/index.htmlArgparse -https://docs.python.org/3/library/argparse.htmlJsonArgparse -https://jsonargparse.readthedocs.io/en/stable/index.htmlFire -https://pypi.org/project/fire/Click -https://pypi.org/project/click/ConfigArgparse -https://pypi.org/project/ConfigArgParse/TODO[ ] Nested Modal CLI’s[X] Policy on nested heirachies (currently disallowed) - jsonargparse will be the solution here.[ ] How to best integrate with jsonargparse[ ] Policy on smartcast (currently enabled)[ ] Find a way to gracefully way to make smartcast do less. (e.g. no list parsing, but int is ok, we may think about accepting YAML)[X] Policy on positional arguments (currently experimental) - we have implemented them permissively with one undefined corner case.[X] Fixed length - nope[X] Variable length[X] Can argparse be modified to always allow for them to appear at the beginning or end? - Probably not.[x] Can we get argparse to allow a positional arg change the value of a prefixed arg and still have a sane help menu?[x] Policy on boolean flags - See theisflagargument ofscriptconfig.Value[x] Improve over argparse’s default autogenerated help docs (needs exploration on what is possible with argparse and where extensions are feasible)