Package: ext-argparse
ext_argparseThis is an extended version of Python'sargparsethat, in a nutshell, supports these three things:using code-completion to access parameters and/or their values (a.k.a. arguments)configuration file input/output, with the option to override/update files via the command linenatural support for Python's Enums, i.e. categorical parameters.Why another CLI / Config File Library?Over about six years I've found myself reusing a basic version of the code in this library over and over again, because
I didn't find anything that supported CLI & config IO functionality which I thought was natural and comfortable.
Recently, I again found myself in need of such a library and discovered that in all those years, nobody published
a library having all these features and characteristics (although some of these have other features that may or may not
interest you.)Features / characteristics of existing libraries and modules in comparison toext_argparse:Feature (down) \ library (right)argparseconfigparserdocopteasyargsparse_it$click_hydraext_argparseCommand line inputyesyesyesyesyesyesyesEnvironment variable inputyesConfiguration file inputyesyesyesyesConfiguration file outputyesyesArgument code-complete supportyesyesyesyesArgument documentation (--help)yesyesyesyesConfig-file commentsyesyesyesNested argumentsyesyesyesNested commandsyesEnum argument typeyesyesAvoids duplicating names in codekind-ofkind-ofyeskind-ofyesyesAvoids magic stringskind-ofyesyesyesyesAnother difference that also deserves note here is thatext_argparsecurrently supports configuration file IOonlyinYAMLformat. Support for config file comments (via theruamel.yamlpackage) may be easily
integrated, as well as backends for other popular formats, such asTOMLandJSON5. Contributions are welcome!In the meantime,parse_itsupports six formats, including JSON, YAML, TOML, HCL, INI, and XML, but obviously has other
limitations as can be observed in the table.(*) Also of note is thateasyargsand$click_in a sense support code-completion of the arguments, since they provide
a one-to-one mapping between function parameters and CLI parameters. In contrast,ext_argparsedoes this directly via
the fields of the user-provided parameter enumeration, i.e. any class extendingParameterEnum(see usage below).hydradeserves an honorable mention here. It supports code-completion, and arguably in a better way thanext_argparse,
via Python's built-indataclasses. The only downside to that approach
is that documentation strings cannot yet be handled and it's not clear how to provide support for such, although that issue
is beingdiscussed.hydrais the only library besidesext_argparsethat naturally handles Enums and allows to override configuration file arguments from the command line. It also provides
several other features thatext_argparsecurrently doesn't: argument override rules, config groups, etc.InstallationThere are three different ways to installext_argparse. Choose your poison.Via PyPI & Pippip install ext_argparseVia Online Repo & Pip:pip install -e git+https://github.com/Algomorph/ext_argparse@mainFrom Source via Pip:git clone https://github.com/Algomorph/ext_argparseorgit clone git@github.com:Algomorph/ext_argparse.gitThen, from repository root,python3 -m build --wheelorpy -m build --wheeldepending on your platform,and, finally, install the resulting wheel:pip install dist/*.whlBasic Usagefromext_argparseimportParameterEnum,Parameter,process_argumentsclassParameters(ParameterEnum):# name your subclass here however you see fit# Parameter class constructor also accepts arguments for 'nargs' and 'action',# similar to ArgumentParser.add_argumentname=Parameter(default="Frodo Baggins",arg_type=str,arg_help="Name of our hero.")lembas_bread=Parameter(arg_type=int,required=True)height=Parameter(default=1.12,arg_type=float,arg_help="Height in meters.")# ...# Then, somewhere in the main function or body of your program:process_arguments(Parameters,program_help_description="A program for estimating chances of hero at success.")# And access arguments like so:print(f"The name of our dear hero is:{Parameters.name.value}")can_fit_through_narrow_caves=Parameters.height.value<1.3provisions_duration_days=Parameters.lembas_bread.value*2Note that IDEs and editors with proper code autocompletion, such as properly-configured VS Code, Sublime Text, or
PyCharm, will now be able to suggest parameter name completion afterParameters., which may prove useful in situations
with a great number of parameters.Names of CLI parameters are handled similar to the argparse library, e.g. the code above might be run with this command:python3 -m estimate_hero_success.py --name="Samwise Gamgee" --lembas_bread=25 --height=1.21Shorthand notation, by default, is autogenerated using the first letters of the parameter name, e.g.:python3 -m estimate_hero_success.py -n="Samwise Gamgee" -lb=25 -h=1.21If there is a need to avoid shorthand parameter naming conflicts or a need for a custom shorthand,shorthandmay be
passed to the constructor of anyParameter:lembas_bread = Parameter(arg_type=int, required=True, shorthand="lembr")Here, the resulting command might look like:python3 -m estimate_hero_success.py -n="Samwise Gamgee" -lembr=25 -h=1.21Enum Type Support and Positional ArgumentsWe can naturally extend the example above to use Enums and to make the hero's name a positional parameter:fromext_argparseimportParameterEnum,Parameter,process_argumentsfromenumimportEnumclassSpecies(Enum):HOBBIT=0ELF=2ORK=3HUMAN=4WIZARD=5DRAGON=6GIANT_SPIDER=7CAVE_TROLL=8classParameters(ParameterEnum):name=Parameter(default="Frodo Baggins",arg_type=str,arg_help="Name of our hero.",positional=True)lembas_bread=Parameter(arg_type=int,required=True)species=Parameter(default=Species.HOBBIT,arg_type=Species,arg_help="Species of our hero.")# ...# Then, somewhere in the main function or body of your program:process_arguments(Parameters,program_help_description="A program for estimating chances of hero at success.")# Argument access:print(f"The name of our dear hero is:{Parameters.name.value}")provisions_duration_days=Parameters.lembas_bread.value*2ifParameters.species.valueisSpecies.CAVE_TROLL:print("Cave trolls turn to stone at daytime. Let's check what time of day it is...")Our command line might then look like this:python3 -m estimate_hero_success.py "Gandalf the Grey" --lembas_bread=5 --height=1.82 --species=WIZARDNote that just the stringWIZARDis used at the command line, not something likeSpecies.WIZARDor lowercasewizard.Configuration File IOWe can easily handle settings from a configuration file via passing some special settings that arealwayshandled.python3 -m estimate_hero_success.py --lembas_bread=25  --settings_file=lord_of_the_rings/the_two_towers/SamwiseConfig.yamlThe command line above will read the parameter values inYAMLformat from the specified file andoverridethelembas_breadsetting with the argument25. The setting fileSamwiseConfig.yamlfor the example
above might look like this:name:"SamwizeGamgee"lembas_bread:20height:1.21If you also wanted to save the updated settings that you've overridden from the command line, you can use something like:python3 -m estimate_hero_success.py --lembas_bread=25  --settings_file=lord_of_the_rings/the_two_towers/SamwiseConfig.yaml --save_settingsNote: it is up to you to ensure there are no shorthand conflicts withsettings_fileandsave_settings(sfandssshorthands, respectively) via innovative naming or customshorthandarguments to the constructors of yourParameterobjects.Note: any subset of arguments can be provided from either or both sources, with any remaining parameters retaining
default values.Wildcard Parameter ValuesThere is also a special value you can use in string arguments acceptingpathsto resources,!setting_file_location,
that will be substituted with the directory of the setting file inside the program (when such is specified).Imagine we had the following parameter in ourParametersenum above:output_path = Parameter(default="./output", arg_type=str, help="Where to store the output data and report.")We could then pass in the!setting_file_locationwildcard like so:python3 -m estimate_hero_success.py --settings_file=lord_of_the_rings/the_two_towers/SamwiseConfig.yaml --output=!setting_file_locationThe value ofParameters.outputwould belord_of_the_rings/the_two_towers.Note that command line arguments appear and are handledexactlythe same as parameter values inside the settings
file, including theEnumparameters, the!setting_file_locationwildcard, and nested arguments (described in Nested Parameter Support).Loading and Saving Defaults & Current SettingsYou can easily save a settings file filled with default values. Also, you can dump the current settings at any point
during the runtime of your application.fromext_argparseimportParameterEnum,save_defaults,process_arguments,dumpfrompathlibimportPathfromdatetimeimportdatetimeclassParameters(ParameterEnum):pass# // fill in to your heart's content -- see prior examplessave_defaults(Parameters,Path("default_settings.yaml"),save_help_comments=False)process_arguments(Parameters,program_help_description="My favorite app that makes coffee.")timestamp=datetime.now().strftime('%y-%m-%d-%H-%M-%S')# save parameters after they've been processeddump(Parameters,Path(f"output/experiment_run_settings_{timestamp}"))If you want the program to read a specific settings file by default, you can do so by passing adefault_settings_fileto the process_arguments method:process_arguments(Parameters,program_help_description="My favorite app that makes coffee.",default_settings_file="coffee_settings.yaml")If you like, you can even make the program generate the default settings file if it is missing
(useful, for example, for repository code):process_arguments(Parameters,program_help_description="My favorite app that makes coffee.",default_settings_file="coffee_settings.yaml",generate_default_settings_if_missing=True)Auto-Generating Help Comments in Setting FilesThe settings file YAML supports (any number of) comments prepended by#before and after parameters.
You can place custom comments, but you can also save the parameter help as comments before each parameter when you save
settings manually, e.g., for the example above, you can call thedumpfunction like so instead:dump(Parameters,Path(f"output/experiment_run_settings_{timestamp}"),save_help_comments=True,line_length_limit=120)Theline_length_limitallows you to control the length of the comments (which will be indented to match the nested
parameters they describe).Nested Parameter SupportAny level of nesting can be handled both via the configuration file or the command line:fromext_argparseimportParameterEnum,Parameter,process_argumentsfromtypingimportTypeclassQuestParameters(ParameterEnum):name=Parameter(default="Ring Destruction",arg_type=str,arg_help="Name of the quest.")destination=Parameter(default="Mount Doom",arg_type=str,arg_help="Final quest destination.")year=Parameter(default=3018,arg_type=int,arg_help="Year of the Third Age when the quest is to begin")classHeroParameters(ParameterEnum):name=Parameter(default="Frodo Baggins",arg_type=str,arg_help="Name of our hero.")lembas_bread=Parameter(arg_type=int,required=True)height=Parameter(default=1.12,arg_type=float,arg_help="Height in meters.")classParameters(ParameterEnum):output=Parameter(default="!setting_file_location",arg_type=str,arg_help="Directory where to output the detailed analysis data and report.")# Note the usage of typing.Type here to give code completion an additional nudge.# Otherwise, it won't work on the nesting for PyCharm (haven't tested in VS Code or Sublime).hero:Type[HeroParameters]=HeroParametersquest:Type[QuestParameters]=QuestParameters# ...# Then, somewhere in the main function or body of your program:process_arguments(Parameters,program_help_description="A program for estimating chances of hero at success in a particular quest.")# And access arguments like so (note the usage of .hero and .quest):print(f"The name of our dear hero is:{Parameters.hero.name.value}")can_fit_through_narrow_caves=Parameters.hero.height.value<1.3provisions_duration_days=Parameters.hero.lembas_bread.value*2print(f"The name of the quest is:{Parameters.quest.name.value}")aragorn_alive_at_start_of_quest=Parameters.quest.year.value>2931# ...A command line for the above program might look like this:python3 -m estimate_hero_success.py --hero.name="Saruman the White" --hero.lembas_bread=0 --hero.height=1.84 --quest.name="Corruption of Rohan" --quest.year=3014 --quest.destination=EdorasA settings file with the above arguments might look like this:output:'!setting_file_location'hero:name:"SarumantheWhite"lembas_bread:0height:1.21quest:name:"CorruptionofRohan"year:3014destination:"Edoras"Licence InformationThe code is released underApache License V2.