Package: yamlparser
YAML Configuration and Argument ParsingThis package is designed to combine the parsing of configuration files and command line options.
Basically, configuration files (currently we support YAML files only) will be parsed and returned in aNameSpace data structure.
Additionally, all options contained in the configuration files will be added to acommand line parser, so that any option from the command line can be overwritten in thatNameSpace-- and all other options will be added to theNameSpaceas well.DISCLAIMERThis package is work in progress. Currently, only YAML configuration files andargparseparsers are supported.
Also, the documentation might be incomplete and not be up-to-date yet.InstallationThis package is available on the Python Package Index, so you can simply do a pip install:pip install yamlparserFor the latest version from github, you can also use pip install:pip install git+https://github.com/AIML-IfI/yamlparser.gitGetting HelpIf you find a bug in the code or wish to propose changes, feel free to file an issue or open a merge request.
Please contactsiebenkopf@googlemail.comin all other cases of issues.DocumentationNameSpaceTheNameSpaceclass is designed to hold configuration options.NameSpace's can be constructed from any type of nested dictionary:namespace = yamlparser.NameSpace({
    "name" : "Jon Doe",
    "age"  : 42,
    "address" : {
        "street" : "Main Street",
        "number" : 10
    }
})Each nested dictionary internally is transferred into a sub-namespace.
Please note that keys in the dictionary are restricted to valid python variable names.Similarly,NameSpace's can load these dictionaries directly from a YAML file:namespace = yamlparser.NameSpace("config.yaml")The options contained in aNameSpacecan be accessed either as attributes, or via indexing:namespace.name
namespace["age"]Sub-namespaces follow the exact same principle:namespace.address.street
namesapce["address"]["number"]Generally, values can be added to aNameSpaceby simple assignment:namespace.address.city = "Zurich"
namespace["address"]["zip"] = 8050It is even possible to create new sub-namespaces on the fly:namespace.children.daughter = "Jane Doe"
namespace["children"].son = "Jake Doe"NameSpaceobjects can be written to YAML files:namespace.save("path/to/my/file.yaml")You can also turn thisNameSpaceinto a fully-quoted dictionary, where sub-namespaces are separated using a period.to avoid name clashes:namespace.attributes()will return:{
    'name': 'Jon Doe',
    'age': 42,
    'address.street': 'Main Street',
    'address.number': 10,
    'address.city': 'Zurich',
    'address.zip': 8050,
    'children.daughter': 'Jane Doe',
    'children.son': 'Jake Doe'
}You canfreezeandunfreezeaNameSpaceobject.
Frozen objects cannot be modified or extended in any way.Finally, you can format a given string with the contents of aNameSpace.
Simply use fully-quoted embraced keys, and you will get the according values.
This feature is particularly useful when you want to build file names according to configurations.
Given thenamespaceobject from above, you can query:namespace.format("Name={name}, Address={address.street} {address.number}")to obtain the result"Name=Jon Doe, Address=Main Street 10".
Please note that format options (such as used in f-strings) are not (yet) supported.
Also, if the key is not part of thenamespace.attributes(), the entry will not be replaced:namespace.format("Name={name}, Unknown={unknown}")will result in"Name=Jon Doe, Unknown={unknown}".
Most importantly, theNameSpaceclass has aformat_allfunction, which formats all internal variables.
This also works recursively (see the "{extension}" and "{module.extension}" example below:namespace = yamlparser.NameSpace({
    "path" : "/path/to/this",
    "file" : "{path}/file{module.extension}",
    "module" : {
        "extension" : ".txt",
        "file1" : "{path}/file1{module.extension}",
        "file2" : "{path}/file2{extension}",
    }
})

namespace.format_self()
print(namespace.dump())

file: /path/to/this/file.txt
module:
    extension: .txt
    file1: /path/to/this/file1.txt
    file2: /path/to/this/file2.txt
path: /path/to/thisParserThe main of this package is to combine configurations read from YAML files with command line parsing.
Precisely, we want to automatically be able to overwrite any parameter that is contained in a configuration file on the command line, but keep the default if it is not updated.
For this purpose, we provide a simple functionconfig_parser, which is called in thescript.pythat you can find in the main directory and writes the configuration to console:[content of script.py]
import yamlparser
namespace = yamlparser.config_parser()
print(namespace.dump())Theconfig_parserfunction internally creates anargparseparser that requests for a (list of) configuration files.
$ python script.py --helpusage: [-h] configuration_files [configuration_files ...]

positional arguments:
  configuration_files  The configuration files to parse. From the second config onward, it be key=value pairs to create sub-configurations

optional arguments:
  -h, --help           show this help message and exitWhen presenting a configuration file, it is automatically parsed and all its contents are added to the parser:$ python script.py config.yaml --help

usage: script.py [-h] [--name NAME] [--age AGE] [--address.street ADDRESS.STREET] [--address.number ADDRESS.NUMBER] configuration_files [configuration_files ...]

positional arguments:
  configuration_files   The configuration files to parse. From the second config onward, it be key=value pairs to create sub-configurations

optional arguments:
  -h, --help            show this help message and exit
  --name NAME           Overwrite value for name, default=Jon Doe
  --age AGE             Overwrite value for age, default=42,
  --address.street ADDRESS.STREET
                        Overwrite value for address.street, default=Main Street
  --address.number ADDRESS.NUMBER
                        Overwrite value for address.number, default=10When removing the--helpoption, you can see the parser configurations (the default behavior ofscript.py):$ python script.py config.yaml

address:
    number: 10
    street: Main Street
age: 42
name: Jon DoeYou can overwrite any of these options on the command line:$ python script.py config.yaml --name "Jane Doe" --address.number 911

address:
    number: 911
    street: Main Street
age: 42
name: Jane DoeNote that by default, the options infer the data types from the YAML file, e.g., if the YAML file contains an integer, only integer values are accepted:$ python script.py config.yaml --age 12.5

usage: script.py [-h] [--name NAME] [--age AGE] [--address.street ADDRESS.STREET] [--address.number ADDRESS.NUMBER] configuration_files [configuration_files ...]
script.py: error: argument --age: invalid int value: '12.5'At least one configuration file needs to be present, but more than one file can be specified, in which case configurations of the former files are overwritten by latter files.
It is also possible to add configuration files into sub-namespaces by defining aname=file.yamlon command line:$ python script.py config.yaml data=config.yaml

address:
    number: 10
    street: Main Street
age: 42
data:
    address:
        number: 10
        street: Main Street
    age: 42
    name: Jon Doe
name: Jon DoeAdditionally, we wish to be able to add command line options to our configurations that do not appear in any configuration file.
This can be done programmatically by providing a parser with specific options, and example is provided inextend.py:[content of extended.py]
import yamlparser, argparse
parser = argparse.ArgumentParser()
parser.add_option("--haircolor")
parser.add_option("--dob.year", type=int)
parser.add_option("--dob.month", type=int, default=8)
namespace = yamlparser.config_parser(parser=parser)
print(namespace.dump())When calling this script, the selected options will be added to the options:$ python extended.py config.yaml --help

usage: extended.py [-h] [--haircolor HAIRCOLOR] [--dob.year DOB.YEAR] [--dob.month DOB.MONTH] [--name NAME] [--age AGE] [--address.street STREET] [--address.number NUMBER]
                  configuration_files [configuration_files ...]

positional arguments:
  configuration_files   The configuration files to parse. From the second config onward, it be key=value pairs to create sub-configurations

optional arguments:
  -h, --help            show this help message and exit
  --haircolor HAIRCOLOR
                        Set hair color
  --dob.year DOB.YEAR   Set year of birth
  --dob.month DOB.MONTH
                        Set month of birth, default=8
  --name NAME           Overwrite value for name, default=Jon Doe
  --age AGE             Overwrite value for age, default=42
  --address.street STREET
                        Overwrite value for address.street, default=Main Street
  --address.number NUMBER
                        Overwrite value for address.number, default=10Any selected option will be reflected in the returned namespace:$ python extended.py config.yaml --haircolor Brown

address:
    number: 10
    street: Main Street
age: 42
dob:
    month: 8
haircolor: Brown
name: Jon DoePlease note that options without default values that are not provided on the command line are not represented in the configuration.
We propose to provide default values for all options (which cannot beNone) to avoid surprises.Finally, by default, theconfig_parserfunction stores the generatedNameSpaceobject internally, calls itsformat_selffunction and freezes it.
This configuration can be obtained via theyamlparser.get_config()function from anywhere in your source code.
Please note that theconfig_parserfunction should be called only once.