pydargs 0.11.0
Easily configure a CLI application using a (Pydantic) dataclass.
License: BSD
Requires Python: >=3.9
Keywords: cli, dataclass

# pydargs

Easily configure a CLI application using a (Pydantic) dataclass.

## Usage

Pydargs instantiates a dataclass that is used as (configuration) input of your entrypoint from command line arguments.
For example, in `example.py`:

```python
from dataclasses import dataclass
from pydargs import parse


@dataclass
class Config:
    number: int
    some_string: str = "abc"


def main(config: Config) -> None:
    """Your main functionality"""
    print(f"> Hello {config.number} + {config.some_string}")


if __name__ == "__main__":
    config = parse(Config)
    main(config)
```

Here the `Config` dataclass serves as input (configuration) of the `main` function. Pydargs facilitates
instantiating the `config` instance, allowing the user to use command line arguments to set or override the
values of its fields:


```shell
$ python example.py --number 1
> Hello 1 abc
$ python example.py --number 2 --some-string def
> Hello 2 def
$ python example.py --help
usage: example.py [-h] --number NUMBER [--some-string SOME_STRING]

options:
  -h, --help            show this help message and exit
  --number NUMBER
  --some-string SOME_STRING
                        (default: abc)
```

This saves you from having to maintain boilerplate code such as

```python
from argparse import ArgumentParser
from dataclasses import dataclass


@dataclass
class Config:
    number: int
    some_string: str = "abc"


def main(config: Config) -> None:
    """Your main functionality"""
    print(f"> Hello {config.number} + {config.some_string}")


if __name__ == "__main__":
    parser = ArgumentParser()
    parser.add_argument("--number", type=int)
    parser.add_argument("--some-string", dest="some_string", default="abc")
    namespace = parser.parse_args()
    config = Config(number=namespace["number"], some_string=namespace["some_string"])
    main(config)
```

Aside from that, pydargs supports:
- [a wide variety of field types](#supported-field-types),
- [nested dataclasses](#nested-dataclasses),
- [subparsers / commands](#subparsers),
- [pydantic dataclasses](https://docs.pydantic.dev/latest/concepts/dataclasses/) to help you with validation.

## Installation

Pydargs can be installed with your favourite package manager. For example:

```
pip install pydargs
```

## ArgumentParser arguments

It's possible to pass additional arguments to the underlying `argparse.ArgumentParser` instance by providing them
as keyword arguments to the `parse` function. For example:

```python
config = parse(Config, prog="myprogram", allow_abbrev=False)
```
will disable abbreviations for long options and set the program name to `myprogram` in help messages. For an extensive list of accepted arguments, see [the argparse docs](https://docs.python.org/3/library/argparse.html#argumentparser-objects).

## Supported Field Types

The dataclass can have fields of the base types: `int`, `float`, `str`, `bool`, as well as:

- **Literals** comprised of those types.
- **Enums**, although these
  are [not recommended](https://docs.python.org/3/library/argparse.html#choices) as they do not play nice in the help
  messages. Only the enum _name_ is accepted as a valid input, not the _value_.
- **Bytes**, with an optional `encoding` metadata field:
  `a_value: bytes = field(metadata=dict(encoding="ascii"))`, which defaults to utf-8.
- **Date** and **datetime**, with an optional `date_format` metadata
  field: `your_date: date = field(metadata=dict(date_format="%m-%d-%Y"))`. When not
  provided dates in ISO 8601 format are accepted.
- **Lists** of those types, either denoted as e.g. `list[int]` or `Sequence[int]`.
  Multiple arguments to a `numbers: list[int]` field can be provided as `--numbers 1 2 3`.
  A list-field without a default will require at least a single value to be provided.
  If a default is provided, it will be completely replaced by any arguments, if provided.
- **Optional types**, denoted as e.g. `typing.Optional[int]` or `int | None` (for Python 3.10 and above).
  Any argument passed is assumed to be of the provided type and can never be `None`.
- **Unions of types**, denoted as e.g. `typing.Union[int, str]` or `int | str`. Each argument
  will be parsed into the first type that returns a valid result. Note that this means
  that `str | int` will _always_ result in a value of type `str`.
- Any other type that can be instantiated from a string, such as `Path`.
- Dataclasses that, in turn, contain fields of supported types. See [Nested Dataclasses](#nested-dataclasses).
- A union of multiple dataclasses, that in turn contain fields of supported types,
  which will be parsed in [Subparsers](#subparsers).

## Overriding defaults from a file

Pydargs can also consume values from a JSON- or YAML-formatted file. To enable
this, pass `add_config_file_argument=True` to the `parse` function, which will add a `--config-file`
command line argument. If provided, the values from this file will override the defaults
of the dataclass fields. Any command line arguments passed will override the
defaults provided in the file.

For example, with the following contents in `defaults.json`:

```json
{
  "a": 1,
  "b": "abc"
}
```

then running this code

```python
from dataclasses import dataclass
from pydargs import parse


@dataclass
class Config:
  a: int = 2
  b: str = "def"


if __name__ == "__main__":
  config = parse(Config, add_config_file_argument=True)
```

with the following arguments

`entrypoint --config-file defaults.json --b xyz`

would result in `Config(a=1, b="xyz")`.

Note that:
- Only _defaults_ can be overridden. Any dataclass fields without a default must always be provided on the command line.
- Any extra keys present inside the file but not matching a field in the dataclass will be ignored,
  and if any are present a warning will be raised.
- In order to load defaults from a YAML-formatted file, [PyYAML](https://pyyaml.org/wiki/PyYAMLDocumentation) has
  to be installed. To install it with pydargs, run `pip install pydargs[pyyaml]`.
- The parsed dataclass may not have a field named `config_file`.
- The defaults provided in the file will not be type-casted by pydargs, and hence only JSON-native types are supported.

## Metadata

Additional options can be provided to the dataclass field metadata.

The following metadata fields are supported:

### `positional`
Set `positional=True` to create a positional argument instead of an option.

```python
from dataclasses import dataclass, field

@dataclass
class Config:
  argument: str = field(metadata=dict(positional=True))
```


### `as_flags`

Set `as_flags=True` for a boolean field:
```python
from dataclasses import dataclass, field

@dataclass
class Config:
  verbose: bool = field(default=False, metadata=dict(as_flags=True))
```
which would create the arguments `--verbose` and `--no-verbose` to
set the value of `verbose` to `True` or `False` respectively, instead
of a single option that requires a value like `--verbose True`.

### `parser`

Provide a custom type converter that parses the argument into the desired type. For example:

```python
from dataclasses import dataclass, field
from json import loads

@dataclass
class Config:
  list_of_numbers: list[int] = field(metadata=dict(parser=loads))
```

This would parse `--list-of-numbers [1, 2, 3]` into the list `[1, 2, 3]`. Note that the error message returned
when providing invalid input is lacking any details. Also, no validation is performed to verify that the returned
type matches the field type. In the above example, `--list-of-numbers '{"a": "b"}'` would result in `list_of_numbers`
being the dictionary `{"a": "b"}` without any kind of warning.

### `short_option`

Provide a short option for a field, which can be used as an alternative to the long option.
For example,

```python
from dataclasses import dataclass, field

@dataclass
class Config:
  a_field_with_a_long_name: int = field(metadata=dict(short_option="-a"))
```

would allow using `-a 42` as an alternative to `--a-field-with-a-long-name 42`.

### Ignoring fields
Fields can be ignored by adding the `ignore_arg` metadata field:

```python
@dataclass
class Config:
    number: int
    ignored: str = field(metadata=dict(ignore_arg=True))
```
When indicated, this field is not added to the parser and cannot be overridden with an argument.

### Fields excluded from the `__init__()`
Fields not included in the `__init__()` (i.e. with `init=False`, see [here](https://docs.python.org/3/library/dataclasses.html#dataclasses.field) ) will be ignored by pydargs and cannot be overridden with an argument.

```python
@dataclass
class Config:
    number: int
    ignored: str = field(init=False)
```

This could be useful in combination with a [`__post_init__()` method](https://docs.python.org/3/library/dataclasses.html#post-init-processing) to set the value of the field.


### `help`

Provide a brief description of the field, used in the help messages generated by argparse.
For example, calling `your_program -h` with the dataclass below,

```python
from dataclasses import dataclass, field

@dataclass
class Config:
  an_integer: int = field(metadata=dict(help="any integer you like"))
```

would result in a message like:

```text
usage: your_program [-h] [--an-integer AN_INTEGER]

optional arguments:
  -h, --help               show this help message and exit
  --an-integer AN_INTEGER  any integer you like
```

### `metavar`

Override the displayed name of an argument in the help messages generated by argparse,
as documented [here](https://docs.python.org/3/library/argparse.html#metavar).

For example, with the following dataclass,
```python
from dataclasses import dataclass, field

@dataclass
class Config:
  an_integer: int = field(metadata=dict(metavar="INT"))
```
calling `your_program -h` would result in a message like:

```text
usage: your_program [-h] [--an-integer INT]

optional arguments:
  -h, --help        show this help message and exit
  --an-integer INT
```

## Nested Dataclasses

Dataclasses may be nested; the type of a dataclass field may be another dataclass type:

```python
from dataclasses import dataclass

@dataclass
class Config:
  field_a: int
  field_b: str = "abc"


@dataclass
class Base:
  config: Config
  verbose: bool = False
```

Argument names of fields of the nested dataclass are prefixed with the field name of the nested dataclass in the base
dataclass. Calling `pydargs.parse(Base, ["-h"])` will result in something like:

```text
usage: your_program.py [-h] --config-field-a CONFIG_FIELD_A
                            [--config-field-b CONFIG_FIELD_B]
                            [--verbose VERBOSE]

options:
  -h, --help            show this help message and exit
  --verbose VERBOSE     (default: False)

config:
  --config-field-a CONFIG_FIELD_A
  --config-field-b CONFIG_FIELD_B
                        (default: abc)

```

Please be aware of the following:
- The default (factory) of fields with a dataclass type is ignored by pydargs, which may yield unexpected results.
  E.g., in the example above, `config: Config = field(default_factory=lambda: Config(field_b="def"))` will not result in a default of "def" for field_b when parsed by pydargs.
  Instead, set `field_b: str = "def"` in the definition of `Config`.
  If you must add a default, for example for instantiating your dataclass elsewhere, do `config: Config = field(default_factory=Config)`, assuming that all fields in `Config` have a default.
- Nested dataclasses can not be positional (although _fields of_ the nested dataclass can be).
- Argument names must not collide. In the example above, the `Base` class should not contain a field named `config_field_a`.
- When reading [defaults from a file](#loading-defaults-from-file), the data inside the file may be nested like the dataclasses as well as flat with prefixes.
  E.g. `{"config": {"field_b": "xyz"}` has the same effect as `{"config_field_b": "xyz"}`. Pydargs will raise an
  exception in the case of collisions between keys in alternative formats.

## Subparsers

Dataclasses can contain a field with a union-of-dataclasses type, e.g.:

```python
from dataclasses import dataclass, field
from typing import Union


@dataclass
class Command1:
  field_a: int
  field_b: str = "abc"


@dataclass
class Command2:
  field_c: str = field(metadata=dict(positional=True))


@dataclass
class Base:
  command: Union[Command1, Command2]
  verbose: bool = False
```

This will result in [sub commands](https://docs.python.org/3/library/argparse.html#sub-commands)
which allow calling your entrypoint as `entrypoint --verbose Command1 --field-a 12`.

Calling `pydargs.parse(Base, ["-h"])` will result in something like:

```text
usage: your_program.py [-h] [--verbose VERBOSE] {Command1,command1,Command2,command2} ...

options:
  -h, --help            show this help message and exit
  --verbose VERBOSE     (default: False)

action:
  {Command1,command1,Command2,command2}
```

Note that:
- Also lower-case command names are accepted.
- Any dataclass can not contain more than one subcommand-field.
- Sub-commands can be nested and mixed with nested dataclasses.
- Any positional fields defined after a subcommand-field can not be parsed.
- Subparsers handle all arguments that come after the command; so all global arguments must come before the command.
  In the above example this means that  `entrypoint --verbose Command2 string`
  is valid but `entrypoint Command2 string --verbose` is not.