Make your Python program highly configurable
Tags: python, todayilearned, productivity, tutorial
Published: 2021-05-05 09:51:42

The more complex a program is, the more parameters it tends to accept in order to tune it's behavior. **A configurable program is easier to adapt to the needs of the user** (and your own) and reduces the amount of code you have to modify if you want to change certain things.

There are three ways to specify parameters in your code:

1. Directly in the **source code**.
2. Inside a separate **configuration file**.
3. With the arguments from the **command line**.

Each one should override the parameters of the previous one. How do we implement this in a clean way?

## 1. Get all the parameters separately

### From the source code

This needs no explanation. The default configuration is hard-coded.

```python
default_config = {
    "mode": "demo",
    "timeout": 5,
    "color": blue,
    "language": "en"
}
```

### From a configuration file

There are countless ways to read a configuration file, from a custom parsing function to a library like [configparser](https://docs.python.org/3/library/configparser.html). Obviously, your choice will determine the format of the file and each one has its own pros and cons.

To keep this simple, I will use a regular JSON file:

```python
import json

with open("config.json", "r") as f:
    file_config = json.load(f)
```

### From the command line

Again, there are multiple ways to parse the arguments from the command line. I think the most extended is [argparse](https://docs.python.org/3/library/argparse.html), because it is flexible and implements directly a lot of commonly expected behaviour, like an autogenerated help message and more.

However, it makes the code too long for an example like this, so I'll use a custom piece of code to parse arguments with the form `--key=value`.

```python
import sys

params = filter(lambda x: x[:2] == "--", sys.argv)
params = map(lambda x: x.split("="), params)
cli_config = {k[2:]: v for [k,v] in params}
```

## 2. Override values present in lower levels

In Python, the double star (`**`) operator is used to unpack dictionaries. It extracts the key-value pairs to be used elsewhere.

One of the most interesting uses of this operator is to **merge dictionaries**, building a new one from other unpacked. As they only allow unique keys, the dictionaries unpacked on the right replace the values repeated on the left.

```python
final_config = {**default_config, **file_config, **cli_config}
```

## Example

Let's put together what we have:

**main.py**
```python
import sys
import json

## default
default_config = {
    "mode": "demo",
    "timeout": 5,
    "color": "blue",
    "language": "en"
}

print("default", default_config)

## file
with open("config.json", "r") as f:
    file_config = json.load(f)

print("file   ", file_config)

## command line
params = filter(lambda x: x[:2] == "--", sys.argv)
params = map(lambda x: x.split("="), params)
cli_config = {k[2:]: v for [k,v] in params}

print("cli    ", cli_config)

## merge them
final_config = {**default_config, **file_config, **cli_config}

print("final  ", final_config)
```
**config.json**
```json
{
    "color": "red",
    "timeout": 10
}
```
### Output
```bash
$ python main.py --color=green --language=es
default {'mode': 'demo', 'timeout': 5, 'color': 'blue', 'language': 'en'}
file    {'color': 'red', 'timeout': 10}
cli     {'color': 'green', 'language': 'es'}
final   {'mode': 'demo', 'timeout': 10, 'color': 'green', 'language': 'es'}
```

## Conclusion

Making your programs configurable will make a lot of your work easier. Allowing the user (and yourself as a developer) to customize parts of your application without modifying the source code both in a persistent and a dynamic way always makes a difference.

***

***Recommended reading***

{% post miguelmj/how-to-make-an-ethical-crawler-in-python-4o1g %}
{% post miguelmj/i-made-a-recommendation-system-for-stack-overflow-unanswered-questions-280a %}

[You can follow me on Twitter! 🐦](https://twitter/MiguelMJdev)