pj-aa-configuration 1.0.10
This package provides a configuration singleton, see the README.md file
License: Unlicense

# Introduction 
This project aim is to implement a simple and extensible way of accessing configuration. 
The configuration values can be provided in json files, environment variables, secrets, etc.
Anyone can extend the functionality building an overrider extending the AbstractOverrider class.
Please do check the unit tests for more detail on the library usage.

# Getting Started
The configuration class is implemented as a singleton.
Main usage:
- define environment vars:
  - AA_ENV - {dev|prod} environment it will become a config section to be read right after "common" - default: dev
  - AA_CONFIGURATION_DIR - absolute path where the config json files can be found - default: local dir
  - PJ_AA_CONFIGURATION_FILES_PATTERN - pattern for finding the json files in $AA_CONFIGURATION_DIR - default: "configuration.*\\.json"
  - PJ_AA_CONFIGURATION_LAYERS - additional sections on the config file to consider on top of "common" and {environment} - default: None
```
config_value = Configuration.get("root.config_group.key")
```

## rationale
As a singleton, Configuration is loaded when it's called by the first time

The initialization process can be depicted in the following diagram:
```
=> ( read environment vars:
  AA_ENV, 
  AA_CONFIGURATION_DIR [D], 
  PJ_AA_CONFIGURATION_FILES_PATTERN [P], 
  PJ_AA_CONFIGURATION_LAYERS [L]
) 
=> ( find json files that match the regexp pattern [P] in [D] )
=> ( define config layers based on common section, environment and provided layers )
=> ( process json files by natural order, building the config dict structure, 
    setting each value according to layers/section priority: common, <env>, <layers>... )
=> ( resolve what overriders [O] should be enabled and build its list, 
    environment overrider is always the last one to be scanned, so it has higher priority )
```

...configuration values can be read providing keys in 2 formats, property and variable:
- property notation: `*.*.*` == variable notation: `*__*__*` => "app.shared.dev" == "APP__SHARED__DEV"
- property notation: `*.*_*.*` == variable notation: `*__*_*__*` => "app.shared_key.dev" == "APP__SHARED_KEY__DEV"
- important detail: configuration files should define keys __only in lower case alphabet characters (ascii, no numbers)__ 

...additional detail: when reading secrets or environment variables, the key should not include the hyphen, "-", 
if so it will be replaced by an underscore "_"
- variable notation: `*__*-*__*` => `*__*_*__*` , "APP__SHARED-KEY__DEV" => "APP__SHARED_KEY__DEV"

## explanation of configuration layers

By default the library will read two configuration layers from the files found, with the following order:
- "common"
- $AA_ENV => {dev|prod|...}

example:
```
{
  "common": {
      "server": {
        "host": "somewhere.com",
        "user": "common_user",
  },
  "dev": {
      "server": {
        "password": "gonna-get-it-from-keyvault",
        "user": "dev_user",
  }
  "prod": {
      "server": {
        "password": "gonna-get-it-from-keyvault",
        "user": "prod_user",
  }
}
```
- with $AA_ENV == dev:
  - Configuration.get("server.host") == Configuration.get("SERVER__HOST") ==  "somewhere.com"
  - Configuration.get("server.user") == Configuration.get("SERVER__USER") == "dev_user"
- with $AA_ENV == prod:
  - Configuration.get("server.host") == Configuration.get("SERVER__HOST") == "somewhere.com"
  - Configuration.get("server.user") == Configuration.get("SERVER__USER") == "prod_user"

### ...additionally to these layers

...we can define more configuration layers through the environment variable PJ_AA_CONFIGURATION_LAYERS, and eventually
include further conditional configuration:

example: PJ_AA_CONFIGURATION_LAYERS="sampled,other"
```
{
  "common": {
      "server": {
        "host": "somewhere.com",
        "user": "common_user",
      },
      "data": {
          "sampling": false
      }
  },
  "dev": {
      "server": {
        "password": "gonna-get-it-from-keyvault",
        "user": "dev_user"
      }
  },
  "sampled": {
        "data": {
          "sampling": true
          "sampling_rational": "xpto"
        }
  }
  "other": {
        ...
  }
}
```
- with $AA_ENV == dev and $PJ_AA_CONFIGURATION_LAYERS == "sampled":
  - Configuration.get("server.host") == Configuration.get("SERVER__HOST") == "somewhere.com"
  - Configuration.get("server.user") == Configuration.get("SERVER__USER") == "dev_user"
  - Configuration.get("data.sampling") == Configuration.get("DATA__SAMPLING") == True
  
## explanation of overriders

We can implement custom overriders to dynamically override any configuration conveyed in the json files. 
For that we should override the abstract class 
```
pjaaconfiguration.abstract_overrider.AbstractOverrider
```
and provide the import pointer `<module>.<class>`, or multiple pointers separated by a comma, 
in the environment variable $PJ_AA_CONFIGURATION_OVERRIDERS, example:
`export PJ_AA_CONFIGURATION_OVERRIDERS="mymod.MyFirstOverrider,myothermod.MYOverrider"` 

...best way to explain how to create an overrider is probably to look to the provided overriders
code and tests in the library source code:
- `pjaaconfiguration.overrider.airflow_env_overrider.Overrider`
  - loaded by setting the environment variable PJ_AA_CONFIGURATION_ENABLE_AIRFLOW_VARS = "true"
- `pjaaconfiguration.overrider.databricks_keyvault_overrider.Overrider`
  - loaded by setting the environment variable AA_CONFIGURATION_DATABRICKS_KEYVAULT_SCOPE = {the keyvault scope}
- `pjaaconfiguration.overrider.environment_overrider.Overrider`
  - always loaded by default and always the last one to be processed, can override everything else


# test, build and publish
- check the `helper.sh` script


# Contribute
go ahead push a pull request