Package: typed-config-aws-sources
typed-config-aws-sourcesAWS config sources for thetyped-configpackage.pip install typed-config-aws-sourcesRequires python 3.6 or above.Basic usagePlease read the readme fortyped-configfirst.# my_app/config.pyfromtypedconfig_awssourceimportIniS3ConfigSourcefromtypedconfigimportConfig,key,section@section('database')classAppConfig(Config):port=key(cast=int)config=AppConfig()config.add_source(IniS3ConfigSource('my_bucket_name','config_key.cfg'))config.read()# my_app/main.pyfrommy_app.configimportconfigprint(config.host)Supplied Config SourcesIniS3ConfigSourceThis loads configuration from an INI file stored in an S3 bucketfromtypedconfig_awssourceimportIniS3ConfigSourcesource=IniS3ConfigSource('bucket_name','key_name.cfg',encoding='utf8',must_exist=True)Supply the bucket name and key name as the first two argumentsencodingdefaults to'utf8'if not suppliedmust_existdefaults toTrueif not supplied. Ifmust_existisFalse, and the bucket or key can't be found, or AWS credentials fail, then no error is thrown and this config source will just return than it cannot find the requested config value every time.An example INI file might look like this:[database]port=2000DynamoDBConfigSourceThis reads configuration from a DynamoDB table. The table should have a partition key which holds the config section, a sort key which holds the config key name, and another 'column' containing the config value as a string.So an item in DynamoDB corresponding to the above INI file example would look like this{"section":"database","key":"port","value":"2000"}Create theDynamoDBConfigSourcelike this:fromtypedconfig_awssourceimportDynamoDbConfigSourcesource=DynamoDbConfigSource('table_name',section_attribute_name='config_section_column_name',key_attribute_name='config_key_column_name',value_attribute_name='config_value_column_name')The first argument is the DynamoDB table name and is requiredThe other three arguments are optional, and are supplying the attribute (or "column") names in the table which store the three things defining a config parameter (section, key, and value)Default attribute names are"section","key", and"value"SecretsManagerConfigSourceThis reads secret values from secrets manager. Permission to read AWS secrets is required. One secrets should be stored for each config section with the name formatprefix/section, and contain json key-value pairs. For example, for a project calledmyprojectthere may be a secret calledmyproject/databasecontaining the following value. Note that even numeric values should be stored as strings.{"user":"secretuser","password":"secretpassword"}Create theSecretsManagerConfigSourcelike this:fromtypedconfig_awssourceimportSecretsManagerConfigSourcesource=SecretsManagerConfigSource('myproject',must_exist=False,only_these_keys={('s','a'),('s','b')})The first argument passed is the prefix which is placed before the/in the secret name. So when I try to get the database password, the secretmyproject/databaseis retrieved, the JSON is parsed and value the fieldpasswordis returned.Themust_existargument specifies whether to error if AWS secretsmanager cannot be accessed, or if the key does not exist. Default isFalse.Theonly_these_keysargument specifies a limited set of configuration keys. They are provided as(section, key)tuples. If provided, the config source will only act when these parameters are requested. This prevents unnecessary AWS API calls, which slow down configuration setup, for config values which you know are not available from secrets manager. Settingonly_these_keys=None(the default) will check secrets manager for all config keys.ParameterStoreConfigSourceThis reads (optionally secret) values from AWS SSM parameter store. Storing secrets here is cheaper than using secrets manager. Permission to read from SSM parameter store is required. Each config parameter should be stored in parameter store as an individualSecureStringparameter. For example, I would store the database password in a key calledmyproject/database/passwordwheredatabaseis the section name andpasswordis the configuration key name.Create aParameterStoreConfigSourcelike this:fromtypedconfig_awssourceimportParameterStoreConfigSourcesource=ParameterStoreConfigSource('myproject',must_exist=False,only_these_keys={('s','a'),('s','b')},batch_preload=False)The first argument passed is the prefix at the start of the SSM parameter name, before the first/.Themust_existargument specifies whether to error if AWS parameter store cannot be accessed, or if the requested key does not exist.Theonly_these_keysargument specifies a limited set of configuration keys. They are provided as(section, key)tuples. If provided, the config source will only act when these parameters are requested. This prevents unnecessary AWS API calls, which slow down configuration setup, for config values which you know are not available from parameter store. Settingonly_these_keys=None(the default) will check parameter store for all config keys.Thebatch_preloadargument specifies whether to makeGetParametersByPathAWS API calls when the source is constructed. Only applicable whenonly_these_keysis supplied. Ifonly_these_keysisNone, this argument is ignored. This is an optimisation which means that everything supplied inonly_these_keyscan be loaded in far fewer API calls, thus speeding up configuration loading.ContributingIdeas for new features and pull requests are welcome. PRs must come with tests included. This was developed using Python 3.7 but Travis tests run with v3.6 too.Development setupClone the git repositoryCreate a virtual environmentvirtualenv venvActivate the environmentvenv/scripts/activateInstall development dependenciespip install -r requirements.txtRunning testspytestTo run with coverage:pytest --covMaking a releaseBump version number intypedconfig_awssource/__version__.pyAdd changes toCHANGELOG.mdCommit your changes and tag withgit tag -a 0.1.0 -m "Summary of changes"Travis will deploy the release to PyPi for you.Staging releaseIf you want to check how a release will look on PyPi before tagging and making it live, you can do the following:pip install twineif you don't already have itBump version number intypedconfig_awssource/__version__.pyClear the dist directoryrm -r distpython setup.py sdist bdist_wheeltwine check dist/*Upload to the test PyPItwine upload --repository-url https://test.pypi.org/legacy/ dist/*Check all looks ok athttps://test.pypi.org/project/typed-config-aws-sourcesIf all looks good you can git tag and push for deploy to live PyPiHere isa good tutorialon publishing packages to PyPI.