Package: xcon
IntroductionDocumentationQuick StartInstallUsing ItQuick OverviewPlaces Configuration is Retrieved FromParam Store Provider SpecificsSecrets Manager Provider SpecificsCase SensitivityAdd PermissionsCachingTime to liveTable Layout DetailsUnit TestsLicensingIntroductionHelps retrieve configuration information from aws/boto services such as Ssm's Param Store and Secrets Manager,
with the ability the cache a flattened list into a dynamodb table.Right now this ispre-release software, as the dynamo cache table and related need further documentation and testing.Retrieving values from Param Store and Secrets Manager should work and be relatively fast, as we bulk-grab values
at the various directory-levels that are checked.More documentation and testing will be coming soon, for a full 1.0.0 release sometime in the next month or so.Seexcon docs.Documentation📄 Detailed Documentation|🐍 PyPiQuick StartInstall# via pippipinstallxcon# via poetrypoetryaddxconUsing ItFrom the get-go and by default, environmental variables will 'just work'.The main class isConfigviafrom xcon import Config.This class usesxinjectto do dependency injection.
You can easily inject a new version/configuration of the object without having to couple you code too close
together to get your configuration settings.You get the current Config object via:fromxconimportConfigcurrent_config=Config.grab()setting_value=current_config.get('some_setting')An easier way to always use the current Config object is to use a proxy object.# Instead of importing the class, we import a proxy to the currently injected instance:fromxconimportconfigsetting_value=config.get('some_setting')Alternatively, ou can also useConfig.proxy()to get a proxy.# Importing proxy object to current Config injectable dependency.# You can use it as if you did `Config.grab()`, as it does this# for you each time you get something from it.fromxconimportconfigimportos# Setting a environmental variable value to showcase retrieving it.os.environ['SOME_CONFIG_VARIABLE']="my-value"# If you had an environmental variable called `SOME_CONFIG_VARIABLE`, this would find it:assertconfig.get('some_config_variable')=="my-value"# Alternate 'dict; syntax, works just like you would expect.# Just like dict, it will raise an exception if value not found.assertconfig['some_config_variable']=="my-value"Config names are case-insensitive (although directory names are case-sensitive).By default,Config{target=_blank} will look at environmental variables first,
and then other remote places second (the order and where to look is all configurable).Quick OverviewPlaces Configuration is Retrieved FromAs a side note for the below paths theSERVICE_NAMEandAPP_ENVvariables
come fromxcon.xcon_settings.environmentandxcon.xcon_settings.service.
By default, these settings will use theSERVICE_NAMEandAPP_ENVenvironmental variables. You can also set/override them expiclity
by setting a value onexcon.xcon_settings.environmentand/orxcon.xcon_settings.service.By default, Config will look in these paths (first)./{APP_NAME}/{APP_ENV}/{variable_name}/{APP_NAME}/all/{variable_name}/global/{APP_ENV}/{variable_name}/global/all/{variable_name}Details:Standard Directory PathsDirectory ChainFor each directory/path, we go through these providers (second):Environmental VariablesDynamo Config CacheWill be skipped if table/permissions don't exist.AWS Secrets ManagerWill be skipped if needed permissions not in placeAWS Param StoreWill be skipped if needed permissions not in placeDetails:Provider ChainSupported Providers** TODO In the order they are specified above (seeStandard Lookup Order).Param Store Provider SpecificsValues are exclusively retrieved via "GetParametersByPath"; which allows for bulk-retrieval of settings.All settings in a particular directory are retrieved in one request, and then whatever value is needed is returned.
These values are cached within the provider retriever object, so when other config values are asked for
there is a good chance it can return a value without having to go back to param store to ask for another value.Secrets Manager Provider SpecificsSecrets manager does not allow for bulk-retrieval of values.
Instead, you can bulk-request get a list of available secret names viaListSecretVersionIds.The secrets provider will grab the full list, and then use that to know what is or is not available to get.
This makes it much faster, as it can quickly determine if it should attempt to retrieve a value or not based on this list.Case SensitivityThe directory/path is case-sensitive; but theVARIABLE_NAMEpart at the end is case-insensitive.So environmental variables are entirely case-insensitive, as they only have theVAIRABLE_NAMEand no directory path.So you can doconfig.get('some_var_name'), and it would still find a value for it,
even if the name in the source/provider of values isSOME_VAR_NAME.Add PermissionsIf you want to receive values from remote locations, the app will need the correct permissions.For example, AWS's Param Store service will restrict access to the param values by path/directory.There is a serverless permission resource template yaml file you can use directory or copy
and change as needed for your purposes.AWS Configuration Store Permissions:Param StoreSecrets ManagerIf you want to use a dynamo table cache (seecachingin next section), use these:DynamoDB Cache Table:App PermissionsTable DefinitionFor more details, seePermissions.CachingThe purpose of the cache is to limit calls to the providers,
to prevent throttling from them.For example, the AWS Param Store will throttle calls if there are too many per-second,
which could happen if several lambdas get launched and each lookup configuration simultaneously.By default, values that are remotely looked up (ie: non-environmental variables)
are cached in a dynamo table.
Each of these lambdas can first check a DynamoDB cache table first, and if the value they need
is in there it will use that instead of attempting to retrieve values from the providers.When something is not in the cache table, Config will look at each configured provider
and when it finds the value (or lack of a value), it will store what it found in the
dynamo cache table for later faster lookup.The cache is a flattened list of all resolved values from all configured sources.
It will correctly cache according to the current providers, paths, and app environment + service.
Any of these variables can dynamically change, this information is added to each cached
entry so the correct value will be used in any situation.Time to liveThe cache table is configured with a time-to-live attribute (namedttl).
The value is set for 12 hours, after which the item will expire.There is an logarithm built intoConfigcaching mechanism that will
pre-expire items sooner than normal randomly.
The algorith makes it more likely a particular item in the cache will expire
sooner as the expire-time approaches.This means something that will expire in one hour will be more likely to
be pre-expired than something  that has 10 hours left.This helps ensure that if a lambda is very busy and has many concurrent
instances running that it's likely only one of the lambdas would pre-expire
the cached items and 'refresh' them by re-looking up the values from the
providers and re-caching the newly looked up values.This is a way to coordinate cache expiring and refreshing without
having to actually have any coordinating communication happening.This allows the configuration refreshing to automatically scale
with the lambda activity in such a way as to limit the possibility
of being throttled from param store or secrets manager.Table Layout DetailsThe dynamo table has a two-part primary key.The first part of the primary key is a hash key made up of appsxcon.xcon_settings.environmentandxcon.xcon_settings.servicevalues.
This is the 'partition' key in the DynamoDB table, and AWS policies can allow
or deny access based on this hash key.
This allows the table to limit access to cached items by app's environment + service.The second part of the primary key is a range-key made up of all provider names
and directory paths in the order they are looked up in. This allows multiple
values to be stored for the same config setting, depending on which providers
and directory paths were used to lookup the config setting.This allows all looked up values for all dynamic situations to be cached
and used correctly.For details seeCaching Details,Historical Background.Unit TestsBy default, unit tests will always start with a Config object that has caching disabled,
and only uses the environmental provider (ie: only looks at environmental variables).This is accomplished via an autouse fixture in a pytest plugin module
(see plugin modulexcon.pytest_pluginor fixturexcon.pytest_plugin.xcon).If a project hasxconas a dependency, pytest will find this plugin module
and automatically use it. Nothing more to do.As an FYI/side-note: There is axinject.pytest_plugin.xyn_contextthat will also
automatically  configure a blank context for each unit test.This does mean you must configure Config using a fixture or at the top of your unit test method,
as any changes at the module-level will be forgotten.The reason we do this is it guarantees that resources/config changes won't be propagaed/leak
into another unit test.The end result is there is need to worry about these basics,
as they are taken care of for you automatically as long as the library is installed
as a dependency.LicensingThis library is licensed under the "The Unlicense" License. See the LICENSE file.