Package: discreetly
discreetlyA secrets manager for AWS and GCP$discreetlysetacme/github/api-tokenmy-secret-api-key$discreetlygetacme/github/api-tokenmy-secret-api-key$discreetly--profileprodlistacme/['acme/prod/postgres/passwd', 'acme/prod/github/api-token']Applications often require access to secrets such as database passwords or API keys. A really bad way to manage those secrets would be to bake them into your code and then check them into source control. Marginally better would be storing secrets in environment variables. If your applications are running on AWS or GCP, both platforms provide tooling for better secrets management.discreetlyprovides a consistent API for managing secrets across both AWS and GCP. It encourages you to partition your secrets and encryption keys so that your web application only has access to the secrets it needs to run and not, for example, access to infrastructure secrets only needed by your infrastructure management tools.Installationdiscreetlydoes not set up any infrastructure for you. At a minimum, you need to have i) credentials available for authenticating with AWS and/or Google, ii) permissions to use KMS to encrypt and decrypt secrets, and iii) permissions to read/write to the underlying store.To install thediscreetlycli with support for AWS and GCP:$pipinstalldiscreetly[cli]For use as a library, installdiscreetlyspecifying which backends you need to support, e.g.:$pipinstalldiscreetly[aws,gcp]Configurationdiscreetlycan be configured with a JSON file, e.g.:{"acme":{"type":"aws"},"default":{"type":"gcp","datastore_project":"acme-corp","keyid":"projects/acme-corp-kms/locations/global/keyRings/discreetly/cryptoKeys/default"}}In the above example, the configuration contains two profiles, "acme" and "default". A profile must specify atype, either "aws" or "gcp".Because a profile can specify akeyid, you can have named profiles not only for different cloud providers but for different KMS keys, e.g. one for dev, another for prod.discreetlywill search for a configuration file at the location provided by the environment variableDISCREETLY_CONFIG_FILE, falling back to a file nameddiscreetly.jsonin the current directory.Frequently Asked Questions (FAQ)What about credstash/Chamber/Vault/etc.?discreetlywas actually inspired by credstash. However, credstash only supports AWS and it pre-dates Parameter Store. That said, there are use cases where a DynamoDB backend might be preferred (e.g. parameters greater than 4096 characters).Chamber is excellent choice, especially in a Go environment or if you only need AWS support.Vault is great if you have the resources to support it.TroubleshootingMost issues are related to authentication or permissions with your cloud provider.With thediscreetlycli, you can also change the logging level by setting theLOGLEVELenvironment variable, e.g.$LOGLEVEL=DEBUGdiscreetlygetmy/super/secretAWSIf you have the AWS CLI installed, verify that you can use it to access Parameter Store, e.g.:$awsssmget-parameter/path/to/parameter--with-decryptionIf that works, then the problem may very well be withdiscreetlyand you should consider opening an issue.However, if that's not working, it's likely a configuration or authentication issue. Under the hood,discreetlyuses Boto 3, so consult their documentation onsetting up your authentication credentials.GCPUnfortunately,gclouddoesn't support much of the Datastore API, so it's a little harder to sanity test any issues in your local environment.Verify that you've followed the steps inObtaining and providing service account credentials manually, in particular:Creating a service accountSaving the JSON file for the service account locallySetting the path to the JSON file in the environment variable,GOOGLE_APPLICATION_CREDENTIALSLibrary usage$pipinstalldiscreetly[aws,gcp]importdiscreetlycfg={"default":{"type":"gcp","datastore_project":"acme-corp","keyid":"projects/acme-corp-kms/locations/global/keyRings/discreetly/cryptoKeys/default"},"acme-aws":{"type":"aws","keyid: "alias/discreetly_key",}}d=discreetly.Session.create(cfg)# uses the 'default' profiled.set('my-service/my-key','my_value')d.get('my-service/my_key')acme=discreetly.session.create(config=cfg,profile='acme-aws')# uses the 'acme-aws' profileacme.set('acme/dev/postgres/password','open-sesame')acme.get('acme/dev/postgres/password')acme.set('acme/prod/postgres/password','super-secret',keyid='alias/prod_key')acme.get('acme/prod/postgres/password')# no need to specify the keyid for getDevelopmentdiscreetlyis written in Python and supports Python 2.7 and 3.6+After cloning the repo, runpip install -r requirements.txtto install the required development dependencies (e.g. pytest, flake8, etc.)You can runpytestto run the unit tests locally ortoxto run all the tests under Python 2.7 and 3.x.Notefrom version0.2.0onwards,discreetlyonly supports Python 3.6+ReleasesReview and updateCHANGELOG.mdin a new release branchThe following will show commits since the last tag beginning with 'v':$gitlog$(gitdescribe--tags--match"v*"--abbrev=0)..HEAD`Replace[Unreleased]with the newsemverand release date, e.g.[0.1.2] - 2019-12-13Add a new[Unreleased]placeholder above the new version you just createdCreate a new PRThe following lists all commits since the last tag, which may be useful for including in the PR description:git log --pretty=format:"* %h %s" $(git describe --tags --abbrev=0 @^)..@Once the PR is merged to master and passes all status checks, create and push a new signed, annotated tag, e.g.:$LAST_VERSION=$(gitdescribe--tags--match"v*"--abbrev=0)$VERSION=v0.1.1$awk"/^## \[${VERSION}/{flag=1;}/## \[${LAST_VERSION}/{flag=0} flag"CHANGELOG.md|gittag-s-a${VERSION}--cleanup=whitespace-F-$gitpush--follow-tagsTheawkcommand above grabs the contents of the CHANGELOG for$VERSIONto include as the message for the annotated git tag. It's fairly brittle in its assumptions about the format of the CHANGELOG. so you might just run the awk command:awk "/^## \[${VERSION}/{flag=1;}/## \[${LAST_VERSION}/{flag=0} flag" CHANGELOG.mdas a sanity check before creating & pushing the tag.