Package: secenv
SecEnvsecenvis an utility tool to list, read and fill secrets from multiple stores.
It also defines contexts, and generates the associated environment values.The end-user documentation is availableon the official website.
This document is indented to cover the technical documentation for current and future developers.SecEnvContributorsCode of conductRepository structuremakecommandsRunningTestingGit flowDevelopmentAdding a storeCaching retrieved secretsDefault config valueKey-value store inside a secretAdding an output format for contextsContributorsBecause an open-source project can't live without a community, we would like to thank our dear contributors.
Thank you so much:Valentin Merlo, @valentin137Code of conductThis project uses Python 3.7+.
It is indented using Black.The philosophy ofsecenvis to keep it simple.
It means we don't want to addfancyfunctionalities that require a lot of maintenance, but instead we want to focus on useful and reliable features.The codebase aims to be easy to read, easy to understand, easy to debug, easy to improve, easy to maintain.When modifications are made to the codebase, please wonder if the statement above is still respected.Repository structureThesecenvcodebase is designed around the different functionalities.
Here is the global structure of the repository:./
│   > Docs are handled by Docusaurus
├── docs/ -> .docusaurus/docs/
├── .docusaurus/
│
│   > General files
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── README.md
│
│   > Sources
├── pyproject.toml
├── secenv/
├── setup.py
└── tests/The./secenvdirectory contains the actual code, and is arranged as this:./secenv/
│   > Entrypoint
├── __init__.py
│
│   > Context generation and output
├── context.py
├── contexts/
│
│   > Stores definition
├── stores/
│
│   > Utils functions
└── utils.pymakecommandsThe project is managed using a Makefile.
To list the available commands, run:$ make help
Usage: make <target>

Installation
  install-local    Install secenv as a local package
  install-test     Install secenv from TestPyPI
  install          Install secenv from PyPI
  uninstall        Uninstall secenv

Package
  dist             Generate wheel package for PyPI
  upload-test      Upload the package to TestPyPI
  upload           Upload the package to PyPI

Documentation
  docs-preview     Generate a live preview of the docs
  docs-bump        Create a new version of the documentation
  docs-publish     Publish the docs

Release
  changelog        Update the changelog with the changes since the last tag
  bump-version     Compute the new version and update files accordingly
  git-publish      Create a new tag/commit and push it

Utils
  deps             Install test & build dependencies
  tests            Run the unit tests
  clean            Remove build and dist files
  docker-node      Run a Node:latest image with '-v $pwd:/app -p 3000'
  docker-python    Run a Python:$VERSION image with '-v $pwd:/app'RunningTo install the required dependencies for development, run:# Install depsmakedeps# Install local packagemakeinstall-localThe program entrypoint is located insecenv.__init__:main.TestingPyTest handles the different tests.
To perform all the tests, run:make testsBefore submitting changes tosecenv, please ensure all the tests pass.Git flowThe development flow ofsecenvis designed around branches.
Each branch should follow this nomenclature:Bug fixes:fix/<bug-to-fix>Features:feature/<feature-to-add>Chore (docs, unit tests, refactoring...):chore/<thing-to-modify>Tests:test/<thing-to-test>Thetest/.*branches are not meant to be merged.The pull requests must be named after the branch.
In example, if the branch isfeature/add-useful-thing, name the PR:Feature: add useful thing.
The names of the PRs are used to determine the version bump, so name them carefully.If the modified code causes breaking changes, the commit messageMUSTcontain a line like this:This is the commit message.

This is the commit body...

BREAKING CHANGE:
Explain what breaks, how, why.
Explain how to migrate to the new behavior and/or new configurationThe parser will accept anything matching '^BREAK' in the commit message.Everything starting from the line containingBREAKING CHANGEwill be included in the Changelog.DevelopmentAdding a storeThe stores are defined in thesecenv/storesdirectory. They implement theStoreInterfacedefined in__init__.py.The stores must implement all of the methods defined in the interface, otherwise the program will crash at some point.It is a good practice to have a look at the other stores to understand how to deal with the parser, the arguments, and how to deal with the secrets themselves.First, read the code of theenvstore to understand how to implement the different methods.
Then, read the code of thevaultstore to understand secrets' filtering using a key, and how to use arguments.
Thepassstore uses default arguments, read its code to understand how it works.To summarize, here is the link between the constructor of a store, and its definition in the configuration file:# ./secenv/stores/mystore.pyclassStore(StoreInterface):def__init__(self,name,infos):self.name=nameself.url=super().get_from_config(name,"url",infos,default="https://vault.example.com")self.token=super().get_from_config(name,"token",infos)...stores:my_store_in_config:type:mystoretoken:S3CR3T_T0K3Nurl:https://vault.custom.domain.tld# `url` defaults to 'https://vault.example.com' if it's not providedIt is also possible to configure the parameters on the new store by usingSECENV_<store_name>_<parameter>.
This behavior doesn't require more configuration than defining the store.Caching retrieved secretsA cache mechanism is available when retrieving the secrets.
It is enabled by adding the@cacheddecorator available in thestoresmodule to theStore.read_secretfunction.In example:from.importStoreInterface,cachedclassStore(StoreInterface):@cacheddefread_secret(self,secret)->str:...When a secret is retrieved, its result is stored in a map so when it is retrieved again, the request is not performed and the value is read from the map instead.Default config valueTheStoreInterface.get_from_configfunction permits to read a value from the config file, or from the environment directly.
This function takes an optional argument,default.If the wanted value is in neither the config file, nor the environment, then thedefaultvalue is returned if one is provided.
If there is nodefaultvalue, then an exception is raised.Key-value store inside a secretIf the secret can be a key-value store, a--keyoption can be added to the parser (see AWS and Vault stores as they implement this).
The new store must provide aStore.filter(secret, key)function.Adding an output format for contextsThe output is computed in the./secenv/context.pyfile.
A list of the available formats is defined at the beginning of the file.To make a new format available, add anifstatement in theformat_outputfunction.
The secrets are stored in thecontextparameter.Here is the format of this parameter:context:Dict[str,Dict[str,str]]context={"my_secret"={"value"="my_value""sensitive"=True}}Thecontext["my_secret"]["sensitive"]option can be used to mask the output value.
It is used in thegithub_actionsoutput format.Theescapefunction is available to escape the quotes and dollar signs.During the formatting, the values must be appended to theoutputarray, with one value per line.
At the end of the function, the output is modified so all the elements are joined using a new line.Here is a full example of an output format:defformat_output(context,format):output=[]...ifformat=="debug":fork,vincontext.items():output.append(f"key ={k}, value (escaped) ={escape(v['value'])}")...