Package: secret-transfer
Secret TransferYaML-based secret manager for secret load/transfer from different sources to destinations.Base ideaThe base idea is to have a simple and flexible way to transfer secrets from different sources to different destinations.The main components are:Source- A place where secrets are stored. It can be a file, environment variables, a database, a cloud service, etc.Destination- A place where secrets are transferred. It can be environment variables, a file, a cloud service, etc.Collection- A collection of secrets from sources. It can be used to combine secrets from different sources.Transfer- A transfer of secrets from a collection to a destination.FeaturesYaML-based- Define sources, destinations, collections, and transfers in a YaML file.Lazy loading- Define objects in no particular order and reference them from one another.Cross-referencing- Reference sources, destinations, collections, and transfers from one another.Built-in classes- Use built-in classes for sources, destinations, collections, and transfers.Extensible- Add custom classes for sources, destinations, collections, and transfers.UsageCLI CommandsRunRun all transfers in the given YaML file.secret-transferrun-f<path_to_yaml_file>Options:-f, --file <path_to_yaml_file>- Path to the YaML file with secrets. [required]-n, --name <name>- Name of the transfer to run. [optional]CleanClean all secrets in all transfer destinations in the given YaML file.secret-transferclean-f<path_to_yaml_file>Options:-f, --file <path_to_yaml_file>- Path to the YaML file with secrets. [required]-n, --name <name>- Name of the transfer to clean. [optional]YaML schemaThe YaML file should contain the following optional sections:Custom class definitions for sources, destinations, collections, and transfers.source_classes:<name>:# Name to be registeredmodule:<module># Module to import fromclass_name:<class_name># Class to be registereddestination_classes:...collection_classes:...transfer_classes:...Sources, destinations, collections, and transfers.sources:<name>:class_name:<source_class_name># Class to use, either from source_classes or built-ininit_args:# Arguments to pass to the class constructor, can be omitted if no arguments are needed<arg_name>:<arg_value>...destinations:...collections:...transfers:...Lasy loadingAll items are lazily loaded, so they are not created until they are used. This allows to define objects in no particular order and reference them from one another. The only exception is built-in classes, which are registered at import-time.Cross-referencing is allowed, so you can reference sources, destinations, collections, transfers and their values from init_args. Circular references are not allowed.Cross-referencingInstance referencesTo reference instances of sources, destinations, collections, and transfers, use the following syntax:$<type>[<name>]Where:<type>- Type of the instance (source, destination, collection, or transfer).<name>- Name of the instance.Value referencesTo reference values from sources or collections, use the following syntax:$<type>[<name>][<key>]Where:<type>- Type of the gettable instance (usually, but not necessarily source or collection).<name>- Name of the instance.<key>- Key of the value.Where:Built-in classesSourcesDotEnvSourceLoad secrets from a .env file usingdotenvpackage.
Arguments:file_path- Path to the .env file. [required]Example:sources:dotenv:class_name:DotEnvSourceinit_args:file_path:.envEnvSourceLoad secrets from environment variables.
Arguments: none.Already registered with nameenv, no need to define in the YaML file.Example:collections:default:init_args:TEST_KEY:source:$sources[env]PresetSourceLoad secrets from a preset dictionary.
Arguments: key-value pairs of variables.Example:sources:preset:class_name:PresetSourceinit_args:TEST_KEY:test_valueUserInputSourceAsks the user for input.
Arguments: none.Already registered with nameuser_input, no need to define in the YaML file.Example:collections:default:init_args:TEST_KEY:source:$sources[user_input]VaultCLIKVSourceLoad secrets from HashiCorp Vault KV usingvault kv getCLI command.
Arguments:address- Address of the Vault server. [required]mount- Mount point of the KV engine. [required]secret_name- Name(path) of the secret. [required]Pre-requisites:vaultCLI installed and authenticated.Example:sources:vault:class_name:VaultCLIKVSourceinit_args:address:https://vault.example.commount:secretssecret_name:TEST_SECRETYCCLILockboxSourceLoad secrets from Yandex.Cloud Lockbox usingyc lockbox payload getCLI command.
Arguments:profile- Name of the Yandex.Cloud CLI profile. [required]folder- Folder name. [required]lockbox- Lockbox name. [required]Pre-requisites:ycCLI installed and authenticated.Example:sources:yc_lockbox:class_name:YCCLILockboxSourceinit_args:profile:my-profilefolder:my-folderlockbox:my-lockboxDestinationsBashExportDestinationPrint secrets asexportcommands to the console. Useful for setting environment variables. Never let stdout to be captured by a process, as it will expose the secrets.
Arguments: none.Already registered with namebash_export, no need to define in the YaML file.Example:transfers:default:init_args:source:...destination:$destinations[bash_export]EnvDestinationSet secrets as environment variables.
Arguments: none.Already registered with nameenv, no need to define in the YaML file.Example:transfers:default:init_args:source:...destination:$destinations[env]GithubCliSecretsDestinationSet secrets as GitHub repository secrets usinggh secret setCLI command.
Arguments:repo_name- Name of the repository. [required]owner_name- Name of the repository owner. [required]base_url- Base URL of the GitHub API. [optional] (default:https://github.com)Pre-requisites:ghCLI installed and authenticated.Example:destinations:github:class_name:GithubCliSecretsDestinationinit_args:repo_name:my-repoowner_name:my-orgCollectionsDefaultCollectionDefault collection to combine secrets from sources. Default collection class, so class_name can be omitted.Example:collections:default:init_args:COLLECTION_KEY:source:$sources[env]key:SOURCE_KEYTransfersDefaultTransferDefault transfer to transfer secrets from collection to destination. Default transfer class, so class_name can be omitted.Example:transfers:default:init_args:collection:$collections[default]destination:$destinations[env]Custom class definitionsYou can define custom classes for sources, destinations, collections, and transfers.Any resourceimporttypingimporttyping_extensionsimportsecret_transfer.coreassecret_transfer_coreclassCustomResource(...):__register__=...# bool: Should the class be registered on creation (optional, default: True)__default__=...# bool: Should the class be used as default for its type (optional, default: False)@classmethoddefparse_init_arguments(cls,**arguments:secret_transfer_core.InitArgumentType)->typing.Mapping[str,typing.Any]:# Optional to implement# Parse and validate init arguments passed from the YaML definition for the future instance creation...@classmethoddefget_default_instances(cls)->typing.Mapping[str,typing_extensions.Self]:# Optional to implement# Return default instances of the class to be registered at import-time...Sourceimportsecret_transfer.coreassecret_transfer_coreimportsecret_transfer.utils.typesassecret_transfer_typesclassCustomSource(secret_transfer_core.AbstractSource):def__getitem__(self,key:str)->secret_transfer_types.Literal:# Required to implement# Return the value of the secret by key...Destinationimportsecret_transfer.coreassecret_transfer_coreimportsecret_transfer.utils.typesassecret_transfer_typesclassCustomDestination(secret_transfer_core.AbstractDestination):def__setitem__(self,key:str,value:secret_transfer_types.Literal)->None:# Required to implement# Set the value of the secret by key...def__delitem__(self,key:str)->None:# Optional to implement# Clean all secrets in the destination...Collectionimporttypingimportsecret_transfer.coreassecret_transfer_coreimportsecret_transfer.utils.typesassecret_transfer_typesclassCustomCollection(secret_transfer_core.AbstractCollection):def__getitem__(self,key:str)->secret_transfer_types.Literal:# Required to implement# Return the value of the secret by key...def__iter__(self)->typing.Iterator[str]:# Required to implement# Return an iterator over the keys of the collection...defitems(self)->typing.Iterator[tuple[str,secret_transfer_types.Literal]]:# Required to implement# Return an iterator over the items of the collection...Transferimportsecret_transfer.coreassecret_transfer_coreclassCustomTransfer(secret_transfer_core.AbstractTransfer):defrun(self)->None:# Required to implement# Transfer secrets...defclean(self)->None:# Required to implement# Clean all secrets in the transfer...Usage ExamplesCheckexamplesfor usage examples.DevelopmentTaskfile commandsFor all commands seeTaskfileortask --list-all.