Package: convection-secrets-manager
Convection Secrets ManagerConvection Secrets ManagerAboutHow does it work?UsageCommands and APIConfigurationAutomated SetupQuestions requiring RootNotesManual SetupManual Setup StepsManual InstallationConfigure the ServerNotesInitializationConfigure the ClientConfiguration FileAuthFile FileClient ConfigurationAuthFile ConfigurationOperational NotesKey RotationRoot Key / Unlock Key Rotationrotate_root_keysKeySet Rotationrotate_keysetUser / Group ManagementInternalsUnlock KeysRotationRevocationLocking / UnlockingThe KeyDBRoot KeyKeyMapDBAuthDBKeySetsAccess Key PairsSecret StoragePluginsPlugin TypesCommunicationRequest ObjectsResponse ObjectLocal SocketNetworkingNetwork ACLs (NACL)Minimal NACLsWebSocketGroupsUsersAccess TokensUser / Group ACLsACL Type - Generic (ACLObject)ACL Type - Command (ACLCommand)ACL Type - Store (ACLStore)Extra NotesAboutConvection Secrets Manager is a Secrets Storage Server and Client, capable of storing many different types of data. This is part of the Convection Suite of tools.This package provides the Server and Server Console only.Client can be found hereHow does it work?Convection Secrets Manager usesFernetviacryptography.Fernetin order to encrypt/decrypt data. Upon initialization, a set ofUnlock Keysare generated, along with aKeyDBfor each key. At startup, theKeyDB(s)arelocked, and one of theUnlock Keysmust be sent in order for the Secrets Manager in order to function. Onceunlocked, a user can obtain a (time limited) token in order to access encrypted data and manage the server.Usersauthorize themselves with anAccess Key Pairin order to obtain a token. Communication between the Server and Client are done via Secure Sockets using TLS 1.3+. This configured on the local unix socket as well. If used with networking enabled,Network ACLsare also used to limit access. Secrets are stored in (one or) many files, with the contents of each file (when decrypted) being dependent on the type of data being stored.UsageThe Convection Secrets Manager client provides both a CLI as well as a Semi-API library. The server provides a secureAF_UNIXsocket for local communication and, optionally a secureAF_INETsocket for remote communication as well as a secureWebSocketfor WebUI operationsCommands and APIConvection Secrets Manager comes with console (CLI) access, as well as a Semi-API library.Information about CLI Commands or the API / Client Library can be foundhereConfigurationSeeCONFIG.mdAutomated SetupConvection Secrets Manager comes with a setup utility in order to quickly get running. This can be accessed usingconvection-secrets-setup. It will very likely need to be run as root, depending on the answers to some of the questions.In order to save time, an Answer File can be created, which can be passed to the setup command via the--dataargument. Any questions not answered in the answer file will be asked before proceeding. If you wish to use the default (the value in[]brackets), simply press enter to continue to the next question. To additionally help with creating an Answer File in the future, the Answers provided are printed to the console so that they can be saved as an answer file for later.Answer Files can be JSON, YAML or TOML. The type will automatically be picked up from the file extension, or you can use the--input-typeargument to force specify a type.Questions requiring RootDepending on the answer to these questions, root will be required in order to perform the neccessary action(s)Create Server User/GroupPID DirServer Log File PathServer Configuration DirectorySecrets DirectorySocket PathGenerate TLS CertificateListen Port,if less than 1024; default is 9670WebSocket Port,if less than 1024; default is 9671Install as ServiceNotesThis process does not create an AuthFile, you will need to do that by hand, if you wish to use one.NACLs are not created as part of the setup process. You will need to configure these yourselfBecause the chance for most users to use /var/run (or some other standardized run directory), the installer does not modify permissions of the PID Dir. The service may be unable to start as a result. Ensure that the user running Convection Secrets Manager can create files in the PID Dir, Socket Path, and Log File Path.Manual SetupManual Setup StepsInstall Convection Secrets ManagerConfigure the ServerCreate an RSA Public/Private KeypairInitialize the Server(Optional) Create an AuthFile(Optional) Create a Recovery UserSetup GroupsSetup ACLsSetup UsersCreate Client ConfigurationCreate SecretsManage your secrets!Manual InstallationDepending on installation choice, additional steps might have to be completed manually.(Optional, Recommended) Create a user such asconvection_smDo not use/home/for the users home directoryDo not use$SECRET_STOREfor the users home directoryDo not allow this user to have shell access (set shell to/sbin/nologin)Create a directory to store the server config files in one of these directories:/etc/convection-secrets/$HOME_OF_USER_RUNNING_CONVECTION_SM/.local/$CONFIG_PATH/convection-secrets/(SeeConfigure the Server)Directory should be:Owner: root (should be$USERif Step 1 was performed)Group: rootPermissions:4700(drws------)Additionally Create$SECRETS_CONFIG/certs/(Same Permissions as above)Create a directory where the secrets will be stored (Referenced below as$SECRET_STORE)Owner: root (should be$USERif Step 1 was performed)Group: rootPermissions:4700(drws------)(Optional, Recommended) Create a python virtual env, and install modules fromrequirements.txtinto it(Optional, Recommended) Copyconvection.secrets/convection-secrets.servicetolib/systemd/system/Ensure Paths forExecStartandPIDFileare correct.PIDFileshould match{pid_dir}/convection-secrets.pidIf you used a virtual env from step 4, yourExecStartshould look like$PATH_TO_VENV/bin/convection-sm-server -v -c $SECRETS_CONFIG service-startRunsudo systemctl enable convection-secrets.serviceIf you do not use the service file, you can stop the service byCtrl+C,kill -HUPorkill -TERMConfigure the ServerSeeCONFIG.mdfor configuration options.NotesIf Networking is enabled (service.use_network = true) you should be sure to configureNetwork ACLs(service.network.acl)InitializationBefore Convection Secrets Manager can be used, it must be initialized. This involves creating the Root Key, Unlock Keys, and First User.To Initialize:Start Convection Secrets Manager (systemd or by hand, if you didnt install the service config)Create a Public/Private Key Pair (same as the one from Step 3 of Setup, if you already did that).Run the following command:convection-secrets-manager initialize --num-keys %KeyCount% --public-key /path/to/step1/public.keyInitialization will generate some data that will not be shown again. Be sure to save it somewhere secure.Unlock KeysAccess Key ID forrootUser.Configure the ClientIf you do not wish to specify paths for the configuration file and AuthFile, you should use one of the paths mentioned below. Otherwise, you will need to use--configand--auth-fileoptions on the commandline.AuthFiles are not created by default. You need to create one in order for the AuthToken to be written to itConfiguration File~/.local/convection-secrets.client.[yaml,toml,json]/etc/convection-secrets.client.[yaml,toml,json]./convection-secrets.client.[yaml,toml,json]AuthFile File~/.local/convection-secrets.auth.[yaml,toml,json]/etc/convection-secrets.auth.[yaml,toml,json]./convection-secrets.auth.[yaml,toml,json]Client ConfigurationThe Client relies a configuration file to specify how to connect to the Server. A Minimal Configuration in TOML is below:[global.secrets.client]use_network=falsetls_ca="/etc/convection-secrets/certs/ca.pem"socket_path="/var/run/convection.secrets.sock"AuthFile ConfigurationAn AuthFile is an optional file you can use after initialization to provide authorization information without having to put it on the commandline. Not all entries are required.private_key="~/.local/convection-secrets.auth.key"access_key_id="<your user access key ID>"key_password="MyPrivateK3yP4ssw0rd"# Optional, only needed if Public/Private Pair is password protected.Anauthorizecall must still be called on the commandline. When an AuthFile is in use, instead of showing the AuthToken to stdout, the token is writen to the AuthFile. Once this is done, further commands can be executed without having to specify either option.Operational NotesKey RotationRoot Key / Unlock Key Rotationrotate_root_keysDuring this operation, new Unlock Keys are generated, a new Root Key is generated and encrypted, and then all KeySets and the KeyMapDB are re-encrypted. This will provide a new list of Unlock Keys to the user.Once everything has been rotated, the system is Locked, and one of the new Unlock Keys must be passed to the system for further operation.If this operation Fails, all files are reverted using their backup, the original Root Key and Unlock Keys will still be usable.KeySet Rotationrotate_keysetKeySet rotation involves generating new Keys for the KeySet, followed by re-encrypting every Secret Store associated with the KeySet.Because this operation relies on the underlying Secrets Store Plugin, it is possible that a rotation operation may fail. If the operation encounters an error during the rotation of a Secrets Store, then all other rotations are canceled, successful rotations are then attempted to be reverted to use the original KeySet Keys.DANGERDANGERDANGERDANGERDANGERDANGERDANGERDANGERIf a reversion fails, Secrets Stores associated with this KeySet may be in danger of being unusable. The Server will be shutdown to prevent further modifications until it can be examined by an admin. The new KeySet Keys will be stored as the KeySet NameROTATED_{keyset_name}, while the original KeySet Keys will remain in{keyset_name}. Any Secrets Stores associated with this KeySet should be encrypted using one of these KeySets. You can manually rename the KeySet between theROTATEDand original version in order to save your secrets. If any problem arises from such reversion or rotation failure, the problem should be brought to the Secrets Store Plugin Developer.User / Group ManagementTherootandrecoveryUser Names are 'protected'. They cannot be deleted entirely; at least one Access Key ID must exist once the user exists. (Note that therecoveryUser does not exist at Initialization, and must be created, if you wish).InternalsUnlock KeysAn Unlock Key is a key used to gain access to theKeyDB. When a server is initialized, a number of these keys are created, based on the input arg--num-keys(default 5) these keys are provided to the user once, and will never be displayed again. These Keys should be kept in a safe place, and should never be seen by untrusted eyes. Choose one of the provided unlock keys for continued use, then store these keys securely elsewhere.WARNING: Unlock Keys are the 'keys to the kingdom'. Loss of all unlock keys will result in permanent data loss. An attacker gaining access to Unlock Keys can enable them full access to all of your stored secretsWARNING: BECAUSE AN UNLOCK KEY MUST BE PROVIDED AFTER STARTUP, THIS UNLOCK KEY MAY BE SEEN IN BASH HISTORY, ETC. BE SURE THAT YOU CLEAR HISTORY, OR USE A MEANS OF NOT LOGGING THE UNLOCK KEY, SUCH AS A SCRIPT OR AS AN ENV VARIABLE, ETCKey Rotationshould occur occasionally for security.RotationUnlock Keys should be periodically rotated to ensure that if they are compromised, they do not remain in use for long. Rotation of Unlock Keys will create new Unlock Keys and KeyDBs from the currently unlocked KeyDB. The new Unlock Keys will be provided to the user once.RevocationUnlock Key Revocation can be performed by rotating Unlock Keys. Because determining which Root Key entry is the affected one would require a decryption attempt using that key, it is recommended to just performKey Rotationinstead.Locking / UnlockingLocking and Unlocking is a part of the process to gaining access to the Secrets Manager. When the server starts up, all secrets andKeySetsare encrypted. AnUnlock Keymust be sent using theunlockcommand. The Server will attempt to unlock theKeyDBusing the provided key. If successful, operation can continue.The KeyDBThe KeyDB is comprised of many pieces. Each piece is described in detail below. At Initialization a Root Key, KeyMapDB, and AuthDB are created. These 3 pieces comprise the core of Encryption/Decryption and Authorization.WARNING: The components of the KeyDB should be treated as sensitive, and should be securely stored, and backed up. Loss of these items in part, or in whole, will result in data lossRoot KeyRoot Keys decrypt the KeyMapDB, AuthDB and KeySets. This key is encrypted multiple times based on the configuration flag--num-keysat initialization. Each encryption is generated using a different key (thus, the Unlock Keys). The encrypted results are stored in$storage_path/root.key. During Unlock, an Unlock Key is passed to the Server, and the Server attempts to decrypt each key located in the Root Keys file. On success, the Root Key is loaded, for future use.WARNING: The Root Key is the 'Key to the Kingdom', gaining access to the decrypted Root Key can allow perpetrators to decrypt and manipulate data.WARNING: The Root Key is a critical component. Loss of the Root Key will result in complete data lossKeyMapDBThe KeyMapDB is the mapping of which KeySets are attached to which Secrets Storage. This file is kept encrypted, and is encrypted by the Root Key. The KeyMapDB is located at$storage_path/keymap.dbWhile this file is critical, loss of this file will not necessarily result in data loss. You would need to rebuild the KeyMapDB by creating new KeySets and Stores with the previous names, replacing the file contents with the original KeySet and Store.AuthDBThe AuthDB is the Secret Store for Users, Groups and ACLs. This contains the mapping of Users and their Access Key Pairs, as well as active Auth Tokens. The AuthDB is stored at$storage_path/auth.db. A Separate KeySet is generated for the AuthDB, calledauthdbKeySetsThe KeyDB also contains the keys for eachSecret Store. These Keys are used to encrypt/decrypt the content of theSecret Store. Similarly, to theRoot Keys, a number of Keys are used for eachSecret Store, meaning that each time a write operation is performed on an item within the Secret Store, there is no guarantee that the same key will be used that was used the first time. These Keys are stored by name, which can be used for multiple Secret Stores. These keys are encrypted by the Root Key and stored at$storage_path/$keyset_name.keyAccess Key PairsEachUsercan generate Access Key Pairs for themselves (and others if authorized). an Access Key Pair contains 3 parts:Access Key ID: This is the ID used to determine which Public Key to to useAccess Private Key: This is the Key the user keepsAccess Public Key: This is the Key the Server gets when a user is createdThe Public and Private Key are generated by the User. During user creation, the Public Key is sent to the Server. The Server then generates an Access Key Id for the User and Public Key. The user will need the Private Key and Access Key Id for authorization. These should be kept secure as they allow access to whatever Secrets the User has access to. The Public Key is sent to, and stored on the server. The Private Key isneversent to the server.Secret StorageNot all Secrets are created equally. Secrets Storage provides a means to store many types of data using the same general API. Secrets Storage is made up ofPlugins, each plugin defines how data is stored, allowing for an extensible way of securely storing anything.The root of a Secret Store contains a Metadata block, allowing for Plugin versioning, and additionally contains information like Storage Type, Creation Date, Creator, and Storage Name.PluginsA Secret Store is a way of securely storing a piece of data. Each Plugin defines how that data is stored. Some may define a single file that holds multiple secrets, another may define a directory with many individually encrypted files.Plugin TypesGeneric (Generic_Secret): Used for General Key/Value storage, can be used to store any text or binary data.RSA (RSA_Secret): Used for RSA Keypair generation and storage. Can ingest existing RSA Keys and generate new ones.Cert (X509_Secret): Used for x509 Certificate generation and storage. Can ingest existing certificates and generate new ones.User (User_Secret): A more targeted Key/Value storage, for storing Usernames and PasswordsCommunicationCommunication is done via SecureLocal Socketusing TLS 1.3+ and optionally a SecureNetwork Socket(also TLS 1.3+), as well as an optionalWeb Socket(also TLS1.3+).Each Inbound Request to the Server is in the format of:<command> <request_data>, whererequest_datais either empty or a JSON blob. Server Responses from the Server will be in a JSON blob. Data required for Request Data will depend on the incoming command. Response Data will always have the same format, and a minimal response.Request ObjectsSeeAPI.mdResponse Object{__any_item__:....,// Command Response Data"result":{// Result State"state":<bool>,// Success / Failure Flag"messages":<list[str]>// Messages from the command, etc.}}Local SocketThe Local Socket is anAF_UNIXsocket with TLS 1.3+. The socket is setup as 0660. User/Group of the socket are controlled byservice.socket_ownerandservice.socket_group.NetworkingSecrets can be accessed remotely by enablingservice.use_networkand then configuring theservice.networkblock. By Default the network socket will listen on127.0.0.1:9670Network ACLs (NACL)For Network Enabled configurations, Convection Secrets Manager comes with a network-level ACL system, to be self contained. This is used to allow or deny the commands an IP address may execute. This is merely one level of access control. SeeUsersfor the other part of access control.NACLs are configured in theServer Configurationunderservice.network.aclNACLs consist of 3 parts:Type: Whether the ACL is for Allow or Deny operationsIP Address: IP Address or Subnet (in cidr)Commands: List of Commands applicable to the NACLMinimal NACLsA set of NACLs are hardcoded into the Server to ensure functionality.{"type":"allow","ip_address":["127.0.0.1"],"commands":["initialize","unlock","rotate","authorize","deauth"]},{"type":"deny","ip_address":["0.0.0.0/0"],"commands":["service-stop","stop","service-start","start"]}WebSocketA WebSocket can be enabled for the WebUI by enablingservice.use_websocketand then configuring theservice.websocketblock. By Default the websocket will listen on127.0.0.1:9671NACLs from theservice.networkblock are used for the WebSocket. The WebSocket does not support using its own NACLs.GroupsGroups are simple containers that group together ACLs and Users. ACLs and Users are attached to a Group, which allows for easier management / application of ACLs.UsersMany of the CLI and API commands require authorization in order for the command to be accessible. This authorization is done with the use of anAccess Key Pair, which provides anAccess Token, which is used for subsequent commands. A User can generate multiple pairs, but these should be limited, to prevent one being lost to the ether and later compromised. Users are granted access perSecret Store, and optionally per secret.User Accessis managed via ACLsAccess TokensAfter a User performs anauthorizecommand, an Access Token is returned. This token is time limited (configurable). Once a user has the token, they can then perform other commands, while passing the recieved token along with their Access Key ID. Access Token expirations are controlled on two levels, at the service configuration level viasecrets.manager.authentication.token_max_expire, and at token creation time. Expiration values at the Token level cannot exceed thetoken_max_expirevalue.User / Group ACLsUser and Group Level ACLs exist to restrict operations within Convection Secrets Manager. Each ACL type provides a different method for control. Commands perform an ACL check before performing their operation.ACL Type - Generic (ACLObject)This is the basis for other ACL Types, and its arguments are required for other ACL Types.Arguments:name: ACL Name, must be uniquemode: Allow / Deny mode.In Console mode use"allow"and"deny"In API / Server useACLObject.MODE_(ALLOW|DENY)access_mode: Specifies what types of access are allowed/disallowed, such as Read/Write/Modify/DeleteIn Console mode use shorthand flags combined as needed:r(Read),w(Write),m(Modify),d(Delete)In API / Server useACLObject.ACCESS_(READ|WRITE|MODIFY|DELETE|NONE), these can be AND'd or OR'd together to form various combinationsACL Type - Command (ACLCommand)This is a Command only limiter, allowing or preventing access to specific commands (such ascreate_acl, etc)Arguments:commands: A list of regex patterns (as strings) to match againstACL Type - Store (ACLStore)This is a Secrets / Store only limiter, allowing or preventing access to specific Secrets Stores or Secrets.Arguments:store_paths: A list of regex patterns (as strings) to check Secrets Storage path againstsecret_names: A list of regex patterns (as strings) to check Secret Names againstExtra NotesShutting Down the server does not involve calling theconvection-secrets-managercommand, instead it is terminated viakill. The table below indicates which signals will shutdown or restartSignalPurposeLinuxWindowsCTRL_C_EVENTShutdown[ ][x]SIGINTShutdown[x][x]SIGABRTShutdown[x][ ]SIGILLShutdown[x][ ]SIGTERMShutdown[x][ ]SIGHUPRestart[x][ ]In the event that the service gets stuck trying to start or stop, you can kill the service with akill -9 <pid>. All Signals above will cause a graceful shutdown attempt.