Package: python-secrets
Python command line app for managing groups of secrets (passwords, API keys, etc) and
other project variables. Reduces security risks from things like weak default passwords,
secrets stored in files in the source code repository directory.Version: 23.4.2Free software:Apache 2.0 LicenseDocumentation:https://python_secrets.readthedocs.org.FeaturesUses theopenstack/cliffcommand line framework for a robust and
full-featured CLI. It is easy to add new commands and features!Supports a “drop-in” model for defining variables in a modular manner
(something like thepython-update-dotdeeprogram), supporting simplified
bulk setting or generating values of variables as needed.Likepython-update-dotdee,psecproduces a single master.jsonfile to hold variables defined by the drop-in group
description files. That means you can use that file directly
to set variables to be used from within other programs like
Ansible (e.g.ansible-playbookplaybook.yml-e@"$(psecsecrets path)")Support multiple simultaneous sets of secrets (environments) for
flexibility and scalability in multi-environment deployments and to
support different use cases or different combinations of secrets.Supports changing the storage location of secrets and variables to
allow them to be stored on secure mobile media (such as self-encrypting
external SSD or Flash drives) or encrypted disk images mounted at
run-time to ensure the confidentiality of data at rest.List the groups of variables (and how many secrets in each group).Describe secrets by their variable name and type (e.g.,password,uuid4,random_base64). You can also include a descriptive
string to prompt the user for a value, a list of options to choose
from (or*for “any value the user enters”), and a list of
environment variables to export for other programs to use at
run time.Allows manual entry of values, setting non-secret variables from
a default value, or automatic generation of secrets according to
their type.Manually setstringvariables based on the output of simple
commands. This allows interfacing with external programs for
obtaining secrets, such asVault by Hashicorp.Generate unique values for variables, or use a single value per
type to simplify use of secrets in access control of services
while supporting a “break-glass” process to quickly regenerate
secrets when needed.Show the variables and their unredacted values (or redacted them
to maintain secrecy during demonstrations or in documentation).Export the variables (optionally with a specific prefix string)
to the environment and run a command that inherits them (e.g.,
to pass variables toterraformfor provisioning cloud
instances).Output the variables and values in multiple different formats (CSV,
JSON, YAML) for use in shell scripts, etc. usingclifffeatures.Send secrets to other users on demand using GPG encrypted email to
protect the secrets while in transit and while at rest in users’
email inboxes.Makes it easy to store temporary files (e.g., the output from
Jinja template rendering)
that may contain secretsoutsideof the source repo directory
in an environment-specifictmp/directory.NoteDue to the use of the Pythonsecretsmodule, which was introduced
in Python 3.6, only Python versions >= 3.6 can be used.LimitationsSecrets are stored inunencryptedform in the environments
directories.  Permissions are set to limit access, but this is not an
“encrypt data at rest” solution likeVault by Hashicorp.Does not handle secure distributed access for users on remote systems. You
must use something likeVault by Hashicorporlibfuse/sshfsfor secure
(realtime) distributed access.Does not handle secure distribution of newly generated secrets out
to distributed systems that need them. You will need to use a program
likeAnsibleand related playbooks for pushing out and changing
secrets (or for retrieving backups). Look at theD2 Ansible
playbooks(https://github.com/davedittrich/ansible-dims-playbooks)
for example playbooks for doing these tasks.Does not clean up the environment-specifictmp/directories.
(You need to handle that in code, but at least they are less likely
to end up in a Git commit.)Usage ConceptsThere is a separateUsagechapter with individual command documentation.
The remainder of this section covers higher level usage concepts necessary to
best use thepython_secretspackage in your open source software project.Directories and filesThere are three file system concepts that are important to understand
regarding secrets storage:The rootsecrets base directoryfor secrets storage;Theenvironmentfor organizing a set of secrets and
secret group descriptions;Thesecretsfile andgroup descriptions.Secrets Base Directorypsecexpects to store all of files in a directory tree known as asecrets base directory. Originally, this was intended to be located in the
current user’s home directory. Unless you over-ride the name of this directory,
it defaults to.secretson Linux andsecretson Windows.The ability to locate this directory in a different file system path is
supported by command line options and an environment variable so you can store
files on an exported file share, in a common location for use by a group on a
workstation, or to move the contents to an encrypted disk or a different
partition with more disk space.The first time you use ever usepsec, there will likely be no
directory:$tree~/.secrets/Users/dittrich/.secrets [error opening dir]

0 directories, 0 filesNoteThe secrets base directory may be created automatically for you the
first time you create an environment.  For more information, seepsec init--help.EnvironmentsEnvironments are sub-directories within the root secrets directory.  You can
just create the directory structure without any files. You create
one environment per set of unique secrets that you need to manage. This could
be one for open sourceProgram A, one forProgram B, etc., or it could be
one fordevelopment, one fortesting, one forproduction, etc. (or any
combination).The commandenvironments createcreates an environment.  Since this
program is designed to support multiple environments, a name for the new
environment is required.  The name of the environment can be provided
explicitly, or it can be inferred from the base name of the current working
directory:$pwd/Users/dittrich/git/python_secrets$psecenvironmentscreateenvironment directory /Users/dittrich/.secrets/python_secrets created$tree~/.secrets/Users/dittrich/.secrets
└── python_secrets
    └── secrets.d

2 directories, 0 filesLet’s say we want to create empty environments for the three deployments
(development,testing, andproduction). The names can be assigned
explicitly by (a) giving an argument on the command line, (b) using the-eor--environmentcommand line flag, or (c) by setting the environment variableD2_ENVIRONMENT:$psecenvironmentscreatedevelopmentenvironment directory /Users/dittrich/.secrets/development created$psec--environmenttestingenvironmentscreateenvironment directory /Users/dittrich/.secrets/testing created$D2_ENVIRONMENT=productionpsecenvironmentscreateenvironment directory /Users/dittrich/.secrets/production created$tree~/.secrets/Users/dittrich/.secrets
├── development
│   └── secrets.d
├── production
│   └── secrets.d
├── python_secrets
│   └── secrets.d
└── testing
    └── secrets.d

8 directories, 0 filesIf you want to create more than one environment at once, you will
have to specify all of the names on the command line as arguments:$psecenvironmentscreatedevelopmenttestingproductionenvironment directory /Users/dittrich/.secrets/development created
environment directory /Users/dittrich/.secrets/testing created
environment directory /Users/dittrich/.secrets/production createdIf you are using one source repository for building multiple deployments, of
course you can’t rely on the basename of the directory for all deployments. The
default environment can be set, shown, or unset, using theenvironments defaultcommand.$psecenvironmentsdefault--helpusage: psec environments default [-h] [--unset-default] [environment]

Manage default environment via file in cwd

positional arguments:
  environment

optional arguments:
  -h, --help       show this help message and exit
  --unset-default  Unset localized environment defaultIf no default is explicitly set, the default that would be
applied is returned:$cd~/git/python_secrets$psecenvironmentsdefaultdefault environment is "python_secrets"You can get a list of all available environments at any time,
including which one would be the default used by sub-commands:$psecenvironmentslist+-------------+---------+
| Environment | Default |
+-------------+---------+
| development | No      |
| testing     | No      |
| production  | No      |
+-------------+---------+The following shows setting and unsetting the default:$psecenvironmentsdefaulttestingdefault environment set to "testing"$psecenvironmentsdefaulttesting$psecenvironmentslist+-------------+---------+
| Environment | Default |
+-------------+---------+
| development | No      |
| testing     | Yes     |
| production  | No      |
+-------------+---------+$psecenvironmentsdefault--unset-defaultdefault environment unsetThe environment directories are useable for storingallsecrets and
sensitive files (e.g., backups of certificates, databases, etc.) associated
with an environment.For convenience, there is a commandenvironments treethat produces
output similar to the Unixtreecommand:$psec-ed2environmentstree/Users/dittrich/.secrets/d2
├── backups
│   ├── black.secretsmgmt.tk
│   │   ├── letsencrypt_2018-04-06T23:36:58PDT.tgz
│   │   └── letsencrypt_2018-04-25T16:32:20PDT.tgz
│   ├── green.secretsmgmt.tk
│   │   ├── letsencrypt_2018-04-06T23:45:49PDT.tgz
│   │   └── letsencrypt_2018-04-25T16:32:20PDT.tgz
│   ├── purple.secretsmgmt.tk
│   │   ├── letsencrypt_2018-04-25T16:32:20PDT.tgz
│   │   ├── trident_2018-01-31T23:38:48PST.tar.bz2
│   │   └── trident_2018-02-04T20:05:33PST.tar.bz2
│   └── red.secretsmgmt.tk
│       ├── letsencrypt_2018-04-06T23:45:49PDT.tgz
│       └── letsencrypt_2018-04-25T16:32:20PDT.tgz
├── dittrich.asc
├── keys
│   └── opendkim
│       └── secretsmgmt.tk
│           ├── 201801.private
│           ├── 201801.txt
│           ├── 201802.private
│           └── 201802.txt
├── secrets.d
│   ├── ca.json
│   ├── consul.json
│   ├── jenkins.json
│   ├── rabbitmq.json
│   ├── trident.json
│   ├── vncserver.json
│   └── zookeper.json
├── secrets.json
└── vault_password.txtTo just see the directory structure and not files, add the--no-filesoption:$psec-ed2environmentstree--no-files/Users/dittrich/.secrets/d2
├── backups
│   ├── black.secretsmgmt.tk
│   ├── green.secretsmgmt.tk
│   ├── purple.secretsmgmt.tk
│   └── red.secretsmgmt.tk
├── keys
│   └── opendkim
│       └── secretsmgmt.tk
└── secrets.dSecrets and group descriptionsThe environment directories just created are all empty. Secrets are stored in a
JSON file (.json) within the environment’s directory, and group descriptions
are stored in a drop-in directory with the same base name, but with an
extention of.dinstead of.json(following the Linux drop-in
configuration style directories used by programs likersyslog,dnsmasq,
etc.)The default secrets file name issecrets.json, which means the default
descriptions directory would be namedsecrets.d.You can define environment variables to point to the secrets base directory
in which a set of different environments can be configured at one
time, to define the current environment, and to change the name
of the secrets file to something else.$env|grep^D2_D2_SECRETS_BASEDIR=/Users/dittrich/.secrets
D2_ENVIRONMENT=doEach environment is in turn rooted in a directory with the environment’s
symbolic name (e.g.,dofor DigitalOcean in this example, andgoSecurefor the GitHubdavedittrich/goSecureVPN project.)$tree-L1~/.secrets/Users/dittrich/.secrets
├── do
└── goSecure

3 directories, 0 filesEach set of secrets for a given service or purpose is described in its own
file..
├── secrets.d
│   ├── ca.json
│   ├── consul.json
│   ├── jenkins.json
│   ├── rabbitmq.json
│   ├── trident.json
│   ├── vncserver.json
│   └── zookeper.json
└── secrets.jsonYou can see one of the descriptions files from the template
in this repository usingcat tests/secrets.d/myapp.json:[{"Variable":"myapp_pi_password","Type":"password","Prompt":"Password for myapp 'pi' user account","Export":"DEMO_pi_password"},{"Variable":"myapp_app_password","Type":"password","Prompt":"Password for myapp web app","Export":"DEMO_app_password"},{"Variable":"myapp_client_psk","Type":"string","Prompt":"Pre-shared key for myapp client WiFi AP","Options":"*","Export":"DEMO_client_psk"},{"Variable":"myapp_client_ssid","Type":"string","Prompt":"SSID for myapp client WiFi AP","Options":"myapp_ssid,*","Export":"DEMO_client_ssid"},{"Variable":"myapp_ondemand_wifi","Type":"boolean","Prompt":"'Connect on demand' when connected to wifi","Options":"true,false","Export":"DEMO_ondemand_wifi"},{"Variable":"myapp_optional_setting","Type":"boolean","Prompt":"Optionally do something","Options":"false,true","Export":"DEMO_options_setting"}]Thepsecprogram uses theopenstack/cliffcommand line
interface framework, which supports multiple output formats. The default
format thetableformat, which makes for nice clean output. (Other
formats will be described later.)The groups can be listed using thegroups listcommand:$psecgroupslist+---------+-------+
| Group   | Items |
+---------+-------+
| jenkins |     1 |
| myapp   |     4 |
| trident |     2 |
+---------+-------+The variables in one or more groups can be shown with
thegroups showcommand:$psecgroupsshowtridentmyapp+---------+-----------------------+
| Group   | Variable              |
+---------+-----------------------+
| trident | trident_sysadmin_pass |
| trident | trident_db_pass       |
| myapp   | myapp_app_password    |
| myapp   | myapp_client_psk      |
| myapp   | myapp_client_ssid     |
| myapp   | myapp_ondemand_wifi   |
| myapp   | myapp_pi_password     |
+---------+-----------------------+When integrating a new open source tool or project, you can create
a new group and clone its secrets descriptions. This does not copy
any values, just the descriptions, allowing the current environment
to manage its own values.$psecgroupscreatenewgroup--clone-from~/git/goSecure/secrets/secrets.d/gosecure.jsoncreated new group "newgroup"$psecgroupslist2>/dev/null+----------+-------+
| Group    | Items |
+----------+-------+
| jenkins  |     1 |
| myapp    |     5 |
| newgroup |    12 |
| trident  |     2 |
+----------+-------+Showing SecretsTo examine the secrets, use thesecrets showcommand:$psecsecretsshow+------------------------+----------+----------+------------------------+
| Variable               | Type     | Value    | Export                 |
+------------------------+----------+----------+------------------------+
| jenkins_admin_password | password | REDACTED | jenkins_admin_password |
| myapp_app_password     | password | REDACTED | DEMO_app_password      |
| myapp_client_psk       | string   | REDACTED | DEMO_client_ssid       |
| myapp_client_ssid      | string   | REDACTED | DEMO_client_ssid       |
| myapp_ondemand_wifi    | boolean  | REDACTED | DEMO_ondemand_wifi     |
| myapp_pi_password      | password | REDACTED | DEMO_pi_password       |
| trident_db_pass        | password | REDACTED | trident_db_pass        |
| trident_sysadmin_pass  | password | REDACTED | trident_sysadmin_pass  |
+------------------------+----------+----------+------------------------+By default, the values of secrets are redacted when output.  To show
the values in clear text in the terminal output, add the--no-redactflag:$psecsecretsshow--no-redact+------------------------+----------+------------------------------+------------------------+
| Variable               | Type     | Value                        | Export                 |
+------------------------+----------+------------------------------+------------------------+
| jenkins_admin_password | password | fetch.outsider.awning.maroon | jenkins_admin_password |
| myapp_app_password     | password | fetch.outsider.awning.maroon | DEMO_app_password      |
| myapp_client_psk       | string   | PSK                          | DEMO_client_psk        |
| myapp_client_ssid      | string   | SSID                         | DEMO_client_ssid       |
| myapp_ondemand_wifi    | boolean  | true                         | DEMO_ondemand_wifi     |
| myapp_pi_password      | password | fetch.outsider.awning.maroon | DEMO_pi_password       |
| trident_db_pass        | password | fetch.outsider.awning.maroon | trident_db_pass        |
| trident_sysadmin_pass  | password | fetch.outsider.awning.maroon | trident_sysadmin_pass  |
+------------------------+----------+------------------------------+------------------------+If you don’t care about redaction and want to turn it off and save
the dozen keystrokes it takes to type `` –no-redact``, you can export
the environment variableD2_NO_REDACTset to (case-insensitive)
“true”, “1”, or “yes”. Anything else leaves the default the same.
We’ll do this now for later examples.$exportD2_NO_REDACT=trueThe default is also to show all secrets. If you only want to process a
subset of secrets, you have two ways to do this.Specify the variables you want to show on the command line as arguments:$psecsecretsshowrabbitmq_default_user_passrabbitmq_admin_user_pass+----------------------------+----------+--------------------------------------+
| Variable                   | Type     | Value                                |
+----------------------------+----------+--------------------------------------+
| rabbitmq_default_user_pass | password | handheld.angrily.letdown.frisk       |
| rabbitmq_admin_user_pass   | password | handheld.angrily.letdown.frisk       |
+----------------------------+----------+--------------------------------------+Use the--groupflag and specify the group(s) you want to show
as command line arguments:$psecsecretsshow--groupjenkinstrident+----------------------------+----------+--------------------------------------+
| Variable                   | Type     | Value                                |
+----------------------------+----------+--------------------------------------+
| jenkins_admin_password     | password | handheld.angrily.letdown.frisk       |
| trident_db_pass            | password | handheld.angrily.letdown.frisk       |
| trident_sysadmin_pass      | password | handheld.angrily.letdown.frisk       |
+----------------------------+----------+--------------------------------------+Describing Secrets and Secret TypesTo describe the secrets in the select environment, use thesecrets describecommand:$psecsecretsdescribe+----------------------------+----------+--------------------------------------------+
| Variable                   | Type     | Prompt                                     |
+----------------------------+----------+--------------------------------------------+
| google_oauth_client_id     | string   | Google OAuth2 client id                    |
| google_oauth_client_secret | string   | Google OAuth2 client secret                |
| google_oauth_refresh_token | string   | Google OAuth2 refresh token                |
| google_oauth_username      | None     | google_oauth_username                      |
| jenkins_admin_password     | password | Password for Jenkins "admin" account       |
| myapp_app_password         | password | Password for myapp web app                 |
| myapp_client_psk           | string   | Pre-shared key for myapp client WiFi AP    |
| myapp_client_ssid          | string   | SSID for myapp client WiFi AP              |
| myapp_ondemand_wifi        | boolean  | "Connect on demand" when connected to wifi |
| myapp_pi_password          | password | Password for myapp "pi" user account       |
| trident_db_pass            | password | Password for Trident postgres database     |
| trident_sysadmin_pass      | password | Password for Trident sysadmin account      |
+----------------------------+----------+--------------------------------------------+$psecsecretsdescribe--grouptrident+-----------------------+----------+----------------------------------------+
| Variable              | Type     | Prompt                                 |
+-----------------------+----------+----------------------------------------+
| trident_db_pass       | password | Password for Trident postgres database |
| trident_sysadmin_pass | password | Password for Trident sysadmin account  |
+-----------------------+----------+----------------------------------------+To get a description of the available secret types, add the--typesflag.$psecsecretsdescribe--types+------------------+----------------------------------+
| Type             | Description                      |
+------------------+----------------------------------+
| password         | Simple (xkcd) password string    |
| string           | Simple string                    |
| boolean          | Boolean ("true"/"false")         |
| crypt_6          | crypt() SHA512 ("$6$")           |
| token_hex        | Hexadecimal token                |
| token_urlsafe    | URL-safe token                   |
| sha256_digest    | DIGEST-SHA256 (user:pass) digest |
| uuid4            | UUID4 token                      |
| random_base64    | Random BASE64 token              |
+------------------+----------------------------------+NoteThe typestringis for secrets that are managed by another entity that you
must obtain and use to access some remote service (e.g., the pre-shared key for
someone’s WiFi network, or an API key for accessing a cloud service provider’s
platform). All other types are structured secret types that you generate for
configuring services.Generating and Setting variablesSecrets are generated using thesecrets generatecommand
and are set manually using thesecrets setcommand.$psechelpsecretsgenerateusage: psec secrets generate [-h] [-U] [args [args ...]]

Generate values for secrets

positional arguments:
  args

optional arguments:
  -h, --help    show this help message and exit
  -U, --unique  Generate unique values for each type of secret (default:
                False)

..$psecsecretsset--helpusage: psec secrets set [-h] [--undefined] [args [args ...]]

Set values manually for secrets

positional arguments:
  args

optional arguments:
  -h, --help   show this help message and exit
  --undefined  Set values for undefined variables (default: False)To regenerate all of the non-string secrets at once, using the same value for
each type of secret to simplify things, use thesecrets generatecommand:$psecsecretsgenerate$psecsecretsshow--columnVariable--columnValue+----------------------------+----------------------------------------------+
| Variable                   | Value                                        |
+----------------------------+----------------------------------------------+
| trident_db_pass            | gargle.earlobe.eggplant.kissable             |
| ca_rootca_password         | gargle.earlobe.eggplant.kissable             |
| consul_key                 | HEvUAItLFZ0+GjxfwTxLDKq5Fbt86UtXrInzpf71GGY= |
| jenkins_admin_password     | gargle.earlobe.eggplant.kissable             |
| rabbitmq_default_user_pass | gargle.earlobe.eggplant.kissable             |
| rabbitmq_admin_user_pass   | gargle.earlobe.eggplant.kissable             |
| trident_sysadmin_pass      | gargle.earlobe.eggplant.kissable             |
| vncserver_password         | gargle.earlobe.eggplant.kissable             |
| zookeeper_uuid4            | 769a77ad-b06f-4018-857e-23f970c777c2         |
+----------------------------+----------------------------------------------+You can set one or more variables manually usingsecrets setand
specifying the variable and value in the formvariable=value:$psecsecretssettrident_db_pass="rural coffee purple sedan"$psecsecretsshow--columnVariable--columnValue+----------------------------+----------------------------------------------+
| Variable                   | Value                                        |
+----------------------------+----------------------------------------------+
| trident_db_pass            | rural coffee purple sedan                    |
| ca_rootca_password         | gargle.earlobe.eggplant.kissable             |
| consul_key                 | HEvUAItLFZ0+GjxfwTxLDKq5Fbt86UtXrInzpf71GGY= |
| jenkins_admin_password     | gargle.earlobe.eggplant.kissable             |
| rabbitmq_default_user_pass | gargle.earlobe.eggplant.kissable             |
| rabbitmq_admin_user_pass   | gargle.earlobe.eggplant.kissable             |
| trident_sysadmin_pass      | gargle.earlobe.eggplant.kissable             |
| vncserver_password         | gargle.earlobe.eggplant.kissable             |
| zookeeper_uuid4            | 769a77ad-b06f-4018-857e-23f970c777c2         |
+----------------------------+----------------------------------------------+Caution!Note in the example above that the command argument istrident_db_pass="ruralcoffee purple sedan"and nottrident_db_pass='ruralcoffee purple sedan'.
When using thevariable=valueform of thesecrets setcommand with a value that contains spaces, youmustquote the value with
the double-quote character (") as opposed to the single-quote
(apostrophe, or') character. The Bash shell (and possibly other
shells) will not properly parse the command line and the resultingsys.argvargument vector will be incorrectly set as seen here:_sys.argv[1:] = {list} <class 'list'>: ['--debug', 'secrets', 'set', 'trident_db_password=rural coffee purple sedan']
 0 = {str} '--debug'
 1 = {str} 'secrets'
 2 = {str} 'set'
 3 = {str} 'trident_db_password=rural coffee purple sedan'
 __len__ = {int} 4


_sys.argv[1:] = {list} <class 'list'>: ['--debug', 'secrets', 'set', "trident_db_password='rural", 'coffee', 'purple', "sedan'"]
 0 = {str} '--debug'
 1 = {str} 'secrets'
 2 = {str} 'set'
 3 = {str} 'trident_db_password=\\'rural'
 4 = {str} 'coffee'
 5 = {str} 'purple'
 6 = {str} 'sedan\\''
 __len__ = {int} 7Or you can generate one or more variables in a similar manner by adding
them to the command line as arguments tosecrets generate:$psecsecretsgeneraterabbitmq_default_user_passrabbitmq_admin_user_pass$psecsecretsshow--columnVariable--columnValue+----------------------------+----------------------------------------------+
| Variable                   | Value                                        |
+----------------------------+----------------------------------------------+
| trident_db_pass            | rural.coffee.purple.sedan                    |
| ca_rootca_password         | gargle.earlobe.eggplant.kissable             |
| consul_key                 | HEvUAItLFZ0+GjxfwTxLDKq5Fbt86UtXrInzpf71GGY= |
| jenkins_admin_password     | gargle.earlobe.eggplant.kissable             |
| rabbitmq_default_user_pass | embezzle.xerox.excess.skydiver               |
| rabbitmq_admin_user_pass   | embezzle.xerox.excess.skydiver               |
| trident_sysadmin_pass      | gargle.earlobe.eggplant.kissable             |
| vncserver_password         | gargle.earlobe.eggplant.kissable             |
| zookeeper_uuid4            | 769a77ad-b06f-4018-857e-23f970c777c2         |
+----------------------------+----------------------------------------------+A set of secrets for an open source project can be bootstrapped using the
following steps:Create a template secrets environment directory that contains just
the secrets definitions. This example uses the template found
in thedavedittrich/goSecurerepository
(directoryhttps://github.com/davedittrich/goSecure/tree/master/secrets).Use this template to clone a secrets environment, which will initially
be empty:$psecenvironmentscreatetest--clone-from~/git/goSecure/secretsnew password variable "gosecure_app_password" is unset
new string variable "gosecure_client_ssid" is unset
new string variable "gosecure_client_ssid" is unset
new string variable "gosecure_client_psk" is unset
new password variable "gosecure_pi_password" is unset
new string variable "gosecure_pi_pubkey" is unset
environment directory /Users/dittrich/.secrets/test createdNoteThe warnings about undefined new variables are presented on the standard
error file handle (a.k.a.,&2). You get rid of them on the console by
redirectingstderrto/dev/nullor a file:$psecenvironmentscreatetest--clone-from~/git/goSecure/secrets2>/dev/nullenvironment directory /Users/dittrich/.secrets/test created$psec-etestsecretsshow--no-redact--fit-width+-----------------------+----------+-------+
| Variable              | Type     | Value |
+-----------------------+----------+-------+
| gosecure_app_password | password | None  |
| gosecure_client_ssid  | string   | None  |
| gosecure_client_psk   | string   | None  |
| gosecure_pi_password  | password | None  |
| gosecure_pi_pubkey    | string   | None  |
+-----------------------+----------+-------+First, generate all secrets whose type is notstring:$psec-etestsecretsgeneratenew password variable "gosecure_app_password" is unset
new string variable "gosecure_client_ssid" is unset
new string variable "gosecure_client_ssid" is unset
new string variable "gosecure_client_psk" is unset
new password variable "gosecure_pi_password" is unset
new string variable "gosecure_pi_pubkey" is unset$psec-etestsecretsshow--no-redact--fit-width+-----------------------+----------+------------------------------+
| Variable              | Type     | Value                        |
+-----------------------+----------+------------------------------+
| gosecure_app_password | password | brunt.outclass.alike.turbine |
| gosecure_client_psk   | string   | None                         |
| gosecure_client_ssid  | string   | None                         |
| gosecure_pi_password  | password | brunt.outclass.alike.turbine |
| gosecure_pi_pubkey    | string   | None                         |
+-----------------------+----------+------------------------------+Finally, manually set the remainingstringtype variables:$psec-etestsecretsset--undefinednew string variable "gosecure_client_psk" is unset
new string variable "gosecure_client_ssid" is unset
new string variable "gosecure_pi_pubkey" is unset
Pre-shared key for goSecure client WiFi AP? [None]: atjhK5AlsQMw3Zh
SSID for goSecure client WiFi AP? [None]: YourWiFiSSID
SSH public key for accessing "pi" account? [None]: @~/.ssh/new_rsa.pub$psec-etestsecretsshow--no-redact--fit-width+-----------------------+----------+------------------------------------------------------------------------------------------+
| Variable              | Type     | Value                                                                                    |
+-----------------------+----------+------------------------------------------------------------------------------------------+
| gosecure_app_password | password | brunt.outclass.alike.turbine                                                             |
| gosecure_client_psk   | string   | atjhK5AlsQMw3Zh
| gosecure_client_ssid  | string   | YourWiFiSSID                                                                             |
| gosecure_pi_password  | password | brunt.outclass.alike.turbine                                                             |
| gosecure_pi_pubkey    | string   | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+qUIucrPvRkTmY0tgxr9ac/VtBUHhYHfOdDVpU99AcryLMWiU |
|                       |          | uQ2/NVikfOfPo5mt9YTQyqRbeBzKlNgbHnsxh0AZatjhK5AlsQMw3ZhZUcLYZbt7szuQy8ineN0potlCJoVaMSOb |
|                       |          | 9htf9gAPvzwxUnHxg35jPCzAXYAi3Erc6y338+CL0XxQvCogXOA+MwH7wZGgdT3WpupLG/7HAr/3KJEQQk1FlS2m |
|                       |          | Rd+WuewnLbKkqBP21N+48ccq6XhEhAmlzzr9SENw5DMmrvMAYIYkoTwUeD3Qx4YebjFkCxZw+w7AafEFn0Kz6vCX |
|                       |          | 4mp/6ZF/Ko+o04HM2sVr6wtCu2dB dittrich@localhost                                          |
+-----------------------+----------+------------------------------------------------------------------------------------------+NoteIf you don’t want to see the warnings about new variables that are not
defined, simply add the-qflag.$psec-qsecretsgenerate$psec-qsecretsset--undefinedPre-shared key for goSecure client WiFi AP? [None]:You are now ready to compile your software, or build your project!There is also a mechanism to run simple commands (i.e., basic arguments with
no special inline command substitution or variable expansion features of
shells likebash) and use the resulting output as the value.For this example, let’s assume an environment that requires a CIDR
notation address for ingres access control (e.g., when using Amazon
Web Services to allow control of instances from your remote laptop).$psec-exgtsecretssetaws_cidr_allowed=""$psec-esecretsshow--no-redactaws_cidr_allowed+------------------+--------+-------+
| Variable         | Type   | Value |
+------------------+--------+-------+
| aws_cidr_allowed | string |       |
+------------------+--------+-------+Thepsecprogram has a utility feature that will return
the current routable IP source address as an IP address, or using CIDR
notation.  The variable can be set in one of two ways:Via (non-interactive) inline command subtitution from the terminal shell:$psec-exgtsecretssetaws_cidr_allowed="$(psecutilsmyip--cidr)"Interactively when prompted using simple command line form:$psec-exgtsecretssetaws_cidr_allowedaws_cidr_allowed? []: !psec utils myip --cidrThe variable now contains the output of the specified program:$psecsecretsshow--no-redactaws_cidr_allowed+------------------+--------+------------------+
| Variable         | Type   | Value            |
+------------------+--------+------------------+
| aws_cidr_allowed | string | 93.184.216.34/32 |
+------------------+--------+------------------+NoteIf you work from behind a static NAT firewall, this IP address will
likely not change very often (if at all). If you are using a mobile device
that is assigned differing DHCP addresses depending on location, the IP address
may change fairly regularly and the initial AWS Security Group setting will
begin to block access to your cloud instances. Programs liketerraformcan refresh their state, allowing you to simply reset the variable used to
create the Security Group and re-apply the plan to regenerate the AWS
Security Group and re-enable your remote access.Sharing secretsThepsecprogram has a mechanism for sharing secrets with
others using GPG encrypted email messages for securing secrets in transit
and at rest in users’ inboxes. Email is sent using Google’s OAuth2
authenticated SMTP services.NoteThe Electronic Frontier Foundation (EFF) has aSurveillance Self-Defense
Guidethat includes guides onHow to Use PGP for Linuxand other operating
systems. Follow their instructions if you are new to PGP/GPG.The command issecrets send.$psecsecretssend--helpusage: psec secrets send [-h] [-T] [--test-smtp] [-H SMTP_HOST]
                         [-U SMTP_USERNAME] [-F SMTP_SENDER] [-S SMTP_SUBJECT]
                         [args [args ...]]

Send secrets using GPG encrypted email. Arguments are USERNAME@EMAIL.ADDRESS
and/or VARIABLE references.

positional arguments:
  args

optional arguments:
  -h, --help            show this help message and exit
  -T, --refresh-token   Refresh Google API Oauth2 token and exit (default:
                        False)
  --test-smtp           Test Oauth2 SMTP authentication and exit (default:
                        False)
  -H SMTP_HOST, --smtp-host SMTP_HOST
                        SMTP host (default: localhost)
  -U SMTP_USERNAME, --smtp-username SMTP_USERNAME
                        SMTP authentication username (default: None)
  -F SMTP_SENDER, --from SMTP_SENDER
                        Sender address (default: 'noreply@nowhere')
  -S SMTP_SUBJECT, --subject SMTP_SUBJECT
                        Subject line (default: 'For Your Information')Any arguments (args) that contain the@symbol are assumed to be email
addresses while the rest are assumed to be the names of secrets variables
to be sent.All recipients must have GPG public keys in your keyring.  An exception is thrown
if no GPG key is associated with the recipient(s) email addresses.$psecsecretssenddittrich@u.washington.edumyapp_app_passwordSetting homedir to '/Users/dittrich/.gnupg'

Initialised settings:
binary: /usr/local/bin/gpg
binary version: 1.4.11\ncfg:pubkey:1;2;3;16;17\ncfg:cipher:2;3;4;7;8;9;10;11;12;13\ncfg:ciphername:3DES;CAST5;BLOWFISH;AES;AES192;AES256;TWOFISH;CAMELLIA128;CAMELLIA192;CAMELLIA256\ncfg:digest:1;2;3;8;9;10;11\ncfg:digestname:MD5;SHA1;RIPEMD160;SHA256;SHA384;SHA512;SHA224\ncfg:compress:0;1;2;3\n'
homedir: /Users/dittrich/.gnupg
ignore_homedir_permissions: False
keyring: /Users/dittrich/.gnupg/pubring.gpg
secring: /Users/dittrich/.gnupg/secring.gpg
default_preference_list: SHA512 SHA384 SHA256 AES256 CAMELLIA256 TWOFISH AES192 ZLIB ZIP Uncompressed
keyserver: hkp://wwwkeys.pgp.net
options: None
verbose: False
use_agent: False

Creating the trustdb is only available with GnuPG>=2.x
sent encrypted secrets to dittrich@u.washington.eduUse-qto produce no extraneous output.$psec-qsecretssenddittrich@u.washington.edumyapp_app_passwordThe resulting email looks like this:Message-ID: <5bac64ce.1c69fb81.b136e.45ae@mx.google.com>
Date: Wed, 26 Sep 2018 22:04:14 -0700 (PDT)
From: dave.dittrich@gmail.com
X-Google-Original-From: noreply@nowhere
Content-Type: multipart/related; boundary="===============6413073026511107073=="
MIME-Version: 1.0
Subject: For Your Information
To: dittrich@u.washington.edu

This is a multi-part message in MIME format.
--===============6413073026511107073==
Content-Type: multipart/alternative; boundary="===============2830935289665347054=="
MIME-Version: 1.0

--===============2830935289665347054==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tCgpoUUlXQStSZlhnK3dLTGJlRUFnZlFNcjZYb0lT
cS9BaTlMbEVpZTFTejd5ckEzUmN4SWdjb01XTUNSM3JBaXBHCjF0TTJoZkpxRGJZOThSOEVST01F
aVltSzR2aVJ4ZjgrSU54NU54SUJPbFh1T1JQTy82NElUKzdrVSt5aDZGV00KNU1MK0Jkb21sQzNF
eC9pd3hwbTJ1R2FPczFpcU9DaDIxbTd5RnJWYkNVSW5NN1ZiMTEwck41aXNOZ3BFdndrQgpaZHhp
alJqazdtYVl1eFNkc2c3Y2RVQ29uSmdBR214QU0vZkFzOTREcHNrYkwzMFpqZE1iRHlMbUk4NWp2
QU45CjU3KzAxLzM1MEMyN1hrbEUxdEZudWNlRkRqZ04zeEd4K2Zud0pqdkFpNUpaVHltanRkQi9r
dUZUMlJTTmJJTlAKMWRZdHp4WGxNeVd0SVphNDVYcHdNenZ1TkFTbEJtbENjQXk4YlluSEJmeFRy
SGdJSUlCMlZNY1N6dmdjR3BtVApkYzZqaDVOeEV1bWljOWdXMmplSnFqRHRtdW9Ib3dxZldZb2xX
bGlXUTMrNDNzeVkrdHFlMGgvWEwzS2ZxSTMrClZzWWdyQmpGd0hnem1INEthMWxucXdUZkMzZTJ3
cUI4Uk5hcllqcXAzbHFQOVBhMHdzSVVWMHVYN2dhL01kVWcKdHNRSktPWWJRTnlXVTFLZEZWNHl4
Ynp1TWVlQ3ltMmxMbXJwVks5T3hCV04vbCtXMjRsWmhkck9TcGFJQnpNdgpnc1p3VWVuVzBXR054
bklwUGhoSWRuVE40ZlNscE5JVDhMcmJYeUhoY2ZVS2lsUDNpeEVPRS9Lc25QUFJNTURFCk9SY0xT
Z3FMMTB4b0toMnNzZTNxNG5RaHZkZW5IVVVxVjJ0WW1UVmRCNVl3cTN1MFdtY3BGSGU2NnBZeTBB
VSsKdzRjb2JVM2crQWtJMHBNQnllRzZYaWV4VzF1UzRLVVVnaFlhWVlYQ2dnazJZNEpZT05QSDJJ
NlIydmxuNjFsVApZdm1tR0NNamw3cC9pTnE2RWJpbndoMnNsbkpLMHd3S1BIbVBPUjJvRjdWREN0
dE9idHA0cEZUWTNHalByc0dRCkNDT3dYR2hCSFVQRnY2c3R4NEdtUi9GUWpBRWxxaEpjQWtTbDFz
WWhsUFRhSmEyVGgyNG81L1lPUmxRaHhhRUgKUEFrNFgzcGVCMk9UVjRNR2RCOD0KPTc0aXEKLS0t
LS1FTkQgUEdQIE1FU1NBR0UtLS0tLQo=

--===============2830935289665347054==
Content-Type: text/html; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

VGhlIGZvbGxvd2luZyBzZWNyZXQgaXMgYmVpbmcgc2hhcmVkIHdpdGggeW91OgoKbXlhcHBfYXBw
X3Bhc3N3b3JkPWJydW50IG91dGNsYXNzIGFsaWtlIHR1cmJpbmU=

--===============2830935289665347054==--

--===============6413073026511107073==--Decrypted, it looks like this:Date: Wed, 26 Sep 2018 22:04:14 -0700 (PDT)
From: dave.dittrich@gmail.com
Subject: For Your Information
To: dittrich@u.washington.edu

The following secret is being shared with you:

myapp_app_password=brunt.outclass.alike.turbine

--
Sent using psec version 23.4.2
https://pypi.org/project/python-secrets/
https://github.com/davedittrich/python_secretsA group of secrets required for Google’sOAuth 2.0 Mechanismis provided
and must be set according to Google’s instructions. See also:https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThroughhttp://blog.macuyiko.com/post/2016/how-to-send-html-mails-with-oauth2-and-gmail-in-python.htmlhttps://developers.google.com/api-client-library/python/guide/aaa_oauthhttps://github.com/google/gmail-oauth2-tools/blob/master/python/oauth2.pyhttps://developers.google.com/identity/protocols/OAuth2$psecgroupsshowoauth+-------+----------------------------+
| Group | Variable                   |
+-------+----------------------------+
| oauth | google_oauth_client_id     |
| oauth | google_oauth_client_secret |
| oauth | google_oauth_refresh_token |
+-------+----------------------------+Processing templatesOutputting structured information for use in other scriptsOnce secrets are created and stored, they will eventually need to be accessed
in order to use them in program execution.  This can be done by passing the.jsonsecrets file itself to a program, or by outputting the variables in
other formats like CSV, JSON, or as environment type variables.Passing the secrets file by pathOne way to do this is to take advantage of command line options likeAnsible’s--extra-varsand passing it a path to the.jsonsecrets
file.  (SeePassing Variables On The Command Line). Here is how to do
it.Let’s assume we want to useconsul_keyvariable to configure Consul
using Ansible. Here is the variable as stored:$psecsecretsshowconsul_key+------------+-----------+----------------------------------------------+
| Variable   | Type      | Value                                        |
+------------+-----------+----------------------------------------------+
| consul_key | token_hex | HEvUAItLFZ0+GjxfwTxLDKq5Fbt86UtXrInzpf71GGY= |
+------------+-----------+----------------------------------------------+Using Ansible’sdebugmodule, we can verify that this variable is not
set by any previously loaded Ansible inventory:$ansible-ilocalhost,-mdebug-a'var=consul_key'localhostlocalhost | SUCCESS => {
    "consul_key": "VARIABLE IS NOT DEFINED!"
}In order for Ansible to set theconsul_keyvariable outside of any
pre-defined inventory files, we need to pass a file path to the--extra-varsoption. The path can be obtained using thepsec secrets pathcommand:$psecsecretspath/Users/dittrich/.secrets/python_secrets/secrets.jsonIt is possible to run this command in an in-line command expansion operation in
Bash. Ansible expects the file path passed to-extra-varsto start with an@character, so the command line to use would look like this:$ansible-ilocalhost,-e@"$(psecsecretspath)"-mdebug-a'var=consul_key'localhostlocalhost | SUCCESS => {
    "consul_key": "HEvUAItLFZ0+GjxfwTxLDKq5Fbt86UtXrInzpf71GGY="
}Ansible now has the value and can use it in templating configuration files, or
so forth.Other programs like Hashicorpterraformlook for environment variables that
begin withTF_VAR_and use them to setterraformvariables for use
in modules. To prove we are running in a sub-shell, we will first change the
shell prompt.$PS1="test> "test> psec -e test --export-env-vars --env-var-prefix="TEST_" run bash$env|grep'^TEST_'TEST_gosecure_pi_pubkey=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+qUIucrPvRkTmY0tgxr9ac/VtBUHhYHfOdDVpU99AcryLMWiU [...]
TEST_gosecure_client_psk=atjhK5AlsQMw3Zh
TEST_gosecure_client_ssid=YourWiFiSSID
TEST_gosecure_pi_password=brunt.outclass.alike.turbine
TEST_gosecure_app_password=brunt.outclass.alike.turbine$exittest>Operational SecurityAs noted in the Limitations section above, secrets are stored in plaintext
plaintext form (i.e., they arenotencrypted) in files. Those files are in
turn stored in a directory in the file system, subject to Linux file
ownership and permission access controls.The default location for storing these files is in anenvironment directoryin a subdirectory of the user’s home directory whose name starts with a period
character (a.k.a., adot).  Files (or directories) whose name starts with a
period are known asdot files, orhidden filesbecause thelscommand
does not show it unless you use the-aflag.The secrets environment directories can also be used to store other files
besides secrets. One such use case is storing JSON Web Tokens (JWTs) used as
bearer tokens by protocols like Google’sOAuth 2.0 Mechanismfor securing
access to web services and APIs. While this improves security in terms of
remote access, is not not without its own risks (including the JWT file being
stored in the file system for an indefinite period of time).JSON Web Tokens (JWT) are Dangerous for User Sessions—Here’s a
Solution, by Raja Rao, June 24, 2021Stop Using JSON Web Tokens For Authentication. Use Stateful Sessions
Instead, by Francisco Sainz, April 4, 2022What’s the Secure Way to Store JWT?, by Yang Liu, July 23, 2020Besides JWTs, other use cases for storing sensitive files withinpsecenvironments include backups of database contents, Let’s Encrypt certificates,
SSH keys, or other secrets necessary for ensuring cloud instances can be
destroyed and recreated without losing state or requiring regeneration
(and redistribution or revalidation) of secrets.The output ofinit –helpmentions this risk and offers a way to mitigate
some of the risk by locating the secrets storage base directory within a
directory that is stored on an encrypted USB-connected disk device or encrypted
disk image, or a removable device or remote file system, that is only mounted
when needed and unmounted as soon as possible. This ensures sensitive data that
are not being actively used are left encrypted in storage.  TheD2_SECRETS_BASEDIRenvironment variable or-doption allow you to specify
the directory to use.ThepsecCLI has a secure deletion mechanism that over-writes file contents
prior to deletion, helping to reduce leaving remnants of secrets in unallocated
file system storage, similar to the way the Linuxshredcommand works.Python Script SecurityLast, but certainly not least, take the time to read up onPython Securityand understand the types and sources of security vulnerabilities related to
Python programs. Keep these ideas in mind when using and/or modifying this
program.As part of testing, theBanditsecurity validation program is used.
(SeeGetting started with Bandit).In situations where Bandit warnings can safely be ignored, the# noseccomment appears on source code lines. Comments as to why these can be
safely ignored are included in the code. (Please feel free to issue pull
requests if you disagree.)One runtime security mechanism employed bypsecis control of the process’umask. This is important when running programs that create files, which
will inherit their permissions per the processumask. Theumaskwill be
inherited by every new child process and can be set in the user’s.bashrc(or other shell initialization) file.Thepsec runcommand can be used to run programs as child processes,
optionally exporting environment variables as well, so controlling theumaskresults in improved file permission security regardless of
whether the user knows to set their processumask.You can see the effect in these two examples.First, by setting theumaskto0you see the very permissive file
permissions (as well as getting a warning frompsecabout finding a file
with lax permissions):$psec--umask0o000run--ddif=/dev/randomcount=1of=$(psecenvironmentspath--tmpdir)/foo1+0 records in
1+0 records out
512 bytes copied, 0.000019 s, 2.7 MB/s$ls-l$(psecenvironmentspath--tmpdir)/foo[!] file /Users/dittrich/.secrets/python_secrets/tmp/foo is mode 0o100666
-rw-rw-rw- 1 dittrich staff 512 Sep  8 13:05 /Users/dittrich/.secrets/python_secrets/tmp/foo$rm$(psecenvironmentspath--tmpdir)/fooNow when using the default--umaskvalue, the file permissions are restricted
(and thus no more warning):$psecrun--ddif=/dev/randomcount=1of=$(psecenvironmentspath--tmpdir)/foo1+0 records in
1+0 records out
512 bytes copied, 0.000243 s, 2.1 MB/s$ls-l$(psecenvironmentspath--tmpdir)/foo-rw------- 1 dittrich staff 512 Sep  8 13:04 /Users/dittrich/.secrets/python_secrets/tmp/foo$rm$(psecenvironmentspath--tmpdir)/fooBugs, Enhancements, and Future WorkFeature requests (and of course bug reports) are highly encouraged. You can
do that byopening an issueon GitHub. Better yet, make apull
requestwith your own fix or feature. (Check there to see if one
may already exist.)If you want to help, there are some things that are on the “to do”
list. These are tracked on this repository’s GitHubProjectspage.General or more elaborate potential enhancements are listed here:Increase test coverage (test driven development is a Good Thing(TM))The Mantl project (GitHubmantl/mantl) employs asecurity-setupscript
that takes care of setting secrets (and non-secret related variables) in a
monolithic manner.  It has specific command line options, specific secret
generation functions, and specific data structures for each of the component
subsystems used bymantl/mantl. This method is not modular or extensible, and
thesecurity-setupscript is not generalized such that it can be used by
any other project.  These limitations are primary motivators for writingpython_secrets, which could eventually replacesecurity-setup.At this point, the Mantlsecurity.ymlfile can be read in and
values can be manually set, as seen here:$psec-d~/git/mantl--secrets-filesecurity.ymlsecretsshow-fyamlsecrets descriptions directory not found
- Value: admin:password
  Variable: chronos_http_credentials
- Value: chronos
  Variable: chronos_principal
- Value: S0JMz5z8oxQGQXMyZjwE0ZCmu4zeJV4oWDUrdc25MBLx
  Variable: chronos_secret
- Value: 88821cbe-c004-4cff-9f91-2bc36cd347dc
  Variable: consul_acl_agent_token
- Value: f9acbe14-28d3-4d06-a1c9-c617da5ebb4e
  Variable: consul_acl_mantl_api_token
- Value: de54ae85-8226-4146-959f-8926b0b8ee55
  Variable: consul_acl_marathon_token
- Value: dfc9b244-5140-41ad-b93a-ac5c2451fb95
  Variable: consul_acl_master_token
- Value: e149b50f-cb5c-4efe-be96-26a52efdc715
  Variable: consul_acl_secure_token
- Value: 719f2328-6446-4647-adf6-310013bac636
  Variable: consul_acl_vault_token
- Value: Z0niD1jeiTkx7xaoewJm2A==
  Variable: consul_gossip_key
- Value: true
  Variable: do_chronos_auth
- Value: true
  Variable: do_chronos_iptables
- Value: true
  Variable: do_chronos_ssl
- Value: true
  Variable: do_consul_auth
- Value: true
  Variable: do_consul_ssl
- Value: true
  Variable: do_mantl_api_auth
- Value: true
  Variable: do_mantlui_auth
- Value: true
  Variable: do_mantlui_ssl
- Value: true
  Variable: do_marathon_auth
- Value: true
  Variable: do_marathon_iptables
- Value: true
  Variable: do_marathon_ssl
- Value: true
  Variable: do_mesos_auth
- Value: true
  Variable: do_mesos_follower_auth
- Value: true
  Variable: do_mesos_framework_auth
- Value: true
  Variable: do_mesos_iptables
- Value: true
  Variable: do_mesos_ssl
- Value: false
  Variable: do_private_docker_registry
- Value: mantl-api
  Variable: mantl_api_principal
- Value: Se4R9nRy8WTAgmU9diJyIPwLYsBU+V1yBxTQumiOriK+
  Variable: mantl_api_secret
- Value: admin:password
  Variable: marathon_http_credentials
- Value: marathon
  Variable: marathon_principal
- Value: +Y5bvIsWliFvcWgbXGWa8kwT6Qf3etogQJe+cK+IV2hX
  Variable: marathon_secret
- Value:
  - principal: marathon
    secret: +Y5bvIsWliFvcWgbXGWa8kwT6Qf3etogQJe+cK+IV2hX
  - principal: chronos
    secret: S0JMz5z8oxQGQXMyZjwE0ZCmu4zeJV4oWDUrdc25MBLx
  - principal: mantl-api
    secret: Se4R9nRy8WTAgmU9diJyIPwLYsBU+V1yBxTQumiOriK+
  Variable: mesos_credentials
- Value: follower
  Variable: mesos_follower_principal
- Value: Q53uAa2mNM0UNe2RUjrX6k7QvK6ojjH1gHXYLcm3Lmfr
  Variable: mesos_follower_secret
- Value: password
  Variable: nginx_admin_password
- Value: true
  Variable: security_enabled
- Value: chronos
  Variable: zk_chronos_user
- Value: JWPO11z4lU5qeilZ
  Variable: zk_chronos_user_secret
- Value: hsr+R6YQBAOXoY84a8ne8bU0opg=
  Variable: zk_chronos_user_secret_digest
- Value: marathon
  Variable: zk_marathon_user
- Value: UBh77ok2svQAqWox
  Variable: zk_marathon_user_secret
- Value: mo2mQGXcsc21zB4wYD18jn+Csks=
  Variable: zk_marathon_user_secret_digest
- Value: mesos
  Variable: zk_mesos_user
- Value: L3t9FEMsXehqeBvl
  Variable: zk_mesos_user_secret
- Value: bHYvGteRBxou4jqJ8XWAYmOmzxs=
  Variable: zk_mesos_user_secret_digest
- Value: super
  Variable: zk_super_user
- Value: 2DyL/n/GLi3Q0pa75z9OjODGZKC1RCaEiKNV1ZXo1Wpk
  Variable: zk_super_user_secret$psec-d~/git/mantl--secrets-filesecurity.ymlsecretsshow-fcsv|grepnginx_admin_passwordsecrets descriptions directory not found
"nginx_admin_password","password"$psec-d~/git/mantl--secrets-filesecurity.ymlsecretssetnginx_admin_password=newpasswordsecrets descriptions directory not found$psec-d~/git/mantl--secrets-filesecurity.ymlsecretsshow-fcsv|grepnginx_admin_passwordsecrets descriptions directory not found
"nginx_admin_password","newpassword"There are a few things that can be done to usepsecas a replacement
for thesecurity-setupscript.  These include:Produce secrets descriptions in asecurity.ddirectory.Remove the variables that are not secrets requiring regeneration for rotation
or “break-glass” procedures (e.g., likechronos_principal, which is a
userID value, anddo_mesos_auth, which is a boolean flag).Break down more complex data structures (specifically, themesos_credentialslist of dictionaries with keysprincipalandsecret). These could
instead be discrete variables likemarathon_secret(which appears to
be the secret associated with the invariant “variable”marathon_principal).NoteAlternatively, these kind of variables could be supported by defining a typeinvariantorstringand prompting the user to provide a new value (using any current value
as the default).CreditsTools used in rendering this package:Cookiecuttercookiecutter-pypackageDevelopment of this program was supported in part under an Open Source
Development Grant from the Comcast Innovation Fund.History23.4.2 (2023-04-20)AddedAdded bats test forsecrets findcommand.ChangedSwitched to using f-strings for formatting.Fixed broken bats tests.Update GitHub Actions workflows.23.4.1 (2023-04-19)AddedAddedsecrets findcommand.Added support for new variable typeboolean.ChangedUpdated GitHub Actions workflows (default to Python 3.9.16).Drop Python 3.7, 3.8, add Python 3.11 (default to 3.10) fortox.Fixed downstream dependency andpipinstallation problems.Resolved newpep8andbanditfindings.22.6.1 (2022-06-21)AddedAdded–ignore-missingoption to continue when settings variables.Added ‘Operational Security’ section to README.22.6.0 (2022-06-10)AddedAddaboutcommand to expose selected settings for situational awareness.Addpytestcode coverage reporting.Add BATS runtime tests related to changes.ChangedFix caching bug with non-unique secret generation.Fix bugs with setting/deleting secrets.Improve secrets basedir initialization logic.Expand use ofpathlib.Path.Improvements to source code, test, and vscode launch configuration quality.22.5.1 (2022-05-25)ChangedSwitch to using factory pattern for secrets generation.General code quality and test improvements.Improvesecrets getcommand logic and help.Fixutils yaml-to-jsonsubcommand and tests.Resolve setuptools warnings.Separate utility functions fromutilssubcommands.RemovedRetireconsul_keysecret type in favor oftoken_base64.Retire insecure secrets types (e.g., use of SHA1).22.5.0 (2022-05-11)AddedTest support for Python 3.10.Add better logging controls.ChangedGeneralize Google OAuth2 email functionality.Improve use and testing of exceptions.22.1.0 (2022-01-22)AddedAddinitcommand and–initflag to initialize secrets base directory.Ensure overridden values via flags are exported to process environment
for subprocesses to use.Add missing tests for features added in a previous release.Add and start using application-specific exception classes.ChangedMove functions and variables toutilsto improve reuse ability.Useget_prefix more consistently for getter method/function names.Over-ride cliff formatter class globally in app parser setup.Usepathlib.Pathfor paths for cleaner code.Fix bugs inenvironments deletecommand.Fix bugs in–from-optionsfeature ofsecrets getandsecrets set.Improvements to source code, test, and vscode launch configuration quality.21.11.0 (2021-11-22)AddedAddHelpattribute to descriptions for URL to more information.ChangedGeneral code quality, documentation, and testing enhancementsMovetmpdirpath creation tosecrets_environment.SecretsEnvironment().Moveumask()function and variables toutils.RemovedDrop Python 3.6 support due to it being EOL.21.9.1 (2021-09-15)AddedAddedsecrets treesubcommand.ChangedFixed bugs withenvironments path –tmpdirsubcommand andrunsubcommand with–elapsedoption when no environment exists.Changed license file name.Improved documentation.21.9.0 (2021-09-07)AddedIncreased test coverage to address bugs (below) being fixed.ChangedFixed bugs inMakefileandtox.inifile.Fixed bug setting undefined variables.Switched fromnumpyto Pythonsecretsmodule for random bytes.Increased key size from 16 to 32 bits forconsul_key,token_hexandtoken_urlsafe.21.8.0 (2021-08-12)ChangedFixed bug in setup.py+setup.cfg21.7.0 (2021-07-30)AddedSecrets descriptions for demoing HypriotOS Flash mods Medium articleChangedImprovesecrets set –from-optionsGeneral code quality, documentation, and testing enhancements21.6.0 (2021-06-23)AddedAbility to set and generate secrets from defaults optionsAbility to create an alias for an existing environmentAllow retroactive mirroring of new secretsChangedSwitched frompbrtosetuptools_scmfor version numberingSwitched to more secure random number generation21.2.0 (20201-02-23)AddedImprove GitHub Actions workflowsOverall documentation and code enhancementsImprove handling of wildcards in options listChangedFix bugs with handling empty lists, cloning environments, BATS testsIncrease password complexity a bit moreFix ReadTheDocs20.11.0 (2020-11-17)AddedAddsecrets createandsecrets deletecommandsChangedNormalize all logger and exception output textRefactoring code for better modulatiryNormalizegroup createandgroup deletecodeNormalizesecrets showandsecrets describecodeFix bug that left variables missing after cloningAdd Python 3.9 to testing matrixSwitch from .yml to .json format for secretsExpand IP address support inutilssubcommand20.8.1 (2020-08-11)ChangedFixes to v20.8.020.8.0 (2020-08-11)AddedAdd GitHub workflow to publish to test.pypi.orgAddsecrets backupandsecrets restorelogicOpen web browser to documentation for helpChangedGo back to date-based version numberingGeneral CI/CD workflow updatesImprove directory handling inenvironments path20.2.15 (2012-02-15)AddedAdded Python 3.8 support to test matrixChangedFix bug inenvironments defaultPut elapsed time (and BELL) on stdoutFix bug inenvironments treeAllow setting vars using diff names+environment19.12.0 (2019-12-16)AddedAdd and document newbooleandata typeAddgroups deletecommandChangedImprove default environment handlingImprove tox+BATS testingAddress security issue per “Your xkcd passwords are pwned” articleGeneral code quality and test improvementsAdd protection from over-writing existing env varsAddOptionsattribute19.11.1 (2019-11-29)ChangedEnhancements to better support Windows 10Allow cloning group descriptions from environmentFix tty/no-tty handling withenvironments deleteExpose terraform command on-vValidate variable exists in environmentFix brokenenvironments treecode19.10.1 (2019-10-20)ChangedMove BATS unit tests into tox testingAvoid attempting interactive things when no ttyImprove file and directory permissions logic19.10.0 (2019-10-14)AddedWorking SSH key and configuration managementUsebulletfor interactive list selectionElapsed timer featureParsing of terraform output to extract SSH public keysumaskcontrol for better new file permission settingsSupport configuring terraformtfstatebackendAllow setting secrets by copying from another environmentChangedNumerous bug fixesRefine testingOption to only show undefined variablesSort environments when listing19.9.0 (2019-09-05)AddedAddenvironments deletesubcommandAllow cloning environment from an existing oneChanged19.8.3 (2019-08-28)ChangedDynamically get version numberGeneral testing enhancementsGeneral code quality enhancementsEnsure more secure file permissions19.8.2 (2019-08-23)ChangedGeneral code quality enhancements19.8.0 (2019-08-22)AddedIP address determinationAllow cloning new group in an empty environmentMakepython -m psecworkJSON output methodEnvironment aliasing featureChangedGeneral code quality and testing enhancementsBe more explicit about default environmentTighten permissions on cloned environments/groupsAdd insecure permissions checking19.5.1 (2019-05-08)ChangedAddHISTORY.rstfile19.4.5 (2019-05-08)AddedAdd commandssh configto manage SSH configuration snippet
for use byupdate-dotdeeto generate ~/.ssh/config fileAdd commandssh known-hosts addandssh known-hosts removeto manage system known_hosts file(s)ChangedGeneralized exception to fix –version bugClean up temporary docs/psec_help.txt file19.4.4 (2019-04-21)ChangedFix Bats dependencies/testsFix broken documentation (wt?)Fix messed up release tagging19.4.0 (2019-04-19)AddedPython 3.7 coverage for Travis CIChangedComplete –help output (epilog text) in all commandsInstall a script ‘psec’ to complement console_script entry pointClarify arguments in –help outputDeprecatedThe ‘python_secrets’ command is now just ‘psec’19.3.1 (2019-04-06)AddedAddenvironments renamecommandAddutilsset-aws-credentialscommand to mirror AWS CLI credentialsUseautoprogram_clifffor self-documentationAddcliff.sphinxextfor documentationChangedRefactoredSecretsEnvironment()soautoprogram_cliffworks18.11.0 (2018-11-09)AddedAdd “–type” option to “secrets describe”Improve visibility into default environmentAdd screencasts to documenationAdd RST checks to ensure PyPi documentation worksAdd feedback about minimum Python versionAdd--jsonoutput toenvironments pathAdd reference to proof-of-concept using goSecure forkChangedThe “secrets describe” command now describes variables and typesAllowsecrets setto set any type (not juststring)18.9.0 (2018-09-27)AddedSwitched to calendar version numberingFinish GPG encrypted email delivery of secretsgroups createcommandImprove error handling consistency when no environment exists0.16.0 (2018-09-12)AddedUse attribute maps instead of lookup loopsAdd Prompt attribute in descriptions for better UX when setting variablesNote new undefined variables when adding groups orenvironments create--clone-fromWhen exporting vars, also export PYTHON_SECRETS_ENVIRONMENT w/environment nameAdd reference to Python Security coding informationenvironments treecommandenvironments pathcommand with features supporting Ansible Lookup Pluginsecrets getcommandgroups pathcommandenvironments defaultcommand0.14.0 (2018-08-30)AddedOption to export secrets as environment variables (with optional prefix)Can now set secrets (any specified or all undefined) via command lineutils myipcommand returns routable IP address (with CIDR option)runcommand allows running commands with exported environment variablesChangedRenamedtemplatecomamnd toutils tfoutputRemovedDropped support for Python 3.4, 3.5, sincesecretsmodule only in Python >= 3.60.10.0 (2018-08-23)AddedNewstringtype for manually set secretssecrets pathcommand provides path to secrets.ymlfiletemplatecommand (Jinja templating)Default environment to basename of cwdClone environment from skeleton directory in repo0.9.1 (2018-08-19)Addedsecrets describecommandenvironments createcommandenvironments listcommandExpand secrets types and generation methodsAdd initial feature for sending secrets via email using Google OAuth2 SMTPRemovedDrop Python 2.7 support (at least for now…)SecurityAddsixfor securinginputcall0.8.0 (2018-05-11)(TBD)0.4.0 (2018-05-01)(TBD)0.3.6 (2018-04-29)(TBD)0.3.0 (2018-04-27)First release on PyPI.