neulab/tldr huggingface dataset:
NOTE: If you are hosting the SPM repo on a system where you can not or do not want to install Salt, you can run the spm create_repo command on the build system and then copy the packages and the generated SPM-METADATA file to the repo. You can also install SPM files directly on a Salt master, bypassing the repository completely.
Setting up a Package Repository After packages are built, the generated SPM files are placed in the srv/spm_build folder.
Where you place the built SPM files on your repository server depends on how you plan to make them available to your Salt masters.
You can share the srv/spm_build folder on the network, or copy the files to your FTP or Web server.
Adding a Package to the repository New packages are added by simply copying the SPM file to the repo folder, and then generating repo metadata.
Generate Repo Metadata Each time you update or add an SPM package to your repository, issue an spm create_repo command:
spm create_repo /srv/spm_build
SPM generates the repository metadata for all of the packages in that directory and places it in an SPM-METADATA file at the folder root. This command is used even if repository metadata already exists in that directory.
Installing SPM Packages SPM packages are installed to your Salt master, where they are available to Salt minions using all of Salt's package management functions.
Configuring Remote Repositories Before SPM can use a repository, two things need to happen. First, the Salt master needs to know where the repository is through a configuration process. Then it needs to pull down the repository metadata.
Repository Configuration Files Repositories are configured by adding each of them to the /etc/salt/spm.repos.d/spm.repo file on each Salt master. This file contains the name of the repository, and the link to the repository:
my_repo: url: https://spm.example.com/
For HTTP/HTTPS Basic authorization you can define credentials:
my_repo: url: https://spm.example.com/ username: user password: pass
Beware of unauthorized access to this file, please set at least 0640 permissions for this configuration file:
The URL can use http, https, ftp, or file.
my_repo: url: file:///srv/spm_build
Updating Local Repository Metadata After the repository is configured on the Salt master, repository metadata is downloaded using the spm update_repo command:
spm update_repo
NOTE: A file for each repo is placed in /var/cache/salt/spm on the Salt master after you run the update_repo command. If you add a repository and it does not seem to be showing up, check this path to verify that the repository was found.
Update File Roots SPM packages are installed to the srv/spm/salt folder on your Salt master.  This path needs to be added to the file roots on your Salt master manually.
file_roots: base: 1. /srv/salt 2. /srv/spm/salt
Restart the salt-master service after updating the file_roots setting.
Installing Packages To install a package, use the spm install command:
spm install apache
WARNING: Currently, SPM does not check to see if files are already in place before installing them. That means that existing files will be overwritten without warning.
Installing directly from an SPM file You can also install SPM packages using a local SPM file using the spm local install command:
spm local install /srv/spm/apache-201506-1.spm
An SPM repository is not required when using spm local install.
Pillars If an installed package includes Pillar data, be sure to target the installed pillar to the necessary systems using the pillar Top file.
Removing Packages Packages may be removed after they are installed using the spm remove command.
spm remove apache
If files have been modified, they will not be removed. Empty directories will also be removed.
SPM Configuration There are a number of options that are specific to SPM. They may be configured in the master configuration file, or in SPM's own spm configuration file (normally located at /etc/salt/spm). If configured in both places, the spm file takes precedence. In general, these values will not need to be changed from the defaults.
spm_logfile Default: /var/log/salt/spm
Where SPM logs messages.
spm_repos_config Default: /etc/salt/spm.repos
SPM repositories are configured with this file. There is also a directory which corresponds to it, which ends in .d. For instance, if the filename is /etc/salt/spm.repos, the directory will be /etc/salt/spm.repos.d/.
spm_cache_dir Default: /var/cache/salt/spm
When SPM updates package repository metadata and downloads packaged, they will be placed in this directory. The package database, normally called packages.db, also lives in this directory.
spm_db Default: /var/cache/salt/spm/packages.db
The location and name of the package database. This database stores the names of all of the SPM packages installed on the system, the files that belong to them, and the metadata for those files.
spm_build_dir Default: /srv/spm_build
When packages are built, they will be placed in this directory.
spm_build_exclude Default: ['.git']
When SPM builds a package, it normally adds all files in the formula directory to the package. Files listed here will be excluded from that package. This option requires a list to be specified.
spm_build_exclude: - .git - .svn
Types of Packages SPM supports different types of formula packages. The function of each package is denoted by its name. For instance, packages which end in -formula are considered to be Salt States (the most common type of formula). Packages which end in -conf contain configuration which is to be placed in the /etc/salt/ directory. Packages which do not contain one of these names are treated as if they have a -formula name.
FORMULA File In addition to the formula itself, a FORMULA file must exist which describes the package. An example of this file is:
name: apache os: RedHat, Debian, Ubuntu, SUSE, FreeBSD os_family: RedHat, Debian, Suse, FreeBSD version: 201506 release: 2 summary: Formula for installing Apache description: Formula for installing Apache
Required Fields This file must contain at least the following fields:
name The name of the package, as it will appear in the package filename, in the repository metadata, and the package database. Even if the source formula has -formula in its name, this name should probably not include that. For instance, when packaging the apache-formula, the name should be set to apache.
os The value of the os grain that this formula supports. This is used to help users know which operating systems can support this package.
os_family The value of the os_family grain that this formula supports. This is used to help users know which operating system families can support this package.
version The version of the package. While it is up to the organization that manages this package, it is suggested that this version is specified in a YYYYMM format.  For instance, if this version was released in June 2015, the package version should be 201506. If multiple releases are made in a month, the release field should be used.
minimum_version Minimum recommended version of Salt to use this formula. Not currently enforced.
release This field refers primarily to a release of a version, but also to multiple versions within a month. In general, if a version has been made public, and immediate updates need to be made to it, this field should also be updated.
summary A one-line description of the package.
description A more detailed description of the package which can contain more than one line.
Optional Fields The following fields may also be present.
top_level_dir This field is optional, but highly recommended. If it is not specified, the package name will be used.
Formula repositories typically do not store .sls files in the root of the repository; instead they are stored in a subdirectory. For instance, an apache-formula repository would contain a directory called apache, which would contain an init.sls, plus a number of other related files. In this instance, the top_level_dir should be set to apache.
Files outside the top_level_dir, such as README.rst, FORMULA, and LICENSE will not be installed. The exceptions to this rule are files that are already treated specially, such as pillar.example and _modules/.
dependencies A comma-separated list of packages that must be installed along with this package. When this package is installed, SPM will attempt to discover and install these packages as well. If it is unable to, then it will refuse to install this package.
This is useful for creating packages which tie together other packages. For instance, a package called wordpress-mariadb-apache would depend upon wordpress, mariadb, and apache.
optional A comma-separated list of packages which are related to this package, but are neither required nor necessarily recommended. This list is displayed in an informational message when the package is installed to SPM.
recommended A comma-separated list of optional packages that are recommended to be installed with the package. This list is displayed in an informational message when the package is installed to SPM.
files A files section can be added, to specify a list of files to add to the SPM.  Such a section might look like:
files: - _pillar - FORMULA - _runners - d|mymodule/index.rst - r|README.rst
When files are specified, then only those files will be added to the SPM, regardless of what other files exist in the directory. They will also be added in the order specified, which is useful if you have a need to lay down files in a specific order.
As can be seen in the example above, you may also tag files as being a specific type. This is done by pre-pending a filename with its type, followed by a pipe (|) character. The above example contains a document file and a readme. The available file types are:
• c: config file
• d: documentation file
• g: ghost file (i.e. the file contents are not included in the package payload)
• l: license file
• r: readme file
• s: SLS file
• m: Salt module
The first 5 of these types (c, d, g, l, r) will be placed in /usr/share/salt/spm/ by default. This can be changed by setting an spm_share_dir value in your /etc/salt/spm configuration file.
The last two types (s and m) are currently ignored, but they are reserved for future use.
Pre and Post States It is possible to run Salt states before and after installing a package by using pre and post states. The following sections may be declared in a FORMULA:
• pre_local_state
• pre_tgt_state
• post_local_state
• post_tgt_state
Sections with pre in their name are evaluated before a package is installed and sections with post are evaluated after a package is installed. local states are evaluated before tgt states.
Each of these sections needs to be evaluated as text, rather than as YAML.  Consider the following block:
pre_local_state: > echo test > /tmp/spmtest: cmd: - run
Note that this declaration uses > after pre_local_state. This is a YAML marker that marks the next multi-line block as text, including newlines. It is important to use this marker whenever declaring pre or post states, so that the text following it can be evaluated properly.
local States local states are evaluated locally; this is analogous to issuing a state run using a salt-call --local command. These commands will be issued on the local machine running the spm command, whether that machine is a master or a minion.
local states do not require any special arguments, but they must still use the > marker to denote that the state is evaluated as text, not a data structure.
tgt States tgt states are issued against a remote target. This is analogous to issuing a state using the salt command. As such it requires that the machine that the spm command is running on is a master.
Because tgt states require that a target be specified, their code blocks are a little different. Consider the following state:
pre_tgt_state: tgt: '*' data: > echo test > /tmp/spmtest: cmd: - run
With tgt states, the state data is placed under a data section, inside the *_tgt_state code block. The target is of course specified as a tgt and you may also optionally specify a tgt_type (the default is glob).
You still need to use the > marker, but this time it follows the data line, rather than the *_tgt_state line.
Templating States The reason that state data must be evaluated as text rather than a data structure is because that state data is first processed through the rendering engine, as it would be with a standard state run.
This means that you can use Jinja or any other supported renderer inside of Salt. All formula variables are available to the renderer, so you can reference FORMULA data inside your state if you need to:
pre_tgt_state: tgt: '*' data: > echo {{ name }} > /tmp/spmtest: cmd: - run
You may also declare your own variables inside the FORMULA. If SPM doesn't recognize them then it will ignore them, so there are no restrictions on variable names, outside of avoiding reserved words.
By default the renderer is set to jinja|yaml. You may change this by changing the renderer setting in the FORMULA itself.
Building a Package Once a FORMULA file has been created, it is placed into the root of the formula that is to be turned into a package. The spm build command is used to turn that formula into a package:
spm build /path/to/saltstack-formulas/apache-formula
The resulting file will be placed in the build directory. By default this directory is located at /srv/spm/.
Loader Modules When an execution module is placed in <file_roots>/_modules/ on the master, it will automatically be synced to minions, the next time a sync operation takes place. Other modules are also propagated this way: state modules can be placed in _states/, and so on.
When SPM detects a file in a package which resides in one of these directories, that directory will be placed in <file_roots> instead of in the formula directory with the rest of the files.
Removing Packages Packages may be removed once they are installed using the spm remove command.
SPM Development Guide This document discusses developing additional code for SPM.
Each function that accepts arguments has a set of required and optional arguments. Take note that SPM will pass all arguments in, and therefore each function must accept each of those arguments. However, arguments that are marked as required are crucial to SPM's core functionality, while arguments that are marked as optional are provided as a benefit to the module, if it needs to use them.
Modules for managing the package database are stored in the salt/spm/pkgdb/ directory. A number of functions must exist to support database management.
init() Get a database connection, and initialize the package database if necessary.
This function accepts no arguments. If a database is used which supports a connection object, then that connection object is returned. For instance, the sqlite3 module returns a connect() object from the sqlite3 library:
def myfunc(): conn = sqlite3.connect(__opts__["spm_db"], isolation_level=None) ... return conn
SPM itself will not use this connection object; it will be passed in as-is to the other functions in the module. Therefore, when you set up this object, make sure to do so in a way that is easily usable throughout the module.
info() Return information for a package. This generally consists of the information that is stored in the FORMULA file in the package.
The arguments that are passed in, in order, are package (required) and conn (optional).
package is the name of the package, as specified in the FORMULA.  conn is the connection object returned from init().
list_files() Return a list of files for an installed package. Only the filename should be returned, and no other information.
register_pkg() Register a package in the package database. Nothing is expected to be returned from this function.
The arguments that are passed in, in order, are name (required), formula_def (required), and conn (optional).
name is the name of the package, as specified in the FORMULA. formula_def is the contents of the FORMULA file, as a dict. conn is the connection object returned from init().
register_file() Register a file in the package database. Nothing is expected to be returned from this function.
The arguments that are passed in are name (required), member (required), path (required), digest (optional), and conn (optional).
name is the name of the package.
member is a tarfile object for the package file. It is included, because it contains most of the information for the file.
path is the location of the file on the local filesystem.
digest is the SHA1 checksum of the file.
conn is the connection object returned from init().
unregister_pkg() Unregister a package from the package database. This usually only involves removing the package's record from the database. Nothing is expected to be returned from this function.
The arguments that are passed in, in order, are name (required) and conn (optional).
name is the name of the package, as specified in the FORMULA. conn is the connection object returned from init().
unregister_file() Unregister a package from the package database. This usually only involves removing the package's record from the database. Nothing is expected to be returned from this function.
The arguments that are passed in, in order, are name (required), pkg (optional) and conn (optional).
name is the path of the file, as it was installed on the filesystem.
pkg is the name of the package that the file belongs to.
db_exists() Check to see whether the package database already exists. This is the path to the package database file. This function will return True or False.
The only argument that is expected is db_, which is the package database file.
Modules for managing the package database are stored in the salt/spm/pkgfiles/ directory. A number of functions must exist to support file management.
init() Initialize the installation location for the package files. Normally these will be directory paths, but other external destinations such as databases can be used. For this reason, this function will return a connection object, which can be a database object. However, in the default local module, this object is a dict containing the paths. This object will be passed into all other functions.
Three directories are used for the destinations: formula_path, pillar_path, and reactor_path.
formula_path is the location of most of the files that will be installed. The default is specific to the operating system, but is normally /srv/salt/.
pillar_path is the location that the pillar.example file will be installed to.  The default is specific to the operating system, but is normally /srv/pillar/.
reactor_path is the location that reactor files will be installed to. The default is specific to the operating system, but is normally /srv/reactor/.
check_existing() Check the filesystem for existing files. All files for the package will be checked, and if any are existing, then this function will normally state that SPM will refuse to install the package.
This function returns a list of the files that exist on the system.
The arguments that are passed into this function are, in order: package (required), pkg_files (required), formula_def (formula_def), and conn (optional).
package is the name of the package that is to be installed.
pkg_files is a list of the files to be checked.
formula_def is a copy of the information that is stored in the FORMULA file.
conn is the file connection object.
install_file() Install a single file to the destination (normally on the filesystem). Nothing is expected to be returned from this function.
This function returns the final location that the file was installed to.
The arguments that are passed into this function are, in order, package (required), formula_tar (required), member (required), formula_def (required), and conn (optional).
formula_tar is the tarfile object for the package. This is passed in so that the function can call formula_tar.extract() for the file.
member is the tarfile object which represents the individual file. This may be modified as necessary, before being passed into formula_tar.extract().
formula_def is a copy of the information from the FORMULA file.
remove_file() Remove a single file from file system. Normally this will be little more than an os.remove(). Nothing is expected to be returned from this function.
The arguments that are passed into this function are, in order, path (required) and conn (optional).
path is the absolute path to the file to be removed.
hash_file() Returns the hexdigest hash value of a file.
The arguments that are passed into this function are, in order, path (required), hashobj (required), and conn (optional).
path is the absolute path to the file.
hashobj is a reference to hashlib.sha1(), which is used to pull the hexdigest() for the file.
This function will not generally be more complex than:
def hash_file(path, hashobj, conn=None): with salt.utils.files.fopen(path, "r") as f: hashobj.update(f.read()) return hashobj.hexdigest()
path_exists() Check to see whether the file already exists on the filesystem. Returns True or False.
This function expects a path argument, which is the absolute path to the file to be checked.
path_isdir() Check to see whether the path specified is a directory. Returns True or False.
This function expects a path argument, which is the absolute path to be checked.
Storing Data in Other Databases The SDB interface is designed to store and retrieve data that, unlike pillars and grains, is not necessarily minion-specific. The initial design goal was to allow passwords to be stored in a secure database, such as one managed by the keyring package, rather than as plain-text files. However, as a generic database interface, it could conceptually be used for a number of other purposes.
SDB was added to Salt in version 2014.7.0.
SDB Configuration In order to use the SDB interface, a configuration profile must be set up.  To be available for master commands, such as runners, it needs to be configured in the master configuration. For modules executed on a minion, it can be set either in the minion configuration file, or as a pillar. The configuration stanza includes the name/ID that the profile will be referred to as, a driver setting, and any other arguments that are necessary for the SDB module that will be used. For instance, a profile called mykeyring, which uses the system service in the keyring module would look like:
mykeyring: driver: keyring service: system
It is recommended to keep the name of the profile simple, as it is used in the SDB URI as well.
SDB URIs SDB is designed to make small database queries (hence the name, SDB) using a compact URL. This allows users to reference a database value quickly inside a number of Salt configuration areas, without a lot of overhead. The basic format of an SDB URI is:
sdb://<profile>/<args>
The profile refers to the configuration profile defined in either the master or the minion configuration file. The args are specific to the module referred to in the profile, but will typically only need to refer to the key of a key/value pair inside the database. This is because the profile itself should define as many other parameters as possible.
For example, a profile might be set up to reference credentials for a specific OpenStack account. The profile might look like:
kevinopenstack: driver: keyring service: salt.cloud.openstack.kevin
And the URI used to reference the password might look like:
sdb://kevinopenstack/password
Getting, Setting and Deleting SDB Values Once an SDB driver is configured, you can use the sdb execution module to get, set and delete values from it. There are two functions that may appear in most SDB modules: get, set and delete.
Getting a value requires only the SDB URI to be specified. To retrieve a value from the kevinopenstack profile above, you would use:
salt-call sdb.get sdb://kevinopenstack/password
WARNING: The vault driver previously only supported splitting the path and key with a question mark. This has since been deprecated in favor of using the standard / to split the path and key. The use of the questions mark will still be supported to ensure backwards compatibility, but please use the preferred method using /.  The deprecated approach required the full path to where the key is stored, followed by a question mark, followed by the key to be retrieved.  If you were using a profile called myvault, you would use a URI that looks like:
salt-call sdb.get 'sdb://myvault/secret/salt?saltstack'
Instead of the above please use the preferred URI using / instead:
salt-call sdb.get 'sdb://myvault/secret/salt/saltstack'
Setting a value uses the same URI as would be used to retrieve it, followed by the value as another argument.
salt-call sdb.set 'sdb://myvault/secret/salt/saltstack' 'super awesome'
Deleting values (if supported by the driver) is done pretty much the same way as getting them. Provided that you have a profile called mykvstore that uses a driver allowing to delete values you would delete a value as shown below:
salt-call sdb.delete 'sdb://mykvstore/foobar'
The sdb.get, sdb.set and sdb.delete functions are also available in the runner system:
salt-run sdb.get 'sdb://myvault/secret/salt/saltstack' salt-run sdb.set 'sdb://myvault/secret/salt/saltstack' 'super awesome' salt-run sdb.delete 'sdb://mykvstore/foobar'
Using SDB URIs in Files SDB URIs can be used in both configuration files, and files that are processed by the renderer system (jinja, mako, etc.). In a configuration file (such as /etc/salt/master, /etc/salt/minion, /etc/salt/cloud, etc.), make an entry as usual, and set the value to the SDB URI. For instance:
mykey: sdb://myetcd/mykey
To retrieve this value using a module, the module in question must use the config.get function to retrieve configuration values. This would look something like:
mykey = __salt__["config.get"]("mykey")
Templating renderers use a similar construct. To get the mykey value from above in Jinja, you would use:
{{ salt['config.get']('mykey') }}
When retrieving data from configuration files using config.get, the SDB URI need only appear in the configuration file itself.
If you would like to retrieve a key directly from SDB, you would call the sdb.get function directly, using the SDB URI. For instance, in Jinja:
{{ salt['sdb.get']('sdb://myetcd/mykey') }}
When writing Salt modules, it is not recommended to call sdb.get directly, as it requires the user to provide values in SDB, using a specific URI. Use config.get instead.
Writing SDB Modules There is currently one function that MUST exist in any SDB module (get()), one that SHOULD exist (set_()) and one that MAY exist (delete()). If using a (set_()) function, a __func_alias__ dictionary MUST be declared in the module as well:
__func_alias__ = { "set_": "set", }
This is because set is a Python built-in, and therefore functions should not be created which are called set(). The __func_alias__ functionality is provided via Salt's loader interfaces, and allows legally-named functions to be referred to using names that would otherwise be unwise to use.
The get() function is required, as it will be called via functions in other areas of the code which make use of the sdb:// URI. For example, the config.get function in the config execution module uses this function.
The set_() function may be provided, but is not required, as some sources may be read-only, or may be otherwise unwise to access via a URI (for instance, because of SQL injection attacks).
The delete() function may be provided as well, but is not required, as many sources may be read-only or restrict such operations.
A simple example of an SDB module is salt/sdb/keyring_db.py, as it provides basic examples of most, if not all, of the types of functionality that are available not only for SDB modules, but for Salt modules in general.
Running the Salt Master/Minion as an Unprivileged User While the default setup runs the master and minion as the root user, some may consider it an extra measure of security to run the master as a non-root user. Keep in mind that doing so does not change the master's capability to access minions as the user they are running as. Due to this many feel that running the master as a non-root user does not grant any real security advantage which is why the master has remained as root by default.
NOTE: Some of Salt's operations cannot execute correctly when the master is not running as root, specifically the pam external auth system, as this system needs root access to check authentication.
As of Salt 0.9.10 it is possible to run Salt as a non-root user. This can be done by setting the user parameter in the master configuration file. and restarting the salt-master service.
The minion has its own user parameter as well, but running the minion as an unprivileged user will keep it from making changes to things like users, installed packages, etc. unless access controls (sudo, etc.) are setup on the minion to permit the non-root user to make the needed changes.
In order to allow Salt to successfully run as a non-root user, ownership, and permissions need to be set such that the desired user can read from and write to the following directories (and their subdirectories, where applicable):
Ownership can be easily changed with chown, like so:
# chown -R user /etc/salt /var/cache/salt /var/log/salt /var/run/salt
WARNING: Running either the master or minion with the root_dir parameter specified will affect these paths, as will setting options like pki_dir, cachedir, log_file, and other options that normally live in the above directories.
Using cron with Salt The Salt Minion can initiate its own highstate using the salt-call command.
$ salt-call state.apply
This will cause the minion to check in with the master and ensure it is in the correct "state".
Use cron to initiate a highstate If you would like the Salt Minion to regularly check in with the master you can use cron to run the salt-call command:
0 0 * * * salt-call state.apply
The above cron entry will run a highstate every day at midnight.
NOTE: When executing Salt using cron, keep in mind that the default PATH for cron may not include the path for any scripts or commands used by Salt, and it may be necessary to set the PATH accordingly in the crontab:
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin
Hardening Salt This topic contains tips you can use to secure and harden your Salt environment. How you best secure and harden your Salt environment depends heavily on how you use Salt, where you use Salt, how your team is structured, where you get data from, and what kinds of access (internal and external) you require.
IMPORTANT: The guidance here should be taken in combination with best-practices.
IMPORTANT: Refer to the saltstack_security_announcements documentation in order to stay updated and secure.
General hardening tips • Restrict who can directly log into your Salt master system.
• Use SSH keys secured with a passphrase to gain access to the Salt master system.
• Track and secure SSH keys and any other login credentials you and your team need to gain access to the Salt master system.
• Use a hardened bastion server or a VPN to restrict direct access to the Salt master from the internet.
• Don't expose the Salt master any more than what is required.
• Harden the system as you would with any high-priority target.
• Keep the system patched and up-to-date.
• Use tight firewall rules. Pay particular attention to TCP/4505 and TCP/4506 on the salt master and avoid exposing these ports unnecessarily.
Salt hardening tips WARNING: Grains can be set by users that have access to the minion configuration files on the local system, making them less secure than other identifiers in Salt. Avoid storing sensitive data, such as passwords or keys, on minions. Instead, make use of pillar and/or sdb.
IMPORTANT: Jinja supports a secure, sandboxed template execution environment that Salt takes advantage of. Other text renderers do not support this functionality, so Salt highly recommends usage of jinja / jinja|yaml.
• Subscribe to salt-users or salt-announce so you know when new Salt releases are available.
• Keep your systems up-to-date with the latest patches.
• Use Salt's Client ACL system to avoid having to give out root access in order to run Salt commands.
• Use Salt's Client ACL system to restrict which users can run what commands.
• Use external Pillar to pull data into Salt from external sources so that non-sysadmins (other teams, junior admins, developers, etc) can provide configuration data without needing access to the Salt master.
• Make heavy use of SLS files that are version-controlled and go through a peer-review/code-review process before they're deployed and run in production. This is good advice even for "one-off" CLI commands because it helps mitigate typos and mistakes.
• Use salt-api, SSL, and restrict authentication with the external auth system if you need to expose your Salt master to external services.
• Make use of Salt's event system and reactor to allow minions to signal the Salt master without requiring direct access.
• Run the salt-master daemon as non-root.
• Disable which modules are loaded onto minions with the disable_modules setting. (for example, disable the cmd module if it makes sense in your environment.)
• Look through the fully-commented sample master and minion config files. There are many options for securing an installation.
• Run masterless-mode minions on particularly sensitive minions. There is also salt-ssh or the modules.sudo if you need to further restrict a minion.
• Monitor specific security related log messages. Salt salt-master logs attempts to access methods which are not exposed to network clients. These log messages are logged at the error log level and start with Requested method not exposed.
Rotating keys There are several reasons to rotate keys. One example is exposure or a compromised key. An easy way to rotate a key is to remove the existing keys and let the salt-master or salt-minion process generate new keys on restart.
Rotate a minion key Run the following on the Salt minion:
salt-call saltutil.regen_keys systemctl stop salt-minion
Run the following on the Salt master:
salt-key -d <minion-id>
Run the following on the Salt minion:
systemctl start salt-minion
salt-key -a <minion-id>
Rotate a master key Run the following on the Salt master:
systemctl stop salt-master rm <pki_dir>/master.{pem,pub} systemctl start salt-master
systemctl stop salt-minion rm <pki_dir>/minion_master.pub systemctl start salt-minion
Security disclosure policy email  security@saltstack.com
gpg key ID 4EA0793D
gpg key fingerprint 8ABE 4EFC F0F4 B24B FF2A  AF90 D570 F2D3 4EA0 793D
gpg public key:
-----BEGIN PGP PUBLIC KEY BLOCK-----
The SaltStack Security Team is available at security@saltstack.com for security-related bug reports or questions.
We request the disclosure of any security-related bugs or issues be reported non-publicly until such time as the issue can be resolved and a security-fix release can be prepared. At that time we will release the fix and make a public announcement with upgrade instructions and download locations.
Security response procedure SaltStack takes security and the trust of our customers and users very seriously. Our disclosure policy is intended to resolve security issues as quickly and safely as is possible.
1. A security report sent to security@saltstack.com is assigned to a team member. This person is the primary contact for questions and will coordinate the fix, release, and announcement.
2. The reported issue is reproduced and confirmed. A list of affected projects and releases is made.
3. Fixes are implemented for all affected projects and releases that are actively supported. Back-ports of the fix are made to any old releases that are actively supported.
4. Packagers are notified via the salt-packagers mailing list that an issue was reported and resolved, and that an announcement is incoming.
5. A new release is created and pushed to all affected repositories. The release documentation provides a full description of the issue, plus any upgrade instructions or other relevant details.
6. An announcement is made to the salt-users and salt-announce mailing lists. The announcement contains a description of the issue and a link to the full release documentation and download locations.
Receiving security announcements The following mailing lists, per the previous tasks identified in our response procedure, will receive security-relevant notifications:
• salt-packagers
• salt-users
• salt-announce
In addition to the mailing lists, SaltStack also provides the following resources:
• SaltStack Security Announcements landing page
• SaltStack Security RSS Feed
• SaltStack Community Slack Workspace
Salt Transport One of fundamental features of Salt is remote execution. Salt has two basic "channels" for communicating with minions. Each channel requires a client (minion) and a server (master) implementation to work within Salt. These pairs of channels will work together to implement the specific message passing required by the channel interface.
Pub Channel The pub channel, or publish channel, is how a master sends a job (payload) to a minion. This is a basic pub/sub paradigm, which has specific targeting semantics.  All data which goes across the publish system should be encrypted such that only members of the Salt cluster can decrypt the publishes.
Req Channel The req channel is how the minions send data to the master. This interface is primarily used for fetching files and returning job returns. The req channels have two basic interfaces when talking to the master. send is the basic method that guarantees the message is encrypted at least so that only minions attached to the same master can read it-- but no guarantee of minion-master confidentiality, whereas the crypted_transfer_decode_dictentry method does guarantee minion-master confidentiality.
Zeromq Transport NOTE: Zeromq is the current default transport within Salt
Zeromq is a messaging library with bindings into many languages. Zeromq implements a socket interface for message passing, with specific semantics for the socket type.
Pub Channel The pub channel is implemented using zeromq's pub/sub sockets. By default we don't use zeromq's filtering, which means that all publish jobs are sent to all minions and filtered minion side. Zeromq does have publisher side filtering which can be enabled in salt using zmq_filtering.
Req Channel The req channel is implemented using zeromq's req/rep sockets. These sockets enforce a send/recv pattern, which forces salt to serialize messages through these socket pairs. This means that although the interface is asynchronous on the minion we cannot send a second message until we have received the reply of the first message.
TCP Transport The tcp transport is an implementation of Salt's channels using raw tcp sockets.  Since this isn't using a pre-defined messaging library we will describe the wire protocol, message semantics, etc. in this document.
The tcp transport is enabled by changing the transport setting to tcp on each Salt minion and Salt master.
transport: tcp
WARNING: We currently recommend that when using Syndics that all Masters and Minions use the same transport. We're investigating a report of an error when using mixed transport types at very heavy loads.
Wire Protocol This implementation over TCP focuses on flexibility over absolute efficiency.  This means we are okay to spend a couple of bytes of wire space for flexibility in the future. That being said, the wire framing is quite efficient and looks like:
msgpack({'head': SOMEHEADER, 'body': SOMEBODY})
Since msgpack is an iterably parsed serialization, we can simply write the serialized payload to the wire. Within that payload we have two items "head" and "body".  Head contains header information (such as "message id"). The Body contains the actual message that we are sending. With this flexible wire protocol we can implement any message semantics that we'd like-- including multiplexed message passing on a single socket.
TLS Support New in version 2016.11.1.
The TCP transport allows for the master/minion communication to be optionally wrapped in a TLS connection. Enabling this is simple, the master and minion need to be using the tcp connection, then the ssl option is enabled. The ssl option is passed as a dict and corresponds to the options passed to the Python ssl.wrap_socket <https://docs.python.org/3/library/ssl.html#ssl.wrap_socket> function.
A simple setup looks like this, on the Salt Master add the ssl option to the master configuration file:
The minimal ssl option in the minion configuration file looks like this:
ssl: True # Versions below 2016.11.4: ssl: {}
Specific options can be sent to the minion also, as defined in the Python ssl.wrap_socket function.
NOTE: While setting the ssl_version is not required, we recommend it. Some older versions of python do not support the latest TLS protocol and if this is the case for your version of python we strongly recommend upgrading your version of Python.
Crypto The current implementation uses the same crypto as the zeromq transport.
Pub Channel For the pub channel we send messages without "message ids" which the remote end interprets as a one-way send.
NOTE: As of today we send all publishes to all minions and rely on minion-side filtering.
Req Channel For the req channel we send messages with a "message id". This "message id" allows us to multiplex messages across the socket.
Master Tops System In 0.10.4 the external_nodes system was upgraded to allow for modular subsystems to be used to generate the top file data for a highstate run on the master.
The old external_nodes option has been removed. The master tops system provides a pluggable and extendable replacement for it, allowing for multiple different subsystems to provide top file data.
Using the new master_tops option is simple:
master_tops: ext_nodes: cobbler-external-nodes
for Cobbler or:
master_tops: reclass: inventory_base_uri: /etc/reclass classes_uri: roles
for Reclass.
master_tops: varstack: /path/to/the/config/file/varstack.yaml
for Varstack.
It's also possible to create custom master_tops modules. Simply place them into salt://_tops in the Salt fileserver and use the saltutil.sync_tops runner to sync them. If this runner function is not available, they can manually be placed into extmods/tops, relative to the master cachedir (in most cases the full path will be /var/cache/salt/master/extmods/tops).
Custom tops modules are written like any other execution module, see the source for the two modules above for examples of fully functional ones. Below is a bare-bones example:
/etc/salt/master:
master_tops: customtop: True
customtop.py: (custom master_tops module)
import logging import sys
# Define the module's virtual name __virtualname__ = "customtop"
log = logging.getLogger(__name__)
def __virtual__(): return __virtualname__
def top(**kwargs): log.debug("Calling top in customtop") return {"base": ["test"]}
salt minion state.show_top should then display something like:
$ salt minion state.show_top
minion ---------- base: - test
NOTE: If a master_tops module returns top file data for a given minion, it will be added to the states configured in the top file. It will not replace it altogether. The 2018.3.0 release adds additional functionality allowing a minion to treat master_tops as the single source of truth, irrespective of the top file.
Returners By default the return values of the commands sent to the Salt minions are returned to the Salt master, however anything at all can be done with the results data.
By using a Salt returner, results data can be redirected to external data-stores for analysis and archival.
Returners pull their configuration values from the Salt minions. Returners are only configured once, which is generally at load time.
The returner interface allows the return data to be sent to any system that can receive data. This means that return data can be sent to a Redis server, a MongoDB server, a MySQL server, or any system.
SEE ALSO: Full list of builtin returners
Using Returners All Salt commands will return the command data back to the master. Specifying returners will ensure that the data is _also_ sent to the specified returner interfaces.
Specifying what returners to use is done when the command is invoked:
salt '*' test.version --return redis_return
This command will ensure that the redis_return returner is used.
It is also possible to specify multiple returners:
salt '*' test.version --return mongo_return,redis_return,cassandra_return
In this scenario all three returners will be called and the data from the test.version command will be sent out to the three named returners.
Writing a Returner Returners are Salt modules that allow the redirection of results data to targets other than the Salt Master.
Returners Are Easy To Write! Writing a Salt returner is straightforward.
A returner is a Python module containing at minimum a returner function. Other optional functions can be included to add support for master_job_cache, external-job-cache, and Event Returners.
returner The returner function must accept a single argument. The argument contains return data from the called minion function. If the minion function test.version is called, the value of the argument will be a dictionary. Run the following command from a Salt master to get a sample of the dictionary:
salt-call --local --metadata test.version --out=pprint
import redis import salt.utils.json
def returner(ret): """ Return information to a redis server """ # Get a redis connection serv = redis.Redis(host="redis-serv.example.com", port=6379, db="0") serv.sadd("%(id)s:jobs" % ret, ret["jid"]) serv.set("%(jid)s:%(id)s" % ret, salt.utils.json.dumps(ret["return"])) serv.sadd("jobs", ret["jid"]) serv.sadd(ret["jid"], ret["id"])
The above example of a returner set to send the data to a Redis server serializes the data as JSON and sets it in redis.
Using Custom Returner Modules Place custom returners in a _returners/ directory within the file_roots specified by the master config file.
Custom returners are distributed when any of the following are called:
• saltutil.sync_returners
• saltutil.sync_all
Any custom returners which have been synced to a minion that are named the same as one of Salt's default set of returners will take the place of the default returner with the same name.
Naming the Returner Note that a returner's default name is its filename (i.e. foo.py becomes returner foo), but that its name can be overridden by using a __virtual__ function. A good example of this can be found in the redis returner, which is named redis_return.py but is loaded as simply redis:
try: import redis
HAS_REDIS = True except ImportError: HAS_REDIS = False
__virtualname__ = "redis"
def __virtual__(): if not HAS_REDIS: return False return __virtualname__
Master Job Cache Support master_job_cache, external-job-cache, and Event Returners.  Salt's master_job_cache allows returners to be used as a pluggable replacement for the default_job_cache. In order to do so, a returner must implement the following functions:
NOTE: The code samples contained in this section were taken from the cassandra_cql returner.
prep_jid Ensures that job ids (jid) don't collide, unless passed_jid is provided.
nocache is an optional boolean that indicates if return data should be cached. passed_jid is a caller provided jid which should be returned unconditionally.
def prep_jid(nocache, passed_jid=None):  # pylint: disable=unused-argument """ Do any work necessary to prepare a JID, including sending a custom id """ return passed_jid if passed_jid is not None else salt.utils.jid.gen_jid()
save_load Save job information.  The jid is generated by prep_jid and should be considered a unique identifier for the job. The jid, for example, could be used as the primary/unique key in a database. The load is what is returned to a Salt master by a minion. minions is a list of minions that the job was run against. The following code example stores the load as a JSON string in the salt.jids table.
import salt.utils.json
def save_load(jid, load, minions=None): """ Save the load to the specified jid id """ query = """INSERT INTO salt.jids ( jid, load ) VALUES ( '{0}', '{1}' );""".format( jid, salt.utils.json.dumps(load) )
# cassandra_cql.cql_query may raise a CommandExecutionError try: __salt__["cassandra_cql.cql_query"](query) except CommandExecutionError: log.critical("Could not save load in jids table.") raise except Exception as e: log.critical("Unexpected error while inserting into jids: {0}".format(e)) raise
get_load must accept a job id (jid) and return the job load stored by save_load, or an empty dictionary when not found.
def get_load(jid): """ Return the load data that marks a specified jid """ query = """SELECT load FROM salt.jids WHERE jid = '{0}';""".format(jid)