neulab/tldr huggingface dataset:
Prepping the master to sign its public key For signing to work, both master and minion must have the signing and/or verification settings enabled. If the master signs the public key but the minion does not verify it, the minion will complain and exit. The same happens, when the master does not sign but the minion tries to verify.
The easiest way to have the master sign its public key is to set
After restarting the salt-master service, the master will automatically generate a new key-pair
master_sign.pem master_sign.pub
A custom name can be set for the signing key-pair by setting
master_sign_key_name: <name_without_suffix>
The master will then generate that key-pair upon restart and use it for creating the public keys signature attached to the auth-reply.
The computation is done for every auth-request of a minion. If many minions auth very often, it is advised to use conf_master:master_pubkey_signature and conf_master:master_use_pubkey_signature settings described below.
If multiple masters are in use and should sign their auth-replies, the signing key-pair master_sign.* has to be copied to each master. Otherwise a minion will fail to verify the masters public when connecting to a different master than it did initially. That is because the public keys signature was created with a different signing key-pair.
Prepping the minion to verify received public keys The minion must have the public key (and only that one!) available to be able to verify a signature it receives. That public key (defaults to master_sign.pub) must be copied from the master to the minions pki-directory.
/etc/salt/pki/minion/master_sign.pub
IMPORTANT: DO NOT COPY THE master_sign.pem FILE. IT MUST STAY ON THE MASTER AND ONLY THERE!
When that is done, enable the signature checking in the minions configuration
and restart the minion. For the first try, the minion should be run in manual debug mode.
salt-minion -l debug
Upon connecting to the master, the following lines should appear on the output:
[DEBUG   ] Attempting to authenticate with the Salt Master at 172.16.0.10 [DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem [DEBUG   ] salt.crypt.verify_signature: Loading public key [DEBUG   ] salt.crypt.verify_signature: Verifying signature [DEBUG   ] Successfully verified signature of master public key with verification public key master_sign.pub [INFO    ] Received signed and verified master pubkey from master 172.16.0.10 [DEBUG   ] Decrypting the current master AES key
If the signature verification fails, something went wrong and it will look like this
[DEBUG   ] Attempting to authenticate with the Salt Master at 172.16.0.10 [DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem [DEBUG   ] salt.crypt.verify_signature: Loading public key [DEBUG   ] salt.crypt.verify_signature: Verifying signature [DEBUG   ] Failed to verify signature of public key [CRITICAL] The Salt Master server's public key did not authenticate!
In a case like this, it should be checked, that the verification pubkey (master_sign.pub) on the minion is the same as the one on the master.
Once the verification is successful, the minion can be started in daemon mode again.
For the paranoid among us, its also possible to verify the publication whenever it is received from the master. That is, for every single auth-attempt which can be quite frequent. For example just the start of the minion will force the signature to be checked 6 times for various things like auth, mine, highstate, etc.
If that is desired, enable the setting
Multiple Masters For A Minion Configuring multiple masters on a minion is done by specifying two settings:
• a list of masters addresses
• what type of master is defined
master: - 172.16.0.10 - 172.16.0.11 - 172.16.0.12
This tells the minion that all the master above are available for it to connect to. When started with this configuration, it will try the master in the order they are defined. To randomize that order, set
The master-list will then be shuffled before the first connection attempt.
The first master that accepts the minion, is used by the minion. If the master does not yet know the minion, that counts as accepted and the minion stays on that master.
For the minion to be able to detect if its still connected to its current master enable the check for it
master_alive_interval: <seconds>
If the loss of the connection is detected, the minion will temporarily remove the failed master from the list and try one of the other masters defined (again shuffled if that is enabled).
Testing the setup At least two running masters are needed to test the failover setup.
Both masters should be running and the minion should be running on the command line in debug mode
The minion will connect to the first master from its master list
A test.version on the master the minion is currently connected to should be run to test connectivity.
If successful, that master should be turned off. A firewall-rule denying the minions packets will also do the trick.
Depending on the configured conf_minion:master_alive_interval, the minion will notice the loss of the connection and log it to its logfile.
[INFO    ] Connection to master 172.16.0.10 lost [INFO    ] Trying to tune in to next master from master-list
The minion will then remove the current master from the list and try connecting to the next master
[INFO    ] Removing possibly failed master 172.16.0.10 from list of masters [WARNING ] Master ip address changed from 172.16.0.10 to 172.16.0.11 [DEBUG   ] Attempting to authenticate with the Salt Master at 172.16.0.11
If everything is configured correctly, the new masters public key will be verified successfully
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem [DEBUG   ] salt.crypt.verify_signature: Loading public key [DEBUG   ] salt.crypt.verify_signature: Verifying signature [DEBUG   ] Successfully verified signature of master public key with verification public key master_sign.pub
the authentication with the new master is successful
[INFO    ] Received signed and verified master pubkey from master 172.16.0.11 [DEBUG   ] Decrypting the current master AES key [DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem [INFO    ] Authentication with master successful!
and the minion can be pinged again from its new master.
Performance Tuning With the setup described above, the master computes a signature for every auth-request of a minion. With many minions and many auth-requests, that can chew up quite a bit of CPU-Power.
To avoid that, the master can use a pre-created signature of its public-key.  The signature is saved as a base64 encoded string which the master reads once when starting and attaches only that string to auth-replies.
Enabling this also gives paranoid users the possibility, to have the signing key-pair on a different system than the actual salt-master and create the public keys signature there. Probably on a system with more restrictive firewall rules, without internet access, less users, etc.
That signature can be created with
salt-key --gen-signature
This will create a default signature file in the master pki-directory
/etc/salt/pki/master/master_pubkey_signature
It is a simple text-file with the binary-signature converted to base64.
If no signing-pair is present yet, this will auto-create the signing pair and the signature file in one call
salt-key --gen-signature --auto-create
Telling the master to use the pre-created signature is done with
That requires the file 'master_pubkey_signature' to be present in the masters pki-directory with the correct signature.
If the signature file is named differently, its name can be set with
With many masters and many public-keys (default and signing), it is advised to use the salt-masters hostname for the signature-files name. Signatures can be easily confused because they do not provide any information about the key the signature was created from.
Verifying that everything works is done the same way as above.
How the signing and verification works The default key-pair of the salt-master is
/etc/salt/pki/master/master.pem /etc/salt/pki/master/master.pub
To be able to create a signature of a message (in this case a public-key), another key-pair has to be added to the setup. Its default name is:
The combination of the master.* and master_sign.* key-pairs give the possibility of generating signatures. The signature of a given message is unique and can be verified, if the public-key of the signing-key-pair is available to the recipient (the minion).
The signature of the masters public-key in master.pub is computed with
master_sign.pem master.pub M2Crypto.EVP.sign_update()
This results in a binary signature which is converted to base64 and attached to the auth-reply send to the minion.
With the signing-pairs public-key available to the minion, the attached signature can be verified with
master_sign.pub master.pub M2Cryptos EVP.verify_update().
When running multiple masters, either the signing key-pair has to be present on all of them, or the master_pubkey_signature has to be pre-computed for each master individually (because they all have different public-keys). DO NOT PUT THE SAME master.pub ON ALL MASTERS FOR EASE OF USE.
Packaging External Modules for Salt External Modules Setuptools Entry-Points Support The salt loader was enhanced to look for external modules by looking at the salt.loader entry-point: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
pkg_resources should be installed, which is normally included in setuptools. https://setuptools.readthedocs.io/en/latest/pkg_resources.html
The package which has custom engines, minion modules, outputters, etc, should require setuptools and should define the following entry points in its setup function:
setup( name=THE_NAME, version=THE_VERSION, description=THE_DESCRIPTION, author=THE_AUTHOR_NAME, author_email=THE_AUTHOR_EMAIL, url=" ... ", packages=find_packages(), entry_points=""" [salt.loader] engines_dirs = <package>.<loader-module>:engines_dirs fileserver_dirs = <package>.<loader-module>:fileserver_dirs pillar_dirs = <package>.<loader-module>:pillar_dirs returner_dirs = <package>.<loader-module>:returner_dirs roster_dirs = <package>.<loader-module>:roster_dirs """, )
The above setup script example mentions a loader module. here's an example of how <package>/<loader-module>.py it should look:
# Import python libs import os
PKG_DIR = os.path.abspath(os.path.dirname(__file__))
def engines_dirs(): """ yield one path per parent directory of where engines can be found """ yield os.path.join(PKG_DIR, "engines_1") yield os.path.join(PKG_DIR, "engines_2")
def fileserver_dirs(): """ yield one path per parent directory of where fileserver modules can be found """ yield os.path.join(PKG_DIR, "fileserver")
def pillar_dirs(): """ yield one path per parent directory of where external pillar modules can be found """ yield os.path.join(PKG_DIR, "pillar")
def returner_dirs(): """ yield one path per parent directory of where returner modules can be found """ yield os.path.join(PKG_DIR, "returners")
def roster_dirs(): """ yield one path per parent directory of where roster modules can be found """ yield os.path.join(PKG_DIR, "roster")
How Do I Use Salt States? Simplicity, Simplicity, Simplicity
Many of the most powerful and useful engineering solutions are founded on simple principles. Salt States strive to do just that: K.I.S.S. (Keep It Stupidly Simple)
The core of the Salt State system is the SLS, or SaLt State file. The SLS is a representation of the state in which a system should be in, and is set up to contain this data in a simple format.  This is often called configuration management.
NOTE: This is just the beginning of using states, make sure to read up on pillar Pillar next.
It is All Just Data Before delving into the particulars, it will help to understand that the SLS file is just a data structure under the hood. While understanding that the SLS is just a data structure isn't critical for understanding and making use of Salt States, it should help bolster knowledge of where the real power is.
SLS files are therefore, in reality, just dictionaries, lists, strings, and numbers.  By using this approach Salt can be much more flexible. As one writes more state files, it becomes clearer exactly what is being written. The result is a system that is easy to understand, yet grows with the needs of the admin or developer.
The Top File The example SLS files in the below sections can be assigned to hosts using a file called top.sls. This file is described in-depth here.
Default Data - YAML By default Salt represents the SLS data in what is one of the simplest serialization formats available - YAML.
A typical SLS file will often look like this in YAML:
NOTE: These demos use some generic service and package names, different distributions often use different names for packages and services. For instance apache should be replaced with httpd on a Red Hat system. Salt uses the name of the init script, systemd name, upstart name etc. based on what the underlying service management for the platform. To get a list of the available service names on a platform execute the service.get_all salt function.
Information on how to make states work with multiple distributions is later in the tutorial.
apache: pkg.installed: [] service.running: - require: - pkg: apache
This SLS data will ensure that the package named apache is installed, and that the apache service is running. The components can be explained in a simple way.
The first line is the ID for a set of data, and it is called the ID Declaration. This ID sets the name of the thing that needs to be manipulated.
The second and third lines contain the state module function to be run, in the format <state_module>.<function>. The pkg.installed state module function ensures that a software package is installed via the system's native package manager. The service.running state module function ensures that a given system daemon is running.
Finally, on line four, is the word require. This is called a Requisite Statement, and it makes sure that the Apache service is only started after a successful installation of the apache package.
Adding Configs and Users When setting up a service like an Apache web server, many more components may need to be added. The Apache configuration file will most likely be managed, and a user and group may need to be set up.
apache: pkg.installed: [] service.running: - watch: - pkg: apache - file: /etc/httpd/conf/httpd.conf - user: apache user.present: - uid: 87 - gid: 87 - home: /var/www/html - shell: /bin/nologin - require: - group: apache group.present: - gid: 87 - require: - pkg: apache
/etc/httpd/conf/httpd.conf: file.managed: - source: salt://apache/httpd.conf - user: root - group: root - mode: 644
This SLS data greatly extends the first example, and includes a config file, a user, a group and new requisite statement: watch.
Adding more states is easy, since the new user and group states are under the Apache ID, the user and group will be the Apache user and group. The require statements will make sure that the user will only be made after the group, and that the group will be made only after the Apache package is installed.
Next, the require statement under service was changed to watch, and is now watching 3 states instead of just one. The watch statement does the same thing as require, making sure that the other states run before running the state with a watch, but it adds an extra component. The watch statement will run the state's watcher function for any changes to the watched states.  So if the package was updated, the config file changed, or the user uid modified, then the service state's watcher will be run. The service state's watcher just restarts the service, so in this case, a change in the config file will also trigger a restart of the respective service.
Moving Beyond a Single SLS When setting up Salt States in a scalable manner, more than one SLS will need to be used. The above examples were in a single SLS file, but two or more SLS files can be combined to build out a State Tree. The above example also references a file with a strange source - salt://apache/httpd.conf. That file will need to be available as well.
The SLS files are laid out in a directory structure on the Salt master; an SLS is just a file and files to download are just files.
The Apache example would be laid out in the root of the Salt file server like this:
apache/init.sls apache/httpd.conf
So the httpd.conf is just a file in the apache directory, and is referenced directly.
Do not use dots in SLS file names or their directories
The initial implementation of top.sls and include-declaration followed the python import model where a slash is represented as a period.  This means that a SLS file with a period in the name ( besides the suffix period) can not be referenced.  For example, webserver_1.0.sls is not referenceable because webserver_1.0 would refer to the directory/file webserver_1/0.sls
The same applies for any subdirectories, this is especially 'tricky' when git repos are created.  Another command that typically can't render its output is `state.show_sls` of a file in a path that contains a dot.
But when using more than one single SLS file, more components can be added to the toolkit. Consider this SSH example:
ssh/init.sls:
openssh-client: pkg.installed
/etc/ssh/ssh_config: file.managed: - user: root - group: root - mode: 644 - source: salt://ssh/ssh_config - require: - pkg: openssh-client
ssh/server.sls:
include: - ssh
openssh-server: pkg.installed
sshd: service.running: - require: - pkg: openssh-client - pkg: openssh-server - file: /etc/ssh/banner - file: /etc/ssh/sshd_config
/etc/ssh/sshd_config: file.managed: - user: root - group: root - mode: 644 - source: salt://ssh/sshd_config - require: - pkg: openssh-server
/etc/ssh/banner: file: - managed - user: root - group: root - mode: 644 - source: salt://ssh/banner - require: - pkg: openssh-server
NOTE: Notice that we use two similar ways of denoting that a file is managed by Salt. In the /etc/ssh/sshd_config state section above, we use the file.managed state declaration whereas with the /etc/ssh/banner state section, we use the file state declaration and add a managed attribute to that state declaration. Both ways produce an identical result; the first way -- using file.managed -- is merely a shortcut.
Now our State Tree looks like this:
apache/init.sls apache/httpd.conf ssh/init.sls ssh/server.sls ssh/banner ssh/ssh_config ssh/sshd_config
This example now introduces the include statement. The include statement includes another SLS file so that components found in it can be required, watched or as will soon be demonstrated - extended.
The include statement allows for states to be cross linked. When an SLS has an include statement it is literally extended to include the contents of the included SLS files.
Note that some of the SLS files are called init.sls, while others are not. More info on what this means can be found in the States Tutorial.
Extending Included SLS Data Sometimes SLS data needs to be extended. Perhaps the apache service needs to watch additional resources, or under certain circumstances a different file needs to be placed.
In these examples, the first will add a custom banner to ssh and the second will add more watchers to apache to include mod_python.
ssh/custom-server.sls:
include: - ssh.server
extend: /etc/ssh/banner: file: - source: salt://ssh/custom-banner
python/mod_python.sls:
include: - apache
extend: apache: service: - watch: - pkg: mod_python
mod_python: pkg.installed
The custom-server.sls file uses the extend statement to overwrite where the banner is being downloaded from, and therefore changing what file is being used to configure the banner.
In the new mod_python SLS the mod_python package is added, but more importantly the apache service was extended to also watch the mod_python package.
Using extend with require or watch
The extend statement works differently for require or watch. It appends to, rather than replacing the requisite component.
Understanding the Render System Since SLS data is simply that (data), it does not need to be represented with YAML. Salt defaults to YAML because it is very straightforward and easy to learn and use. But the SLS files can be rendered from almost any imaginable medium, so long as a renderer module is provided.
The default rendering system is the jinja|yaml renderer. The jinja|yaml renderer will first pass the template through the Jinja2 templating system, and then through the YAML parser. The benefit here is that full programming constructs are available when creating SLS files.
Other renderers available are yaml_mako and yaml_wempy which each use the Mako or Wempy templating system respectively rather than the jinja templating system, and more notably, the pure Python or py, pydsl & pyobjects renderers.  The py renderer allows for SLS files to be written in pure Python, allowing for the utmost level of flexibility and power when preparing SLS data; while the pydsl renderer provides a flexible, domain-specific language for authoring SLS data in Python; and the pyobjects renderer gives you a "Pythonic" interface to building state data.
NOTE: The templating engines described above aren't just available in SLS files.  They can also be used in file.managed states, making file management much more dynamic and flexible. Some examples for using templates in managed files can be found in the documentation for the file state, as well as the MooseFS example below.
Getting to Know the Default - jinja|yaml The default renderer - jinja|yaml, allows for use of the jinja templating system. A guide to the Jinja templating system can be found here: https://jinja.palletsprojects.com/en/2.11.x/
When working with renderers a few very useful bits of data are passed in. In the case of templating engine based renderers, three critical components are available, salt, grains, and pillar. The salt object allows for any Salt function to be called from within the template, and grains allows for the Grains to be accessed from within the template. A few examples:
apache/init.sls:
apache: pkg.installed: {% if grains['os'] == 'RedHat'%} - name: httpd {% endif %} service.running: {% if grains['os'] == 'RedHat'%} - name: httpd {% endif %} - watch: - pkg: apache - file: /etc/httpd/conf/httpd.conf - user: apache user.present: - uid: 87 - gid: 87 - home: /var/www/html - shell: /bin/nologin - require: - group: apache group.present: - gid: 87 - require: - pkg: apache
This example is simple. If the os grain states that the operating system is Red Hat, then the name of the Apache package and service needs to be httpd.
A more aggressive way to use Jinja can be found here, in a module to set up a MooseFS distributed filesystem chunkserver:
moosefs/chunk.sls:
include: - moosefs
{% for mnt in salt['cmd.run']('ls /dev/data/moose*').split() %} /mnt/moose{{ mnt[-1] }}: mount.mounted: - device: {{ mnt }} - fstype: xfs - mkmnt: True file.directory: - user: mfs - group: mfs - require: - user: mfs - group: mfs {% endfor %}
/etc/mfshdd.cfg: file.managed: - source: salt://moosefs/mfshdd.cfg - user: root - group: root - mode: 644 - template: jinja - require: - pkg: mfs-chunkserver
/etc/mfschunkserver.cfg: file.managed: - source: salt://moosefs/mfschunkserver.cfg - user: root - group: root - mode: 644 - template: jinja - require: - pkg: mfs-chunkserver
mfs-chunkserver: pkg.installed: [] mfschunkserver: service.running: - require: {% for mnt in salt['cmd.run']('ls /dev/data/moose*') %} - mount: /mnt/moose{{ mnt[-1] }} - file: /mnt/moose{{ mnt[-1] }} {% endfor %} - file: /etc/mfschunkserver.cfg - file: /etc/mfshdd.cfg - file: /var/lib/mfs
This example shows much more of the available power of Jinja.  Multiple for loops are used to dynamically detect available hard drives and set them up to be mounted, and the salt object is used multiple times to call shell commands to gather data.
Introducing the Python, PyDSL, and the Pyobjects Renderers Sometimes the chosen default renderer might not have enough logical power to accomplish the needed task. When this happens, the Python renderer can be used. Normally a YAML renderer should be used for the majority of SLS files, but an SLS file set to use another renderer can be easily added to the tree.
This example shows a very basic Python SLS file:
python/django.sls:
def run(): """ Install the django package """ return {"include": ["python"], "django": {"pkg": ["installed"]}}
This is a very simple example; the first line has an SLS shebang that tells Salt to not use the default renderer, but to use the py renderer. Then the run function is defined, the return value from the run function must be a Salt friendly data structure, or better known as a Salt HighState data structure.
Alternatively, using the pydsl renderer, the above example can be written more succinctly as:
include("python", delayed=True) state("django").pkg.installed()
The pyobjects renderer provides an "Pythonic" object based approach for building the state data.  The above example could be written as:
include("python") Pkg.installed("django")
These Python examples would look like this if they were written in YAML:
django: pkg.installed
This example clearly illustrates that; one, using the YAML renderer by default is a wise decision and two, unbridled power can be obtained where needed by using a pure Python SLS.
Running and Debugging Salt States Once the rules in an SLS are ready, they should be tested to ensure they work properly. To invoke these rules, simply execute salt '*' state.apply on the command line. If you get back only hostnames with a : after, but no return, chances are there is a problem with one or more of the sls files. On the minion, use the salt-call command to examine the output for errors:
salt-call state.apply -l debug
This should help troubleshoot the issue. The minion can also be started in the foreground in debug mode by running salt-minion -l debug.
Next Reading With an understanding of states, the next recommendation is to become familiar with Salt's pillar interface: Pillar Walkthrough
States tutorial, part 1 - Basic Usage The purpose of this tutorial is to demonstrate how quickly you can configure a system to be managed by Salt States. For detailed information about the state system please refer to the full states reference.
This tutorial will walk you through using Salt to configure a minion to run the Apache HTTP server and to ensure the server is running.
Before continuing make sure you have a working Salt installation by following the installation and the configuration instructions.
Setting up the Salt State Tree States are stored in text files on the master and transferred to the minions on demand via the master's File Server. The collection of state files make up the State Tree.
To start using a central state system in Salt, the Salt File Server must first be set up. Edit the master config file (file_roots) and uncomment the following lines:
NOTE: If you are deploying on FreeBSD via ports, the file_roots path defaults to /usr/local/etc/salt/states.
Restart the Salt master in order to pick up this change:
pkill salt-master salt-master -d
Preparing the Top File On the master, in the directory uncommented in the previous step, (/srv/salt by default), create a new file called top.sls and add the following:
The top file is separated into environments (discussed later). The default environment is base. Under the base environment a collection of minion matches is defined; for now simply specify all hosts (*).
Targeting minions
The expressions can use any of the targeting mechanisms used by Salt — minions can be matched by glob, PCRE regular expression, or by grains. For example:
base: 'os:Fedora': - match: grain - webserver
Create an sls file In the same directory as the top file, create a file named webserver.sls, containing the following:
apache:                 # ID declaration pkg:                  # state declaration - installed         # function declaration
The first line, called the id-declaration, is an arbitrary identifier. In this case it defines the name of the package to be installed.
NOTE: The package name for the Apache httpd web server may differ depending on OS or distro — for example, on Fedora it is httpd but on Debian/Ubuntu it is apache2.
The second line, called the state-declaration, defines which of the Salt States we are using. In this example, we are using the pkg state to ensure that a given package is installed.
The third line, called the function-declaration, defines which function in the pkg state module to call.
States sls files can be written in many formats. Salt requires only a simple data structure and is not concerned with how that data structure is built. Templating languages and DSLs are a dime-a-dozen and everyone has a favorite.
Building the expected data structure is the job of Salt renderers and they are dead-simple to write.
In this tutorial we will be using YAML in Jinja2 templates, which is the default format. The default can be changed by editing renderer in the master configuration file.
Install the package Next, let's run the state we created. Open a terminal on the master and run:
salt '*' state.apply
Our master is instructing all targeted minions to run state.apply. When this function is executed without any SLS targets, a minion will download the top file and attempt to match the expressions within it. When the minion does match an expression the modules listed for it will be downloaded, compiled, and executed.
NOTE: This action is referred to as a "highstate", and can be run using the state.highstate function.  However, to make the usage easier to understand ("highstate" is not necessarily an intuitive name), a state.apply function was added in version 2015.5.0, which when invoked without any SLS names will trigger a highstate.  state.highstate still exists and can be used, but the documentation (as can be seen above) has been updated to reference state.apply, so keep the following in mind as you read the documentation:
• state.apply invoked without any SLS names will run state.highstate
• state.apply invoked with SLS names will run state.sls
Once completed, the minion will report back with a summary of all actions taken and all changes made.
WARNING: If you have created custom grain modules, they will not be available in the top file until after the first highstate. To make custom grains available on a minion's first highstate, it is recommended to use this example to ensure that the custom grains are synced when the minion starts.
SLS File Namespace
Note that in the example above, the SLS file webserver.sls was referred to simply as webserver. The namespace for SLS files when referenced in top.sls or an include-declaration follows a few simple rules:
1. The .sls is discarded (i.e. webserver.sls becomes webserver).
Subdirectories can be used for better organization.
a. Each subdirectory is represented with a dot (following the Python import model) in Salt states and on the command line .  webserver/dev.sls on the filesystem is referred to as webserver.dev in Salt
b. Because slashes are represented as dots, SLS files can not contain dots in the name (other than the dot for the SLS suffix).  The SLS file webserver_1.0.sls can not be matched, and webserver_1.0 would match the directory/file webserver_1/0.sls
3. A file called init.sls in a subdirectory is referred to by the path of the directory. So, webserver/init.sls is referred to as webserver.
4. If both webserver.sls and webserver/init.sls happen to exist, webserver/init.sls will be ignored and webserver.sls will be the file referred to as webserver.
Troubleshooting Salt
If the expected output isn't seen, the following tips can help to narrow down the problem.
Turn up logging Salt can be quite chatty when you change the logging setting to debug:
Run the minion in the foreground By not starting the minion in daemon mode (-d) one can view any output from the minion as it works:
Increase the default timeout value when running salt. For example, to change the default timeout to 60 seconds:
salt -t 60
For best results, combine all three:
salt-minion -l debug        # On the minion salt '*' state.apply -t 60  # On the master
Next steps This tutorial focused on getting a simple Salt States configuration working.  Part 2 will build on this example to cover more advanced sls syntax and will explore more of the states that ship with Salt.
States tutorial, part 2 - More Complex States, Requisites NOTE: This tutorial builds on topics covered in part 1. It is recommended that you begin there.
In the last part of the Salt States tutorial we covered the basics of installing a package. We will now modify our webserver.sls file to have requirements, and use even more Salt States.
Call multiple States You can specify multiple state-declaration under an id-declaration. For example, a quick modification to our webserver.sls to also start Apache if it is not running:
Try stopping Apache before running state.apply once again and observe the output.
NOTE: For those running RedhatOS derivatives (Centos, AWS), you will want to specify the service name to be httpd. More on state service here, service state.  With the example above, just add "- name: httpd" above the require line and with the same spacing.
Require other states We now have a working installation of Apache so let's add an HTML file to customize our website. It isn't exactly useful to have a website without a webserver so we don't want Salt to install our HTML file until Apache is installed and running. Include the following at the bottom of your webserver/init.sls file:
/var/www/index.html:                        # ID declaration file:                                     # state declaration - managed                               # function - source: salt://webserver/index.html   # function arg - require:                              # requisite declaration - pkg: apache                         # requisite reference
line 7 is the id-declaration. In this example it is the location we want to install our custom HTML file. (Note: the default location that Apache serves may differ from the above on your OS or distro. /srv/www could also be a likely place to look.)
Line 8 the state-declaration. This example uses the Salt file state.
Line 9 is the function-declaration. The managed function will download a file from the master and install it in the location specified.
Line 10 is a function-arg-declaration which, in this example, passes the source argument to the managed function.
Line 11 is a requisite-declaration.
Line 12 is a requisite-reference which refers to a state and an ID.  In this example, it is referring to the ID declaration from our example in part 1. This declaration tells Salt not to install the HTML file until Apache is installed.
Next, create the index.html file and save it in the webserver directory:
<!DOCTYPE html> <html> <head><title>Salt rocks</title></head> <body> <h1>This file brought to you by Salt</h1> </body> </html>
Last, call state.apply again and the minion will fetch and execute the highstate as well as our HTML file from the master using Salt's File Server:
Verify that Apache is now serving your custom HTML.
require vs. watch
There are two requisite-declaration, “require”, and “watch”. Not every state supports “watch”. The service state does support “watch” and will restart a service based on the watch condition.
For example, if you use Salt to install an Apache virtual host configuration file and want to restart Apache whenever that file is changed you could modify our Apache example from earlier as follows:
/etc/httpd/extra/httpd-vhosts.conf: file.managed: - source: salt://webserver/httpd-vhosts.conf
apache: pkg.installed: [] service.running: - watch: - file: /etc/httpd/extra/httpd-vhosts.conf - require: - pkg: apache
If the pkg and service names differ on your OS or distro of choice you can specify each one separately using a name-declaration which explained in Part 3.
Next steps In part 3 we will discuss how to use includes, extends, and templating to make a more complete State Tree configuration.
States tutorial, part 3 - Templating, Includes, Extends NOTE: This tutorial builds on topics covered in part 1 and part 2. It is recommended that you begin there.
This part of the tutorial will cover more advanced templating and configuration techniques for sls files.
Templating SLS modules SLS modules may require programming logic or inline execution. This is accomplished with module templating. The default module templating system used is Jinja2  and may be configured by changing the renderer value in the master config.
All states are passed through a templating system when they are initially read.  To make use of the templating system, simply add some templating markup.  An example of an sls module with templating markup may look like this:
{% for usr in ['moe','larry','curly'] %} {{ usr }}: user.present {% endfor %}
This templated sls file once generated will look like this:
moe: user.present larry: user.present curly: user.present
# Comments in yaml start with a hash symbol. # Since jinja rendering occurs before yaml parsing, if you want to include jinja # in the comments you may need to escape them using 'jinja' comments to prevent # jinja from trying to render something which is not well-defined jinja. # e.g. # {# iterate over the Three Stooges using a {% for %}..{% endfor %} loop # with the iterator variable {{ usr }} becoming the state ID. #} {% for usr in 'moe','larry','curly' %} {{ usr }}: group: - present user: - present - gid_from_name: True - require: - group: {{ usr }} {% endfor %}
Using Grains in SLS modules Often times a state will need to behave differently on different systems. Salt grains objects are made available in the template context. The grains can be used from within sls modules:
apache: pkg.installed: {% if grains['os'] == 'RedHat' %} - name: httpd {% elif grains['os'] == 'Ubuntu' %} - name: apache2 {% endif %}
Using Environment Variables in SLS modules You can use salt['environ.get']('VARNAME') to use an environment variable in a Salt state.
MYENVVAR="world" salt-call state.template test.sls
Create a file with contents from an environment variable: file.managed: - name: /tmp/hello - contents: {{ salt['environ.get']('MYENVVAR') }}
Error checking:
{% set myenvvar = salt['environ.get']('MYENVVAR') %} {% if myenvvar %}
{% else %}
Fail - no environment passed in: test.fail_without_changes
Calling Salt modules from templates All of the Salt modules loaded by the minion are available within the templating system. This allows data to be gathered in real time on the target system. It also allows for shell commands to be run easily from within the sls modules.
The Salt module functions are also made available in the template context as salt:
The following example illustrates calling the group_to_gid function in the file execution module with a single positional argument called some_group_that_exists.
moe: user.present: - gid: {{ salt['file.group_to_gid']('some_group_that_exists') }}
One way to think about this might be that the gid key is being assigned a value equivalent to the following python pseudo-code:
import salt.modules.file
file.group_to_gid("some_group_that_exists")
Note that for the above example to work, some_group_that_exists must exist before the state file is processed by the templating engine.
Below is an example that uses the network.hw_addr function to retrieve the MAC address for eth0:
salt["network.hw_addr"]("eth0")
To examine the possible arguments to each execution module function, one can examine the module reference documentation:
Advanced SLS module syntax Lastly, we will cover some incredibly useful techniques for more complex State trees.
Include declaration A previous example showed how to spread a Salt tree across several files. Similarly, requisites span multiple files by using an include-declaration. For example:
python/python-libs.sls:
python-dateutil: pkg.installed
include: - python.python-libs
django: pkg.installed: - require: - pkg: python-dateutil
Extend declaration You can modify previous declarations by using an extend-declaration. For example the following modifies the Apache tree to also restart Apache when the vhosts file is changed:
apache/apache.sls:
apache: pkg.installed
apache/mywebsite.sls:
include: - apache.apache
extend: apache: service: - running - watch: - file: /etc/httpd/extra/httpd-vhosts.conf
/etc/httpd/extra/httpd-vhosts.conf: file.managed: - source: salt://apache/httpd-vhosts.conf
Name declaration You can override the id-declaration by using a name-declaration.  For example, the previous example is a bit more maintainable if rewritten as follows:
extend: apache: service: - running - watch: - file: mywebsite
mywebsite: file.managed: - name: /etc/httpd/extra/httpd-vhosts.conf - source: salt://apache/httpd-vhosts.conf
Names declaration Even more powerful is using a names-declaration to override the id-declaration for multiple states at once. This often can remove the need for looping in a template. For example, the first example in this tutorial can be rewritten without the loop:
stooges: user.present: - names: - moe - larry - curly
Next steps In part 4 we will discuss how to use salt's file_roots to set up a workflow in which states can be "promoted" from dev, to QA, to production.
States tutorial, part 4 NOTE: This tutorial builds on topics covered in part 1, part 2, and part 3. It is recommended that you begin there.
This part of the tutorial will show how to use salt's file_roots to set up a workflow in which states can be "promoted" from dev, to QA, to production.
Salt fileserver path inheritance Salt's fileserver allows for more than one root directory per environment, like in the below example, which uses both a local directory and a secondary location shared to the salt master via NFS:
# In the master config file (/etc/salt/master) file_roots: base: - /srv/salt - /mnt/salt-nfs/base
Salt's fileserver collapses the list of root directories into a single virtual environment containing all files from each root. If the same file exists at the same relative path in more than one root, then the top-most match "wins". For example, if /srv/salt/foo.txt and /mnt/salt-nfs/base/foo.txt both exist, then salt://foo.txt will point to /srv/salt/foo.txt.
NOTE: When using multiple fileserver backends, the order in which they are listed in the fileserver_backend parameter also matters. If both roots and git backends contain a file with the same relative path, and roots appears before git in the fileserver_backend list, then the file in roots will "win", and the file in gitfs will be ignored.
A more thorough explanation of how Salt's modular fileserver works can be found here. We recommend reading this.
Environment configuration Configure a multiple-environment setup like so:
Given the path inheritance described above, files within /srv/salt/prod would be available in all environments. Files within /srv/salt/qa would be available in both qa, and dev. Finally, the files within /srv/salt/dev would only be available within the dev environment.
Based on the order in which the roots are defined, new files/states can be placed within /srv/salt/dev, and pushed out to the dev hosts for testing.
Those files/states can then be moved to the same relative path within /srv/salt/qa, and they are now available only in the dev and qa environments, allowing them to be pushed to QA hosts and tested.
Finally, if moved to the same relative path within /srv/salt/prod, the files are now available in all three environments.
Requesting files from specific fileserver environments See here for documentation on how to request files from specific environments.
Practical Example As an example, consider a simple website, installed to /var/www/foobarcom.  Below is a top.sls that can be used to deploy the website:
/srv/salt/prod/top.sls:
base: 'web*prod*': - webserver.foobarcom qa: 'web*qa*': - webserver.foobarcom dev: 'web*dev*': - webserver.foobarcom
Using pillar, roles can be assigned to the hosts:
base: 'web*prod*': - webserver.prod 'web*qa*': - webserver.qa 'web*dev*': - webserver.dev
/srv/pillar/webserver/prod.sls:
webserver_role: prod
/srv/pillar/webserver/qa.sls:
webserver_role: qa
/srv/pillar/webserver/dev.sls:
webserver_role: dev
And finally, the SLS to deploy the website:
/srv/salt/prod/webserver/foobarcom.sls:
{% if pillar.get('webserver_role', '') %} /var/www/foobarcom: file.recurse: - source: salt://webserver/src/foobarcom - env: {{ pillar['webserver_role'] }} - user: www - group: www - dir_mode: 755 - file_mode: 644 {% endif %}
Given the above SLS, the source for the website should initially be placed in /srv/salt/dev/webserver/src/foobarcom.
First, let's deploy to dev. Given the configuration in the top file, this can be done using state.apply:
salt --pillar 'webserver_role:dev' state.apply
However, in the event that it is not desirable to apply all states configured in the top file (which could be likely in more complex setups), it is possible to apply just the states for the foobarcom website, by invoking state.apply with the desired SLS target as an argument:
salt --pillar 'webserver_role:dev' state.apply webserver.foobarcom
Once the site has been tested in dev, then the files can be moved from /srv/salt/dev/webserver/src/foobarcom to /srv/salt/qa/webserver/src/foobarcom, and deployed using the following:
salt --pillar 'webserver_role:qa' state.apply webserver.foobarcom
Finally, once the site has been tested in qa, then the files can be moved from /srv/salt/qa/webserver/src/foobarcom to /srv/salt/prod/webserver/src/foobarcom, and deployed using the following:
salt --pillar 'webserver_role:prod' state.apply webserver.foobarcom
Thanks to Salt's fileserver inheritance, even though the files have been moved to within /srv/salt/prod, they are still available from the same salt:// URI in both the qa and dev environments.
Continue Learning The best way to continue learning about Salt States is to read through the reference documentation and to look through examples of existing state trees. Many pre-configured state trees can be found on GitHub in the saltstack-formulas collection of repositories.
If you have any questions, suggestions, or just want to chat with other people who are using Salt, we have a very active community and we'd love to hear from you.
In addition, by continuing to the Orchestrate Runner docs, you can learn about the powerful orchestration of which Salt is capable.
States Tutorial, Part 5 - Orchestration with Salt This was moved to Orchestrate Runner.
Syslog-ng usage Overview Syslog_ng state module is for generating syslog-ng configurations. You can do the following things:
• generate syslog-ng configuration from YAML,
• use non-YAML configuration,
• start, stop or reload syslog-ng.
There is also an execution module, which can check the syntax of the configuration, get the version and other information about syslog-ng.
Configuration Users can create syslog-ng configuration statements with the syslog_ng.config function. It requires a name and a config parameter. The name parameter determines the name of the generated statement and the config parameter holds a parsed YAML structure.
A statement can be declared in the following forms (both are equivalent):
source.s_localhost: syslog_ng.config: - config: - tcp: - ip: "127.0.0.1" - port: 1233
s_localhost: syslog_ng.config: - config: source: - tcp: - ip: "127.0.0.1" - port: 1233
The first one is called short form, because it needs less typing. Users can use lists and dictionaries to specify their configuration. The format is quite self describing and there are more examples [at the end](#examples) of this document.
Quotation The quotation can be tricky sometimes but here are some rules to follow:
• when a string meant to be "string" in the generated configuration, it should be like '"string"' in the YAML document
• similarly, users should write "'string'" to get 'string' in the generated configuration
Full example The following configuration is an example, how a complete syslog-ng configuration looks like:
# Set the location of the configuration file set_location: module.run: - name: syslog_ng.set_config_file - m_name: "/home/tibi/install/syslog-ng/etc/syslog-ng.conf"
# The syslog-ng and syslog-ng-ctl binaries are here. You needn't use # this method if these binaries can be found in a directory in your PATH. set_bin_path: module.run: - name: syslog_ng.set_binary_path - m_name: "/home/tibi/install/syslog-ng/sbin"
# Writes the first lines into the config file, also erases its previous # content write_version: module.run: - name: syslog_ng.write_version - m_name: "3.6"