neulab/tldr huggingface dataset:
To generate a cipher from a secret:
$ echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r <KEY-name>
#!yaml|gpg
a-secret: | -----BEGIN PGP MESSAGE----- Version: GnuPG v1
hQEMAweRHKaPCfNeAQf9GLTN16hCfXAbPwU6BbBK0unOc7i9/etGuVc5CyU9Q6um QuetdvQVLFO/HkrC4lgeNQdM6D9E8PKonMlgJPyUvC8ggxhj0/IPFEKmrsnv2k6+ cnEfmVexS7o/U1VOVjoyUeliMCJlAz/30RXaME49Cpi6No2+vKD8a4q4nZN1UZcG RhkhC0S22zNxOXQ38TBkmtJcqxnqT6YWKTUsjVubW3bVC+u2HGqJHu79wmwuN8tz m4wBkfCAd8Eyo2jEnWQcM4TcXiF01XPL4z4g1/9AAxh+Q4d8RIRP4fbw7ct4nCJv Gr9v2DTF7HNigIMl4ivMIn9fp+EZurJNiQskLgNbktJGAeEKYkqX5iCuB1b693hJ FKlwHiJt5yA8X2dDtfk8/Ph1Jx2TwGS+lGjlZaNqp3R1xuAZzXzZMLyZDe5+i3RJ skqmFTbOiA===Eqsm -----END PGP MESSAGE-----
Encrypted CLI Pillar Data New in version 2016.3.0.
Functions like state.highstate and state.sls allow for pillar data to be passed on the CLI.
salt myminion state.highstate pillar="{'mypillar': 'foo'}"
Starting with the 2016.3.0 release of Salt, it is now possible for this pillar data to be GPG-encrypted, and to use the GPG renderer to decrypt it.
Replacing Newlines To pass encrypted pillar data on the CLI, the ciphertext must have its newlines replaced with a literal backslash-n (\n), as newlines are not supported within Salt CLI arguments. There are a number of ways to do this:
With awk or Perl:
# awk ciphertext=`echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | awk '{printf "%s\\n",$0} END {print ""}'` # Perl ciphertext=`echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | perl -pe 's/\n/\\n/g'`
With Python:
secret, stderr = subprocess.Popen( ['gpg', '--armor', '--batch', '--trust-model', 'always', '--encrypt', '-r', 'user@domain.com'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate(input='supersecret')
if secret: print(secret.replace('\n', r'\n')) else: raise ValueError('No ciphertext found: {0}'.format(stderr))
ciphertext=`python /path/to/script.py`
The ciphertext can be included in the CLI pillar data like so:
salt myminion state.sls secretstuff pillar_enc=gpg pillar="{secret_pillar: '$ciphertext'}"
The pillar_enc=gpg argument tells Salt that there is GPG-encrypted pillar data, so that the CLI pillar data is passed through the GPG renderer, which will iterate recursively though the CLI pillar dictionary to decrypt any encrypted values.
Encrypting the Entire CLI Pillar Dictionary If several values need to be encrypted, it may be more convenient to encrypt the entire CLI pillar dictionary. Again, this can be done in several ways:
# awk ciphertext=`echo -n "{'secret_a': 'CorrectHorseBatteryStaple', 'secret_b': 'GPG is fun!'}" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | awk '{printf "%s\\n",$0} END {print ""}'` # Perl ciphertext=`echo -n "{'secret_a': 'CorrectHorseBatteryStaple', 'secret_b': 'GPG is fun!'}" | gpg --armor --batch --trust-model always --encrypt -r user@domain.com | perl -pe 's/\n/\\n/g'`
pillar_data = {'secret_a': 'CorrectHorseBatteryStaple', 'secret_b': 'GPG is fun!'}
secret, stderr = subprocess.Popen( ['gpg', '--armor', '--batch', '--trust-model', 'always', '--encrypt', '-r', 'user@domain.com'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate(input=repr(pillar_data))
With the entire pillar dictionary now encrypted, it can be included in the CLI pillar data like so:
salt myminion state.sls secretstuff pillar_enc=gpg pillar="$ciphertext"
salt.renderers.gpg.render(gpg_data, saltenv='base', sls='', argline='', **kwargs) Create a gpg object given a gpg_keydir, and then use it to try to decrypt the data to be rendered.
salt.renderers.hjson hjson renderer for Salt
See the hjson documentation for more information
salt.renderers.hjson.render(hjson_data, saltenv='base', sls='', **kws) Accepts HJSON as a string or as a file object and runs it through the HJSON parser.
salt.renderers.jinja Jinja loading utils to enable a more powerful backend for jinja templates
salt.renderers.jinja.render(template_file, saltenv='base', sls='', argline='', context=None, tmplpath=None, **kws) Render the template_file, passing the functions and grains into the Jinja rendering system.
Return type string
class salt.utils.jinja.SerializerExtension(environment) Yaml and Json manipulation.
Format filters
Allows jsonifying or yamlifying any data structure. For example, this dataset:
data = { 'foo': True, 'bar': 42, 'baz': [1, 2, 3], 'qux': 2.0 }
yaml = {{ data|yaml }} json = {{ data|json }} python = {{ data|python }} xml  = {{ {'root_node': data}|xml }}
will be rendered as:
yaml = {bar: 42, baz: [1, 2, 3], foo: true, qux: 2.0} json = {"baz": [1, 2, 3], "foo": true, "bar": 42, "qux": 2.0} python = {'bar': 42, 'baz': [1, 2, 3], 'foo': True, 'qux': 2.0} xml = """<<?xml version="1.0" ?> <root_node bar="42" foo="True" qux="2.0"> <baz>1</baz> <baz>2</baz> <baz>3</baz> </root_node>"""
The yaml filter takes an optional flow_style parameter to control the default-flow-style parameter of the YAML dumper.
{{ data|yaml(False) }}
bar: 42 baz: - 1 - 2 - 3 foo: true qux: 2.0
Load filters
Strings and variables can be deserialized with load_yaml and load_json tags and filters. It allows one to manipulate data directly in templates, easily:
{%- set yaml_src = "{foo: it works}"|load_yaml %} {%- set json_src = "{'bar': 'for real'}"|load_json %} Dude, {{ yaml_src.foo }} {{ json_src.bar }}!
Dude, it works for real!
Load tags
Salt implements load_yaml and load_json tags. They work like the import tag, except that the document is also deserialized.
Syntaxes are {% load_yaml as [VARIABLE] %}[YOUR DATA]{% endload %} and {% load_json as [VARIABLE] %}[YOUR DATA]{% endload %}
{% load_yaml as yaml_src %} foo: it works {% endload %} {% load_json as json_src %} { "bar": "for real" } {% endload %} Dude, {{ yaml_src.foo }} {{ json_src.bar }}!
External files can be imported and made available as a Jinja variable.
{% import_yaml "myfile.yml" as myfile %} {% import_json "defaults.json" as defaults %} {% import_text "completeworksofshakespeare.txt" as poems %}
import_* and load_* tags will automatically expose their target variable to import. This feature makes catalog of data to handle.
# doc1.sls {% load_yaml as var1 %} foo: it works {% endload %} {% load_yaml as var2 %} bar: for real {% endload %}
# doc2.sls {% from "doc1.sls" import var1, var2 as local2 %} {{ var1.foo }} {{ local2.bar }}
** Escape Filters **
Allows escaping of strings so they can be interpreted literally by another function.
regex_escape = {{ 'https://example.com?foo=bar%20baz' | regex_escape }}
regex_escape = https\:\/\/example\.com\?foo\=bar\%20baz
** Set Theory Filters **
Performs set math using Jinja filters.
unique = {{ ['foo', 'foo', 'bar'] | unique }}
unique = ['foo', 'bar']
salt.renderers.json JSON Renderer for Salt
salt.renderers.json.render(json_data, saltenv='base', sls='', **kws) Accepts JSON as a string or as a file object and runs it through the JSON parser.
salt.renderers.json5 JSON5 Renderer for Salt
JSON5 is an unofficial extension to JSON. See http://json5.org/ for more information.
This renderer requires the json5 python bindings, installable via pip.
salt.renderers.json5.render(json_data, saltenv='base', sls='', **kws) Accepts JSON as a string or as a file object and runs it through the JSON parser.
salt.renderers.mako Mako Renderer for Salt
salt.renderers.mako.render(template_file, saltenv='base', sls='', context=None, tmplpath=None, **kws) Render the template_file, passing the functions and grains into the Mako rendering system.
salt.renderers.msgpack salt.renderers.msgpack.render(msgpack_data, saltenv='base', sls='', **kws) Accepts a message pack string or a file object, renders said data back to a python dict.
salt.renderers.nacl Renderer that will decrypt NACL ciphers
Any key in the SLS file can be an NACL cipher, and this renderer will decrypt it before passing it off to Salt. This allows you to safely store secrets in source control, in such a way that only your Salt master can decrypt them and distribute them only to the minions that need them.
This renderer requires the libsodium library binary and libnacl >= 1.5.1 python package (support for sealed boxes came in 1.5.1 version).
# salt-call --local nacl.keygen sk_file=/root/.nacl
Using encrypted pillar To encrypt secrets, copy the public key to your local machine and run:
$ salt-call --local nacl.enc datatoenc pk_file=/root/.nacl.pub
To apply the renderer on a file-by-file basis add the following line to the top of any pillar with nacl encrypted data in it:
#!yaml|nacl
a-secret: "NACL[MRN3cc+fmdxyQbz6WMF+jq1hKdU5X5BBI7OjK+atvHo1ll+w1gZ7XyWtZVfq9gK9rQaMfkDxmidJKwE0Mw==]"
salt.renderers.nacl.render(nacl_data, saltenv='base', sls='', argline='', **kwargs) Decrypt the data to be rendered using the given nacl key or the one given in config
salt.renderers.pass Pass Renderer for Salt pass is an encrypted on-disk password store.
Setup Note: <user> needs to be replaced with the user salt-master will be running as.
Have private gpg loaded into user's gpg keyring
load_private_gpg_key: cmd.run: - name: gpg --import <location_of_private_gpg_key> - unless: gpg --list-keys '<gpg_name>'
Said private key's public key should have been used when encrypting pass entries that are of interest for pillar data.
Fetch and keep local pass git repo up-to-date
update_pass: git.latest: - force_reset: True - name: <git_repo> - target: /<user>/.password-store - identity: <location_of_ssh_private_key> - require: - cmd: load_private_gpg_key
Install pass binary
pass: pkg.installed
salt.renderers.pass.render(pass_info, saltenv='base', sls='', argline='', **kwargs) Fetch secret from pass based on pass_path
salt.renderers.py Pure python state renderer To use this renderer, the SLS file should contain a function called run which returns highstate data.
The highstate data is a dictionary containing identifiers as keys, and execution dictionaries as values. For example the following state declaration in YAML:
common_packages: pkg.installed: - pkgs: - curl - vim
translates to:
{'common_packages': {'pkg.installed': [{'pkgs': ['curl', 'vim']}]}}
In this module, a few objects are defined for you, giving access to Salt's execution functions, grains, pillar, etc. They are:
• __salt__ - Execution functions (i.e.  __salt__['test.echo']('foo'))
• __grains__ - Grains (i.e. __grains__['os'])
• __pillar__ - Pillar data (i.e. __pillar__['foo'])
• __opts__ - Minion configuration options
• __env__ - The effective salt fileserver environment (i.e. base). Also referred to as a "saltenv". __env__ should not be modified in a pure python SLS file. To use a different environment, the environment should be set when executing the state. This can be done in a couple different ways:
• Using the saltenv argument on the salt CLI (i.e. salt '*' state.sls foo.bar.baz saltenv=env_name).
• By adding a saltenv argument to an individual state within the SLS file. In other words, adding a line like this to the state's data structure: {'saltenv': 'env_name'}
• __sls__ - The SLS path of the file. For example, if the root of the base environment is /srv/salt, and the SLS file is /srv/salt/foo/bar/baz.sls, then __sls__ in that file will be foo.bar.baz.
When writing a reactor SLS file the global context data (same as context {{ data }} for states written with Jinja + YAML) is available. The following YAML + Jinja state declaration:
{% if data['id'] == 'mysql1' %} highstate_run: local.state.apply: - tgt: mysql1 {% endif %}
if data['id'] == 'mysql1': return {'highstate_run': {'local.state.apply': [{'tgt': 'mysql1'}]}}
Full Example #!py
def run(): config = {}
if __grains__['os'] == 'Ubuntu': user = 'ubuntu' group = 'ubuntu' home = '/home/{0}'.format(user) else: user = 'root' group = 'root' home = '/root/'
config['s3cmd'] = { 'pkg': [ 'installed', {'name': 's3cmd'}, ], }
config[home + '/.s3cfg'] = { 'file.managed': [ {'source': 'salt://s3cfg/templates/s3cfg'}, {'template': 'jinja'}, {'user': user}, {'group': group}, {'mode': 600}, {'context': { 'aws_key': __pillar__['AWS_ACCESS_KEY_ID'], 'aws_secret_key': __pillar__['AWS_SECRET_ACCESS_KEY'], }, }, ], }
salt.renderers.py.render(template, saltenv='base', sls='', tmplpath=None, **kws) Render the python module's components
salt.renderers.pydsl A Python-based DSL
maintainer Jack Kuan <kjkuan@gmail.com>
The pydsl renderer allows one to author salt formulas (.sls files) in pure Python using a DSL that's easy to write and easy to read. Here's an example:
#!pydsl
apache = state('apache') apache.pkg.installed() apache.service.running() state('/var/www/index.html') \ .file('managed', source='salt://webserver/index.html') \ .require(pkg='apache')
Notice that any Python code is allow in the file as it's really a Python module, so you have the full power of Python at your disposal. In this module, a few objects are defined for you, including the usual (with __ added) __salt__ dictionary, __grains__, __pillar__, __opts__, __env__, and __sls__, plus a few more: __file__ local file system path to the sls module.
__pydsl__ Salt PyDSL object, useful for configuring DSL behavior per sls rendering.
include Salt PyDSL function for creating include-declaration's.
extend Salt PyDSL function for creating extend-declaration's.
state Salt PyDSL function for creating ID-declaration's.
A state ID-declaration is created with a state(id) function call. Subsequent state(id) call with the same id returns the same object. This singleton access pattern applies to all declaration objects created with the DSL.
state('example') assert state('example') is state('example') assert state('example').cmd is state('example').cmd assert state('example').cmd.running is state('example').cmd.running
The id argument is optional. If omitted, an UUID will be generated and used as the id.
state(id) returns an object under which you can create a state-declaration object by accessing an attribute named after any state module available in Salt.
state('example').cmd state('example').file state('example').pkg ...
Then, a function-declaration object can be created from a state-declaration object by one of the following two ways:
1. by calling a method named after the state function on the state-declaration object.
state('example').file.managed(...)
2. by directly calling the attribute named for the state-declaration, and supplying the state function name as the first argument.
state('example').file('managed', ...)
With either way of creating a function-declaration object, any function-arg-declaration's can be passed as keyword arguments to the call. Subsequent calls of a function-declaration will update the arg declarations.
state('example').file('managed', source='salt://webserver/index.html') state('example').file.managed(source='salt://webserver/index.html')
As a shortcut, the special name argument can also be passed as the first or second positional argument depending on the first or second way of calling the state-declaration object. In the following two examples ls -la is the name argument.
state('example').cmd.run('ls -la', cwd='/') state('example').cmd('run', 'ls -la', cwd='/')
Finally, a requisite-declaration object with its requisite-reference's can be created by invoking one of the requisite methods (see State Requisites) on either a function-declaration object or a state-declaration object.  The return value of a requisite call is also a function-declaration object, so you can chain several requisite calls together.
Arguments to a requisite call can be a list of state-declaration objects and/or a set of keyword arguments whose names are state modules and values are IDs of ID-declaration's or names of name-declaration's.
apache2 = state('apache2') apache2.pkg.installed() state('libapache2-mod-wsgi').pkg.installed()
# you can call requisites on function declaration apache2.service.running() \ .require(apache2.pkg, pkg='libapache2-mod-wsgi') \ .watch(file='/etc/apache2/httpd.conf')
# or you can call requisites on state declaration. # this actually creates an anonymous function declaration object # to add the requisites. apache2.service.require(state('libapache2-mod-wsgi').pkg, pkg='apache2') \ .watch(file='/etc/apache2/httpd.conf')
# we still need to set the name of the function declaration. apache2.service.running()
include-declaration objects can be created with the include function, while extend-declaration objects can be created with the extend function, whose arguments are just function-declaration objects.
include('edit.vim', 'http.server') extend(state('apache2').service.watch(file='/etc/httpd/httpd.conf')
The include function, by default, causes the included sls file to be rendered as soon as the include function is called. It returns a list of rendered module objects; sls files not rendered with the pydsl renderer return None's.  This behavior creates no include-declaration's in the resulting high state data structure.
# including multiple sls returns a list. _, mod = include('a-non-pydsl-sls', 'a-pydsl-sls')
assert _ is None assert isinstance(slsmods[1], types.ModuleType)
# including a single sls returns a single object mod = include('a-pydsl-sls')
# myfunc is a function that calls state(...) to create more states. mod.myfunc(1, 2, "three")
Notice how you can define a reusable function in your pydsl sls module and then call it via the module returned by include.
It's still possible to do late includes by passing the delayed=True keyword argument to include.
include('edit.vim', 'http.server', delayed=True)
Above will just create a include-declaration in the rendered result, and such call always returns None.
Special integration with the cmd state Taking advantage of rendering a Python module, PyDSL allows you to declare a state that calls a pre-defined Python function when the state is executed.
greeting = "hello world" def helper(something, *args, **kws): print greeting                # hello world print something, args, kws    # test123 ['a', 'b', 'c'] {'x': 1, 'y': 2}
state().cmd.call(helper, "test123", 'a', 'b', 'c', x=1, y=2)
The cmd.call state function takes care of calling our helper function with the arguments we specified in the states, and translates the return value of our function into a structure expected by the state system.  See salt.states.cmd.call() for more information.
Implicit ordering of states Salt states are explicitly ordered via requisite-declaration's.  However, with pydsl it's possible to let the renderer track the order of creation for function-declaration objects, and implicitly add require requisites for your states to enforce the ordering. This feature is enabled by setting the ordered option on __pydsl__.
NOTE: this feature is only available if your minions are using Python >= 2.7.
include('some.sls.file')
A = state('A').cmd.run(cwd='/var/tmp') extend(A)
__pydsl__.set(ordered=True)
for i in range(10): i = str(i) state(i).cmd.run('echo '+i, cwd='/') state('1').cmd.run('echo one') state('2').cmd.run(name='echo two')
Notice that the ordered option needs to be set after any extend calls. This is to prevent pydsl from tracking the creation of a state function that's passed to an extend call.
Above example should create states from 0 to 9 that will output 0, one, two, 3, ... 9, in that order.
It's important to know that pydsl tracks the creations of function-declaration objects, and automatically adds a require requisite to a function-declaration object that requires the last function-declaration object created before it in the sls file.
This means later calls (perhaps to update the function's function-arg-declaration) to a previously created function declaration will not change the order.
Render time state execution When Salt processes a salt formula file, the file is rendered to salt's high state data representation by a renderer before the states can be executed.  In the case of the pydsl renderer, the .sls file is executed as a python module as it is being rendered which makes it easy to execute a state at render time.  In pydsl, executing one or more states at render time can be done by calling a configured ID-declaration object.
s = state() # save for later invocation
# configure it s.cmd.run('echo at render time', cwd='/') s.file.managed('target.txt', source='salt://source.txt')
s() # execute the two states now
Once an ID-declaration is called at render time it is detached from the sls module as if it was never defined.
NOTE: If implicit ordering is enabled (i.e., via __pydsl__.set(ordered=True)) then the first invocation of a ID-declaration object must be done before a new function-declaration is created.
Integration with the stateconf renderer The salt.renderers.stateconf renderer offers a few interesting features that can be leveraged by the pydsl renderer. In particular, when using with the pydsl renderer, we are interested in stateconf's sls namespacing feature (via dot-prefixed id declarations), as well as, the automatic start and goal states generation.
Now you can use pydsl with stateconf like this:
#!pydsl|stateconf -ps
include('xxx', 'yyy')
# ensure that states in xxx run BEFORE states in this file. extend(state('.start').stateconf.require(stateconf='xxx::goal'))
# ensure that states in yyy run AFTER states in this file. extend(state('.goal').stateconf.require_in(stateconf='yyy::start'))
-s enables the generation of a stateconf start state, and -p lets us pipe high state data rendered by pydsl to stateconf. This example shows that by require-ing or require_in-ing the included sls' start or goal states, it's possible to ensure that the included sls files can be made to execute before or after a state in the including sls file.
Importing custom Python modules To use a custom Python module inside a PyDSL state, place the module somewhere that it can be loaded by the Salt loader, such as _modules in the /srv/salt directory.
Then, copy it to any minions as necessary by using saltutil.sync_modules.
To import into a PyDSL SLS, one must bypass the Python importer and insert it manually by getting a reference from Python's sys.modules dictionary.
def main(): my_mod = sys.modules['salt.loaded.ext.module.my_mod']
exception salt.renderers.pydsl.PyDslError
exception salt.renderers.pydsl.SaltRenderError(message, line_num=None, buf='', marker=' <======================', trace=None) Used when a renderer needs to raise an explicit error. If a line number and buffer string are passed, get_context will be invoked to get the location of the error.
salt.renderers.pydsl.render(template, saltenv='base', sls='', tmplpath=None, rendered_sls=None, **kws)
salt.renderers.pyobjects Python renderer that includes a Pythonic Object based interface
maintainer Evan Borgstrom <evan@borgstrom.ca>
Let's take a look at how you use pyobjects in a state file. Here's a quick example that ensures the /tmp directory is in the correct state.
#!pyobjects
File.managed("/tmp", user='root', group='root', mode='1777')
Nice and Pythonic!
By using the "shebang" syntax to switch to the pyobjects renderer we can now write our state data using an object based interface that should feel at home to python developers. You can import any module and do anything that you'd like (with caution, importing sqlalchemy, django or other large frameworks has not been tested yet). Using the pyobjects renderer is exactly the same as using the built-in Python renderer with the exception that pyobjects provides you with an object based interface for generating state data.
Creating state data Pyobjects takes care of creating an object for each of the available states on the minion. Each state is represented by an object that is the CamelCase version of its name (i.e. File, Service, User, etc), and these objects expose all of their available state functions (i.e. File.managed, Service.running, etc).
The name of the state is split based upon underscores (_), then each part is capitalized and finally the parts are joined back together.
• postgres_user becomes PostgresUser
• ssh_known_hosts becomes SshKnownHosts
Context Managers and requisites How about something a little more complex. Here we're going to get into the core of how to use pyobjects to write states.
with Pkg.installed("nginx"): Service.running("nginx", enable=True)
with Service("nginx", "watch_in"): File.managed("/etc/nginx/conf.d/mysite.conf", owner='root', group='root', mode='0444', source='salt://nginx/mysite.conf')
The objects that are returned from each of the magic method calls are setup to be used a Python context managers (with) and when you use them as such all declarations made within the scope will automatically use the enclosing state as a requisite!
The above could have also been written use direct requisite statements as.
Pkg.installed("nginx") Service.running("nginx", enable=True, require=Pkg("nginx")) File.managed("/etc/nginx/conf.d/mysite.conf", owner='root', group='root', mode='0444', source='salt://nginx/mysite.conf', watch_in=Service("nginx"))
You can use the direct requisite statement for referencing states that are generated outside of the current file.
# some-other-package is defined in some other state file Pkg.installed("nginx", require=Pkg("some-other-package"))
The last thing that direct requisites provide is the ability to select which of the SaltStack requisites you want to use (require, require_in, watch, watch_in, use & use_in) when using the requisite as a context manager.
with Service("my-service", "watch_in"): ...
The above example would cause all declarations inside the scope of the context manager to automatically have their watch_in set to Service("my-service").
Including and Extending To include other states use the include() function. It takes one name per state to include.
To extend another state use the extend() function on the name when creating a state.
include('http', 'ssh')
Service.running(extend('apache'), watch=[File('/etc/httpd/extra/httpd-vhosts.conf')])
Importing from other state files Like any Python project that grows you will likely reach a point where you want to create reusability in your state tree and share objects between state files, Map Data (described below) is a perfect example of this.
To facilitate this Python's import statement has been augmented to allow for a special case when working with a Salt state tree. If you specify a Salt url (salt://...) as the target for importing from then the pyobjects renderer will take care of fetching the file for you, parsing it with all of the pyobjects features available and then place the requested objects in the global scope of the template being rendered.
This works for all types of import statements; import X, from X import Y, and from X import Y as Z.
import salt://myfile.sls from salt://something/data.sls import Object from salt://something/data.sls import Object as Other
See the Map Data section for a more practical use.
• Imported objects are ALWAYS put into the global scope of your template, regardless of where your import statement is.
Salt object In the spirit of the object interface for creating state data pyobjects also provides a simple object interface to the __salt__ object.
A function named salt exists in scope for your sls files and will dispatch its attributes to the __salt__ dictionary.
The following lines are functionally equivalent:
ret = salt.cmd.run(bar) ret = __salt__['cmd.run'](bar)
Pillar, grain, mine & config data Pyobjects provides shortcut functions for calling pillar.get, grains.get, mine.get & config.get on the __salt__ object. This helps maintain the readability of your state files.
Each type of data can be access by a function of the same name: pillar(), grains(), mine() and config().
The following pairs of lines are functionally equivalent:
value = pillar('foo:bar:baz', 'qux') value = __salt__['pillar.get']('foo:bar:baz', 'qux')
value = grains('pkg:apache') value = __salt__['grains.get']('pkg:apache')
value = mine('os:Fedora', 'network.interfaces', 'grain') value = __salt__['mine.get']('os:Fedora', 'network.interfaces', 'grain')
value = config('foo:bar:baz', 'qux') value = __salt__['config.get']('foo:bar:baz', 'qux')
Map Data When building complex states or formulas you often need a way of building up a map of data based on grain data. The most common use of this is tracking the package and service name differences between distributions.
To build map data using pyobjects we provide a class named Map that you use to build your own classes with inner classes for each set of values for the different grain matches.
class Samba(Map): merge = 'samba:lookup' # NOTE: priority is new to 2017.7.0 priority = ('os_family', 'os')
class Ubuntu: __grain__ = 'os' service = 'smbd'
class Debian: server = 'samba' client = 'samba-client' service = 'samba'
class RHEL: __match__ = 'RedHat' server = 'samba' client = 'samba' service = 'smb'
NOTE: By default, the os_family grain will be used as the target for matching. This can be overridden by specifying a __grain__ attribute.
If a __match__ attribute is defined for a given class, then that value will be matched against the targeted grain, otherwise the class name's value will be be matched.
Given the above example, the following is true:
1. Minions with an os_family of Debian will be assigned the attributes defined in the Debian class.
2. Minions with an os grain of Ubuntu will be assigned the attributes defined in the Ubuntu class.
3. Minions with an os_family grain of RedHat will be assigned the attributes defined in the RHEL class.
That said, sometimes a minion may match more than one class. For instance, in the above example, Ubuntu minions will match both the Debian and Ubuntu classes, since Ubuntu has an os_family grain of Debian and an os grain of Ubuntu. As of the 2017.7.0 release, the order is dictated by the order of declaration, with classes defined later overriding earlier ones. Additionally, 2017.7.0 adds support for explicitly defining the ordering using an optional attribute called priority.
Given the above example, os_family matches will be processed first, with os matches processed after. This would have the effect of assigning smbd as the service attribute on Ubuntu minions. If the priority item was not defined, or if the order of the items in the priority tuple were reversed, Ubuntu minions would have a service attribute of samba, since os_family matches would have been processed second.
To use this new data you can import it into your state file and then access your attributes. To access the data in the map you simply access the attribute name on the base class that is extending Map. Assuming the above Map was in the file samba/map.sls, you could do the following.
from salt://samba/map.sls import Samba
with Pkg.installed("samba", names=[Samba.server, Samba.client]): Service.running("samba", name=Samba.service)
class salt.renderers.pyobjects.PyobjectsModule(name, attrs) This provides a wrapper for bare imports.
salt.renderers.pyobjects.load_states() This loads our states into the salt __context__
salt.renderers.pyobjects.render(template, saltenv='base', sls='', salt_data=True, **kwargs)
salt.renderers.stateconf maintainer Jack Kuan <kjkuan@gmail.com>
This module provides a custom renderer that processes a salt file with a specified templating engine (e.g. Jinja) and a chosen data renderer (e.g. YAML), extracts arguments for any stateconf.set state, and provides the extracted arguments (including Salt-specific args, such as require, etc) as template context. The goal is to make writing reusable/configurable/parameterized salt files easier and cleaner.
To use this renderer, either set it as the default renderer via the renderer option in master/minion's config, or use the shebang line in each individual sls file, like so: #!stateconf. Note, due to the way this renderer works, it must be specified as the first renderer in a render pipeline. That is, you cannot specify #!mako|yaml|stateconf, for example. Instead, you specify them as renderer arguments: #!stateconf mako . yaml.
Here's a list of features enabled by this renderer.
• Prefixes any state id (declaration or reference) that starts with a dot (.)  to avoid duplicated state ids when the salt file is included by other salt files.
For example, in the salt://some/file.sls, a state id such as .sls_params will be turned into some.file::sls_params. Example:
#!stateconf yaml . jinja
.vim: pkg.installed
Above will be translated into:
some.file::vim: pkg.installed: - name: vim
Notice how that if a state under a dot-prefixed state id has no name argument then one will be added automatically by using the state id with the leading dot stripped off.
The leading dot trick can be used with extending state ids as well, so you can include relatively and extend relatively. For example, when extending a state in salt://some/other_file.sls, e.g.:
include: - .file
extend: .file::sls_params: stateconf.set: - name1: something
Above will be pre-processed into:
include: - some.file
extend: some.file::sls_params: stateconf.set: - name1: something
• Adds a sls_dir context variable that expands to the directory containing the rendering salt file. So, you can write salt://{{sls_dir}}/... to reference templates files used by your salt file.
• Recognizes the special state function, stateconf.set, that configures a default list of named arguments usable within the template context of the salt file. Example:
.sls_params: stateconf.set: - name1: value1 - name2: value2 - name3: - value1 - value2 - value3 - require_in: - cmd: output
# --- end of state config ---
.output: cmd.run: - name: | echo 'name1={{sls_params.name1}} name2={{sls_params.name2}} name3[1]={{sls_params.name3[1]}} '
This even works with include + extend so that you can override the default configured arguments by including the salt file and then extend the stateconf.set states that come from the included salt file. (IMPORTANT: Both the included and the extending sls files must use the stateconf renderer for this ``extend`` to work!)
Notice that the end of configuration marker (# --- end of state config --) is needed to separate the use of 'stateconf.set' form the rest of your salt file. The regex that matches such marker can be configured via the stateconf_end_marker option in your master or minion config file.
Sometimes, it is desirable to set a default argument value that's based on earlier arguments in the same stateconf.set. For example, it may be tempting to do something like this:
.apache: stateconf.set: - host: localhost - port: 1234 - url: 'http://{{host}}:{{port}}/'
.test: cmd.run: - name: echo '{{apache.url}}' - cwd: /
However, this won't work. It can however be worked around like so:
.apache: stateconf.set: - host: localhost - port: 1234 {#  - url: 'http://{{host}}:{{port}}/' #}
# --- end of state config --- # {{ apache.setdefault('url', "http://%(host)s:%(port)s/" % apache) }}
• Adds support for relative include and exclude of .sls files. Example:
include: - .apache - .db.mysql - ..app.django
exclude: - sls: .users
If the above is written in a salt file at salt://some/where.sls then it will include salt://some/apache.sls, salt://some/db/mysql.sls and salt://app/django.sls, and exclude salt://some/users.ssl. Actually, it does that by rewriting the above include and exclude into:
include: - some.apache - some.db.mysql - app.django
exclude: - sls: some.users
• Optionally (enabled by default, disable via the -G renderer option, e.g. in the shebang line: #!stateconf -G), generates a stateconf.set goal state (state id named as .goal by default, configurable via the master/minion config option, stateconf_goal_state) that requires all other states in the salt file. Note, the .goal state id is subject to dot-prefix rename rule mentioned earlier.
Such goal state is intended to be required by some state in an including salt file. For example, in your webapp salt file, if you include a sls file that is supposed to setup Tomcat, you might want to make sure that all states in the Tomcat sls file will be executed before some state in the webapp sls file.
• Optionally (enable via the -o renderer option, e.g. in the shebang line: #!stateconf -o), orders the states in a sls file by adding a require requisite to each state such that every state requires the state defined just before it. The order of the states here is the order they are defined in the sls file. (Note: this feature is only available if your minions are using Python >= 2.7. For Python2.6, it should also work if you install the ordereddict module from PyPI)
By enabling this feature, you are basically agreeing to author your sls files in a way that gives up the explicit (or implicit?) ordering imposed by the use of require, watch, require_in or watch_in requisites, and instead, you rely on the order of states you define in the sls files. This may or may not be a better way for you. However, if there are many states defined in a sls file, then it tends to be easier to see the order they will be executed with this feature.
You are still allowed to use all the requisites, with a few restrictions.  You cannot require or watch a state defined after the current state. Similarly, in a state, you cannot require_in or watch_in a state defined before it. Breaking any of the two restrictions above will result in a state loop. The renderer will check for such incorrect uses if this feature is enabled.
Additionally, names declarations cannot be used with this feature because the way they are compiled into low states make it impossible to guarantee the order in which they will be executed. This is also checked by the renderer. As a workaround for not being able to use names, you can achieve the same effect, by generate your states with the template engine available within your sls file.
Finally, with the use of this feature, it becomes possible to easily make an included sls file execute all its states after some state (say, with id X) in the including sls file.  All you have to do is to make state, X, require_in the first state defined in the included sls file.
When writing sls files with this renderer, one should avoid using what can be defined in a name argument of a state as the state's id. That is, avoid writing states like this:
/path/to/some/file: file.managed: - source: salt://some/file
cp /path/to/some/file file2: cmd.run: - cwd: / - require: - file: /path/to/some/file
Instead, define the state id and the name argument separately for each state. Also, the ID should be something meaningful and easy to reference within a requisite (which is a good habit anyway, and such extra indirection would also makes the sls file easier to modify later). Thus, the above states should be written like this:
add-some-file: file.managed: - name: /path/to/some/file - source: salt://some/file
copy-files: cmd.run: - name: cp /path/to/some/file file2 - cwd: / - require: - file: add-some-file
Moreover, when referencing a state from a requisite, you should reference the state's id plus the state name rather than the state name plus its name argument. (Yes, in the above example, you can actually require the file: /path/to/some/file, instead of the file: add-some-file). The reason is that this renderer will re-write or rename state id's and their references for state id's prefixed with .. So, if you reference name then there's no way to reliably rewrite such reference.
salt.renderers.toml salt.renderers.tomlmod.render(sls_data, saltenv='base', sls='', **kws) Accepts TOML as a string or as a file object and runs it through the parser.
salt.renderers.wempy salt.renderers.wempy.render(template_file, saltenv='base', sls='', argline='', context=None, **kws) Render the data passing the functions and grains into the rendering system
salt.renderers.yaml Understanding YAML The default renderer for SLS files is the YAML renderer. YAML is a markup language with many powerful features. However, Salt uses a small subset of YAML that maps over very commonly used data structures, like lists and dictionaries. It is the job of the YAML renderer to take the YAML data structure and compile it into a Python data structure for use by Salt.
Though YAML syntax may seem daunting and terse at first, there are only three very simple rules to remember when writing YAML for SLS files.
Rule One: Indentation YAML uses a fixed indentation scheme to represent relationships between data layers. Salt requires that the indentation for each level consists of exactly two spaces. Do not use tabs.
Rule Two: Colons Python dictionaries are, of course, simply key-value pairs. Users from other languages may recognize this data type as hashes or associative arrays.
Dictionary keys are represented in YAML as strings terminated by a trailing colon.  Values are represented by either a string following the colon, separated by a space:
my_key: my_value
In Python, the above maps to:
{"my_key": "my_value"}
Dictionaries can be nested:
first_level_dict_key: second_level_dict_key: value_in_second_level_dict
And in Python:
{"first_level_dict_key": {"second_level_dict_key": "value_in_second_level_dict"}}
Rule Three: Dashes To represent lists of items, a single dash followed by a space is used. Multiple items are a part of the same list as a function of their having the same level of indentation.
- list_value_one - list_value_two - list_value_three
Lists can be the value of a key-value pair. This is quite common in Salt:
my_dictionary: - list_value_one - list_value_two - list_value_three
Reference YAML Renderer for Salt
For YAML usage information see Understanding YAML.
salt.renderers.yaml.get_yaml_loader(argline) Return the ordered dict yaml loader
salt.renderers.yaml.render(yaml_data, saltenv='base', sls='', argline='', **kws) Accepts YAML as a string or as a file object and runs it through the YAML parser.
salt.renderers.yamlex YAMLEX renderer is a replacement of the YAML renderer.  It's 100% YAML with a pinch of Salt magic:
• All mappings are automatically OrderedDict
• All strings are automatically str obj
• data aggregation with !aggregation yaml tag, based on the salt.utils.aggregation module.
• data aggregation over documents for pillar
Instructed aggregation within the !aggregation and the !reset tags:
#!yamlex foo: !aggregate first foo: !aggregate second bar: !aggregate {first: foo} bar: !aggregate {second: bar} baz: !aggregate 42 qux: !aggregate default !reset qux: !aggregate my custom data
is roughly equivalent to
foo: [first, second] bar: {first: foo, second: bar} baz: [42] qux: [my custom data]
Reference salt.renderers.yamlex.render(sls_data, saltenv='base', sls='', **kws) Accepts YAML_EX as a string or as a file object and runs it through the YAML_EX parser.
This section describes the fundamental components and concepts that you need to understand to use Salt.
Grains Salt comes with an interface to derive information about the underlying system.  This is called the grains interface, because it presents salt with grains of information. Grains are collected for the operating system, domain name, IP address, kernel, OS type, memory, and many other system properties.
The grains interface is made available to Salt modules and components so that the right salt minion commands are automatically available on the right systems.
Grain data is relatively static, though if system information changes (for example, if network settings are changed), or if a new value is assigned to a custom grain, grain data is refreshed.
NOTE: Grains resolve to lowercase letters. For example, FOO, and foo target the same grain.
Listing Grains Available grains can be listed by using the 'grains.ls' module:
salt '*' grains.ls
Grains data can be listed by using the 'grains.items' module:
salt '*' grains.items
Using grains in a state To use a grain in a state you can access it via {{ grains['key'] }}.
Grains in the Minion Config Grains can also be statically assigned within the minion configuration file.  Just add the option grains and pass options to it:
Then status data specific to your servers can be retrieved via Salt, or used inside of the State system for matching. It also makes it possible to target based on specific data about your deployment, as in the example above.
Grains in /etc/salt/grains If you do not want to place your custom static grains in the minion config file, you can also put them in /etc/salt/grains on the minion. They are configured in the same way as in the above example, only without a top-level grains: key:
roles: - webserver - memcache deployment: datacenter4 cabinet: 13 cab_u: 14-15
NOTE: Grains in /etc/salt/grains are ignored if you specify the same grains in the minion config.
NOTE: Grains are static, and since they are not often changed, they will need a grains refresh when they are updated. You can do this by calling: salt minion saltutil.refresh_modules
NOTE: You can equally configure static grains for Proxy Minions.  As multiple Proxy Minion processes can run on the same machine, you need to index the files using the Minion ID, under /etc/salt/proxy.d/<minion ID>/grains.  For example, the grains for the Proxy Minion router1 can be defined under /etc/salt/proxy.d/router1/grains, while the grains for the Proxy Minion switch7 can be put in /etc/salt/proxy.d/switch7/grains.
Matching Grains in the Top File With correctly configured grains on the Minion, the top file used in Pillar or during Highstate can be made very efficient. For example, consider the following configuration:
'roles:webserver': - match: grain - state0
'roles:memcache': - match: grain - state1 - state2
For this example to work, you would need to have defined the grain role for the minions you wish to match.
Writing Grains WARNING: Grains can be set by users that have access to the minion configuration files on the local system, making them less secure than other identifiers in Salt. Avoid storing sensitive data, such as passwords or keys, on minions. Instead, make use of pillar and/or sdb.
The grains are derived by executing all of the "public" functions (i.e. those which do not begin with an underscore) found in the modules located in the Salt's core grains code, followed by those in any custom grains modules. The functions in a grains module must return a Python dictionary, where the dictionary keys are the names of grains, and each key's value is that value for that grain.
Custom grains modules should be placed in a subdirectory named _grains located under the file_roots specified by the master config file. The default path would be /srv/salt/_grains. Custom grains modules will be distributed to the minions when state.highstate is run, or by executing the saltutil.sync_grains or saltutil.sync_all functions.
Grains modules are easy to write, and (as noted above) only need to return a dictionary. For example:
def yourfunction(): # initialize a grains dictionary grains = {} # Some code for logic that sets grains like grains["yourcustomgrain"] = True grains["anothergrain"] = "somevalue" return grains
The name of the function does not matter and will not factor into the grains data at all; only the keys/values returned become part of the grains.
When to Use a Custom Grain Before adding new grains, consider what the data is and remember that grains should (for the most part) be static data.
If the data is something that is likely to change, consider using Pillar or an execution module instead. If it's a simple set of key/value pairs, pillar is a good match. If compiling the information requires that system commands be run, then putting this information in an execution module is likely a better idea.
Good candidates for grains are data that is useful for targeting minions in the top file or the Salt CLI. The name and data structure of the grain should be designed to support many platforms, operating systems or applications. Also, keep in mind that Jinja templating in Salt supports referencing pillar data as well as invoking functions from execution modules, so there's no need to place information in grains to make it available to Jinja templates. For example:
... ... {{ salt['module.function_name']('argument_1', 'argument_2') }} {{ pillar['my_pillar_key'] }} ... ...
WARNING: Custom grains will not be available in the top file until after the first highstate. To make custom grains available on a minion's first highstate, it is recommended to use this example to ensure that the custom grains are synced when the minion starts.
Loading Custom Grains If you have multiple functions specifying grains that are called from a main function, be sure to prepend grain function names with an underscore. This prevents Salt from including the loaded grains from the grain functions in the final grain data structure. For example, consider this custom grain file:
#!/usr/bin/env python def _my_custom_grain(): my_grain = {"foo": "bar", "hello": "world"} return my_grain
def main(): # initialize a grains dictionary grains = {} grains["my_grains"] = _my_custom_grain() return grains
The output of this example renders like so: