neulab/tldr huggingface dataset:
• retrieve a specific field value from a Redis hash:
{{ salt.redis.hget('foo_hash', 'bar_field') }}
• get the routes to 0.0.0.0/0 using the NAPALM route:
{{ salt.route.show('0.0.0.0/0') }}
Tutorials Index Autoaccept minions from Grains New in version 2018.3.0.
To automatically accept minions based on certain characteristics, e.g. the uuid you can specify certain grain values on the salt master. Minions with matching grains will have their keys automatically accepted.
1. Configure the autosign_grains_dir in the master config file:
2. Configure the grain values to be accepted
Place a file named like the grain in the autosign_grains_dir and write the values that should be accepted automatically inside that file. For example to automatically accept minions based on their uuid create a file named /etc/salt/autosign_grains/uuid:
8f7d68e2-30c5-40c6-b84a-df7e978a03ee 1d3c5473-1fbc-479e-b0c7-877705a0730f
The master is now setup to accept minions with either of the two specified uuids.  Multiple values must always be written into separate lines.  Lines starting with a # are ignored.
3. Configure the minion to send the specific grains to the master in the minion config file:
autosign_grains: - uuid
Now you should be able to start salt-minion and run salt-call state.apply or any other salt commands that require master authentication.
Salt as a Cloud Controller In Salt 0.14.0, an advanced cloud control system was introduced, allowing private cloud VMs to be managed directly with Salt. This system is generally referred to as Salt Virt.
The Salt Virt system already exists and is installed within Salt itself. This means that besides setting up Salt, no additional salt code needs to be deployed.
NOTE: The libvirt python module and the certtool binary are required.
The main goal of Salt Virt is to facilitate a very fast and simple cloud that can scale and is fully featured. Salt Virt comes with the ability to set up and manage complex virtual machine networking, powerful image and disk management, and virtual machine migration with and without shared storage.
This means that Salt Virt can be used to create a cloud from a blade center and a SAN, but can also create a cloud out of a swarm of Linux Desktops without a single shared storage system. Salt Virt can make clouds from truly commodity hardware, but can also stand up the power of specialized hardware as well.
Setting up Hypervisors The first step to set up the hypervisors involves getting the correct software installed and setting up the hypervisor network interfaces.
Installing Hypervisor Software Salt Virt is made to be hypervisor agnostic but currently, the only fully implemented hypervisor is KVM via libvirt.
The required software for a hypervisor is libvirt and kvm. For advanced features, install libguestfs or qemu-nbd.
NOTE: Libguestfs and qemu-nbd allow for virtual machine images to be mounted before startup and get pre-seeded with configurations and a salt minion.
This sls will set up the needed software for a hypervisor, and run the routines to set up the libvirt pki keys.
NOTE: Package names and setup used is Red Hat specific. Different package names will be required for different platforms.
libvirt: pkg.installed: [] file.managed: - name: /etc/sysconfig/libvirtd - contents: 'LIBVIRTD_ARGS="--listen"' - require: - pkg: libvirt virt.keys: - require: - pkg: libvirt service.running: - name: libvirtd - require: - pkg: libvirt - network: br0 - libvirt: libvirt - watch: - file: libvirt
libvirt-python: pkg.installed: []
libguestfs: pkg.installed: - pkgs: - libguestfs - libguestfs-tools
Hypervisor Network Setup The hypervisors will need to be running a network bridge to serve up network devices for virtual machines. This formula will set up a standard bridge on a hypervisor connecting the bridge to eth0:
eth0: network.managed: - enabled: True - type: eth - bridge: br0
br0: network.managed: - enabled: True - type: bridge - proto: dhcp - require: - network: eth0
Virtual Machine Network Setup Salt Virt comes with a system to model the network interfaces used by the deployed virtual machines. By default, a single interface is created for the deployed virtual machine and is bridged to br0. To get going with the default networking setup, ensure that the bridge interface named br0 exists on the hypervisor and is bridged to an active network device.
NOTE: To use more advanced networking in Salt Virt, read the Salt Virt Networking document:
Salt Virt Networking
Libvirt State One of the challenges of deploying a libvirt based cloud is the distribution of libvirt certificates. These certificates allow for virtual machine migration. Salt comes with a system used to auto deploy these certificates.  Salt manages the signing authority key and generates keys for libvirt clients on the master, signs them with the certificate authority, and uses pillar to distribute them. This is managed via the libvirt state. Simply execute this formula on the minion to ensure that the certificate is in place and up to date:
NOTE: The above formula includes the calls needed to set up libvirt keys.
libvirt_keys: virt.keys
Getting Virtual Machine Images Ready Salt Virt requires that virtual machine images be provided as these are not generated on the fly. Generating these virtual machine images differs greatly based on the underlying platform.
Virtual machine images can be manually created using KVM and running through the installer, but this process is not recommended since it is very manual and prone to errors.
Virtual Machine generation applications are available for many platforms:
kiwi: (openSUSE, SLES, RHEL, CentOS) https://opensuse.github.io/kiwi/
vm-builder: https://wiki.debian.org/VMBuilder
SEE ALSO: vmbuilder-formula
Once virtual machine images are available, the easiest way to make them available to Salt Virt is to place them in the Salt file server. Just copy an image into /srv/salt and it can now be used by Salt Virt.
For purposes of this demo, the file name centos.img will be used.
Existing Virtual Machine Images Many existing Linux distributions distribute virtual machine images which can be used with Salt Virt. Please be advised that NONE OF THESE IMAGES ARE SUPPORTED BY SALTSTACK.
CentOS These images have been prepared for OpenNebula but should work without issue with Salt Virt, only the raw qcow image file is needed: https://wiki.centos.org/Cloud/OpenNebula
Fedora Linux Images for Fedora Linux can be found here: https://alt.fedoraproject.org/cloud
openSUSE https://download.opensuse.org/distribution/leap/15.1/jeos/openSUSE-Leap-15.1-JeOS.x86_64-15.1.0-kvm-and-xen-Current.qcow2.meta4
SUSE https://www.suse.com/products/server/jeos
Ubuntu Linux Images for Ubuntu Linux can be found here: http://cloud-images.ubuntu.com/
Using Salt Virt With hypervisors set up and virtual machine images ready, Salt can start issuing cloud commands using the virt runner.
Start by running a Salt Virt hypervisor info command:
salt-run virt.host_info
This will query the running hypervisor(s) for stats and display useful information such as the number of CPUs and amount of memory.
You can also list all VMs and their current states on all hypervisor nodes:
salt-run virt.list
Now that hypervisors are available a virtual machine can be provisioned, the virt.init routine will create a new virtual machine:
salt-run virt.init centos1 2 512 salt://centos.img
The Salt Virt runner will now automatically select a hypervisor to deploy the new virtual machine on. Using salt:// assumes that the CentOS virtual machine image is located in the root of the file-server on the master. When images are cloned (i.e. copied locally after retrieval from the file server), the destination directory on the hypervisor minion is determined by the virt:images config option; by default this is /srv/salt-images/.
When a VM is initialized using virt.init, the image is copied to the hypervisor using cp.cache_file and will be mounted and seeded with a minion.  Seeding includes setting pre-authenticated keys on the new machine. A minion will only be installed if one can not be found on the image using the default arguments to seed.apply.
NOTE: The biggest bottleneck in starting VMs is when the Salt Minion needs to be installed. Making sure that the source VM images already have Salt installed will GREATLY speed up virtual machine deployment.
You can also deploy an image on a particular minion by directly calling the virt execution module with an absolute image path. This can be quite handy for testing:
salt 'hypervisor*' virt.init centos1 2 512 image=/var/lib/libvirt/images/centos.img
Now that the new VM has been prepared, it can be seen via the virt.query command:
salt-run virt.query
This command will return data about all of the hypervisors and respective virtual machines.
Now that the new VM is booted, it should have contacted the Salt Master. A test.ping will reveal if the new VM is running.
QEMU Copy on Write Support For fast image cloning, you can use the qcow disk image format.  Pass the enable_qcow flag and a .qcow2 image path to virt.init:
salt 'hypervisor*' virt.init centos1 2 512 image=/var/lib/libvirt/images/centos.qcow2 enable_qcow=True start=False
NOTE: Beware that attempting to boot a qcow image too quickly after cloning can result in a race condition where libvirt may try to boot the machine before image seeding has completed. For that reason, it is recommended to also pass start=False to virt.init.
Also know that you must not modify the original base image without first making a copy and then rebasing all overlay images onto it.  See the qemu-img rebase usage docs.
Migrating Virtual Machines Salt Virt comes with full support for virtual machine migration. Using the libvirt state in the above formula makes migration possible.
A few things need to be available to support migration. Many operating systems turn on firewalls when originally set up; the firewall needs to be opened up to allow for libvirt and kvm to cross communicate and execution migration routines. On Red Hat based hypervisors in particular, port 16514 needs to be opened on hypervisors:
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 16514 -j ACCEPT
NOTE: More in-depth information regarding distribution specific firewall settings can be found in:
Opening the Firewall up for Salt
Salt also needs the virt:tunnel option to be turned on. This flag tells Salt to run migrations securely via the libvirt TLS tunnel and to use port 16514.  Without virt:tunnel, libvirt tries to bind to random ports when running migrations.
To turn on virt:tunnel, simply apply it to the master config file:
virt: tunnel: True
Once the master config has been updated, restart the master and send out a call to the minions to refresh the pillar to pick up on the change:
salt \* saltutil.refresh_modules
Now, migration routines can be run! To migrate a VM, simply run the Salt Virt migrate routine:
salt-run virt.migrate centos <new hypervisor>
VNC Consoles Although not enabled by default, Salt Virt can also set up VNC consoles allowing for remote visual consoles to be opened up. When creating a new VM using virt.init, pass the enable_vnc=True parameter to have a console configured for the new VM.
The information from a virt.query routine will display the VNC console port for the specific VMs:
centos CPU: 2 Memory: 524288 State: running Graphics: vnc - hyper6:5900 Disk - vda: Size: 2.0G File: /srv/salt-images/ubuntu2/system.qcow2 File Format: qcow2 Nic - ac:de:48:98:08:77: Source: br0 Type: bridge
The line Graphics: vnc - hyper6:5900 holds the key. First the port named, in this case 5900, will need to be available in the hypervisor's firewall.  Once the port is open, then the console can be easily opened via vncviewer:
vncviewer hyper6:5900
By default there is no VNC security set up on these ports, which suggests that keeping them firewalled and mandating that SSH tunnels be used to access these VNC interfaces. Keep in mind that activity on a VNC interface that is accessed can be viewed by any other user that accesses that same VNC interface, and any other user logging in can also operate with the logged in user on the virtual machine.
Conclusion Now with Salt Virt running, new hypervisors can be seamlessly added just by running the above states on new bare metal machines, and these machines will be instantly available to Salt Virt.
Running Salt States and Commands in Docker Containers The 2016.11.0 release of Salt introduces the ability to execute Salt States and Salt remote execution commands directly inside of Docker containers.
This addition makes it possible to not only deploy fresh containers using Salt States. This also allows for running containers to be audited and modified using Salt, but without running a Salt Minion inside the container.  Some of the applications include security audits of running containers as well as gathering operating data from containers.
This new feature is simple and straightforward, and can be used via a running Salt Minion, the Salt Call command, or via Salt SSH. For this tutorial we will use the salt-call command, but like all salt commands these calls are directly translatable to salt and salt-ssh.
Step 1 - Install Docker Since setting up Docker is well covered in the Docker documentation we will make no such effort to describe it here. Please see the Docker Installation Documentation for installing and setting up Docker: https://docs.docker.com/engine/installation/
The Docker integration also requires that the docker-py library is installed.  This can easily be done using pip or via your system package manager:
pip install docker-py
Step 2 - Install Salt For this tutorial we will be using Salt Call, which is available in the salt-minion package, please follow the Salt Installation docs found here: https://repo.saltproject.io/
Step 3 - Create With Salt States Next some Salt States are needed, for this example a very basic state which installs vim is used, but anything Salt States can do can be done here, please see the Salt States Introduction Tutorial to learn more about Salt States: https://docs.saltproject.io/en/stage/getstarted/config/
For this tutorial, simply create a small state file in /srv/salt/vim.sls:
vim: pkg.installed
NOTE: The base image you choose will need to have python 2.6 or 2.7 installed.  We are hoping to resolve this constraint in a future release.
If base is omitted the default image used is a minimal openSUSE image with Python support, maintained by SUSE
Next run the docker.sls_build command:
salt-call --local dockerng.sls_build test base=my_base_image mods=vim
Now we have a fresh image called test to work with and vim has been installed.
Step 4 - Running Commands Inside the Container Salt can now run remote execution functions inside the container with another simple salt-call command:
salt-call --local dockerng.call test test.version salt-call --local dockerng.call test network.interfaces salt-call --local dockerng.call test disk.usage salt-call --local dockerng.call test pkg.list_pkgs salt-call --local dockerng.call test service.running httpd salt-call --local dockerng.call test cmd.run 'ls -l /etc'
Automatic Updates / Frozen Deployments New in version 0.10.3.d.
Salt has support for the Esky application freezing and update tool. This tool allows one to build a complete zipfile out of the salt scripts and all their dependencies - including shared objects / DLLs.
Getting Started To build frozen applications, suitable build environment will be needed for each platform. You should probably set up a virtualenv in order to limit the scope of Q/A.
This process does work on Windows. Directions are available at https://github.com/saltstack/salt-windows-install for details on installing Salt in Windows. Only the 32-bit Python and dependencies have been tested, but they have been tested on 64-bit Windows.
Install bbfreeze, and then esky from PyPI in order to enable the bdist_esky command in setup.py. Salt itself must also be installed, in addition to its dependencies.
Building and Freezing Once you have your tools installed and the environment configured, use setup.py to prepare the distribution files.
python setup.py sdist python setup.py bdist
Once the distribution files are in place, Esky can be used traverse the module tree and pack all the scripts up into a redistributable.
python setup.py bdist_esky
There will be an appropriately versioned salt-VERSION.zip in dist/ if everything went smoothly.
Windows C:\Python27\lib\site-packages\zmq will need to be added to the PATH variable. This helps bbfreeze find the zmq DLL so it can pack it up.
Using the Frozen Build Unpack the zip file in the desired install location. Scripts like salt-minion and salt-call will be in the root of the zip file. The associated libraries and bootstrapping will be in the directories at the same level. (Check the Esky documentation for more information)
To support updating your minions in the wild, put the builds on a web server that the minions can reach. salt.modules.saltutil.update() will trigger an update and (optionally) a restart of the minion service under the new version.
Troubleshooting A Windows minion isn't responding The process dispatch on Windows is slower than it is on *nix. It may be necessary to add '-t 15' to salt commands to give minions plenty of time to return.
Windows and the Visual Studio Redist The Visual C++ 2008 32-bit redistributable will need to be installed on all Windows minions. Esky has an option to pack the library into the zipfile, but OpenSSL does not seem to acknowledge the new location. If a no OPENSSL_Applink error appears on the console when trying to start a frozen minion, the redistributable is not installed.
Mixed Linux environments and Yum The Yum Python module doesn't appear to be available on any of the standard Python package mirrors. If RHEL/CentOS systems need to be supported, the frozen build should created on that platform to support all the Linux nodes. Remember to build the virtualenv with --system-site-packages so that the yum module is included.
Automatic (Python) module discovery Automatic (Python) module discovery does not work with the late-loaded scheme that Salt uses for (Salt) modules. Any misbehaving modules will need to be explicitly added to the freezer_includes in Salt's setup.py. Always check the zipped application to make sure that the necessary modules were included.
ESXi Proxy Minion New in version 2015.8.4.
NOTE: This tutorial assumes basic knowledge of Salt. To get up to speed, check out the Salt Walkthrough.
This tutorial also assumes a basic understanding of Salt Proxy Minions. If you're unfamiliar with Salt's Proxy Minion system, please read the Salt Proxy Minion documentation and the Salt Proxy Minion End-to-End Example tutorial.
The third assumption that this tutorial makes is that you also have a basic understanding of ESXi hosts. You can learn more about ESXi hosts on VMware's various resources.
Salt's ESXi Proxy Minion allows a VMware ESXi host to be treated as an individual Salt Minion, without installing a Salt Minion on the ESXi host.
Since an ESXi host may not necessarily run on an OS capable of hosting a Python stack, the ESXi host can't run a regular Salt Minion directly. Therefore, Salt's Proxy Minion functionality enables you to designate another machine to host a proxy process that "proxies" communication from the Salt Master to the ESXi host.  The master does not know or care that the ESXi target is not a "real" Salt Minion.
More in-depth conceptual reading on Proxy Minions can be found in the Proxy Minion section of Salt's documentation.
Salt's ESXi Proxy Minion was added in the 2015.8.4 release of Salt.
NOTE: Be aware that some functionality for the ESXi Proxy Minion may depend on the type of license attached the ESXi host(s).
For example, certain services are only available to manipulate service state or policies with a VMware vSphere Enterprise or Enterprise Plus license, while others are available with a Standard license. The ntpd service is restricted to an Enterprise Plus license, while ssh is available via the Standard license.
Please see the vSphere Comparison page for more information.
Dependencies Manipulation of the ESXi host via a Proxy Minion requires the machine running the Proxy Minion process to have the ESXCLI package (and all of its dependencies) and the pyVmomi Python Library to be installed.
ESXi Password The ESXi Proxy Minion uses VMware's API to perform tasks on the host as if it was a regular Salt Minion. In order to access the API that is already running on the ESXi host, the ESXi host must have a username and password that is used to log into the host. The username is usually root. Before Salt can access the ESXi host via VMware's API, a default password must be set on the host.
pyVmomi The pyVmomi Python library must be installed on the machine that is running the proxy process. pyVmomi can be installed via pip:
pip install pyVmomi
NOTE: Version 6.0 of pyVmomi has some problems with SSL error handling on certain versions of Python. If using version 6.0 of pyVmomi, the machine that you are running the proxy minion process from must have either Python 2.6, Python 2.7.9, or newer. This is due to an upstream dependency in pyVmomi 6.0 that is not supported in Python version 2.7 to 2.7.8. If the version of Python running the proxy process is not in the supported range, you will need to install an earlier version of pyVmomi. See Issue #29537 for more information.
Based on the note above, to install an earlier version of pyVmomi than the version currently listed in PyPi, run the following:
pip install pyVmomi==5.5.0.2014.1.1
The 5.5.0.2014.1.1 is a known stable version that the original ESXi Proxy Minion was developed against.
ESXCLI Currently, about a third of the functions used for the ESXi Proxy Minion require the ESXCLI package be installed on the machine running the Proxy Minion process.
The ESXCLI package is also referred to as the VMware vSphere CLI, or vCLI. VMware provides vCLI package installation instructions for vSphere 5.5 and vSphere 6.0.
Once all of the required dependencies are in place and the vCLI package is installed, you can check to see if you can connect to your ESXi host by running the following command:
esxcli -s <host-location> -u <username> -p <password> system syslog config get
If the connection was successful, ESXCLI was successfully installed on your system.  You should see output related to the ESXi host's syslog configuration.
Configuration There are several places where various configuration values need to be set in order for the ESXi Proxy Minion to run and connect properly.
Proxy Config File On the machine that will be running the Proxy Minion process(es), a proxy config file must be in place. This file should be located in the /etc/salt/ directory and should be named proxy. If the file is not there by default, create it.
This file should contain the location of your Salt Master that the Salt Proxy will connect to.
Example Proxy Config File:
# /etc/salt/proxy
master: <salt-master-location>
Pillar Profiles Proxy minions get their configuration from Salt's Pillar. Every proxy must have a stanza in Pillar and a reference in the Pillar top-file that matches the Proxy ID. At a minimum for communication with the ESXi host, the pillar should look like this:
proxy: proxytype: esxi host: <ip or dns name of esxi host> username: <ESXi username> passwords: - first_password - second_password - third_password
Some other optional settings are protocol and port. These can be added to the pillar configuration.
proxytype The proxytype key and value pair is critical, as it tells Salt which interface to load from the proxy directory in Salt's install hierarchy, or from /srv/salt/_proxy on the Salt Master (if you have created your own proxy module, for example). To use this ESXi Proxy Module, set this to esxi.
host The location, or ip/dns, of the ESXi host. Required.
username The username used to login to the ESXi host, such as root. Required.
passwords A list of passwords to be used to try and login to the ESXi host. At least one password in this list is required.
The proxy integration will try the passwords listed in order. It is configured this way so you can have a regular password and the password you may be updating for an ESXi host either via the vsphere.update_host_password execution module function or via the esxi.password_present state function. This way, after the password is changed, you should not need to restart the proxy minion--it should just pick up the new password provided in the list. You can then change pillar at will to move that password to the front and retire the unused ones.
Use-case/reasoning for using a list of passwords: You are setting up an ESXi host for the first time, and the host comes with a default password. You know that you'll be changing this password during your initial setup from the default to a new password. If you only have one password option, and if you have a state changing the password, any remote execution commands or states that run after the password change will not be able to run on the host until the password is updated in Pillar and the Proxy Minion process is restarted.
This allows you to use any number of potential fallback passwords.
NOTE: When a password is changed on the host to one in the list of possible passwords, the further down on the list the password is, the longer individual commands will take to return. This is due to the nature of pyVmomi's login system. We have to wait for the first attempt to fail before trying the next password on the list.
This scenario is especially true, and even slower, when the proxy minion first starts. If the correct password is not the first password on the list, it may take up to a minute for test.version to respond with salt's version installed (Example: 2018.3.4. Once the initial authorization is complete, the responses for commands will be a little faster.
To avoid these longer waiting periods, SaltStack recommends moving the correct password to the top of the list and restarting the proxy minion at your earliest convenience.
protocol If the ESXi host is not using the default protocol, set this value to an alternate protocol. Default is https. For example:
port If the ESXi host is not using the default port, set this value to an alternate port. Default is 443.
Example Configuration Files An example of all of the basic configurations that need to be in place before starting the Proxy Minion processes includes the Proxy Config File, Pillar Top File, and any individual Proxy Minion Pillar files.
In this example, we'll assuming there are two ESXi hosts to connect to. Therefore, we'll be creating two Proxy Minion config files, one config for each ESXi host.
Proxy Config File:
Pillar Top File:
# /srv/pillar/top.sls
base: 'esxi-1': - esxi-1 'esxi-2': - esxi-2
Pillar Config File for the first ESXi host, esxi-1:
# /srv/pillar/esxi-1.sls
proxy: proxytype: esxi host: esxi-1.example.com username: 'root' passwords: - bad-password-1 - backup-bad-password-1
Pillar Config File for the second ESXi host, esxi-2:
# /srv/pillar/esxi-2.sls
proxy: proxytype: esxi host: esxi-2.example.com username: 'root' passwords: - bad-password-2 - backup-bad-password-2
Starting the Proxy Minion Once all of the correct configuration files are in place, it is time to start the proxy processes!
1. First, make sure your Salt Master is running.
2. Start the first Salt Proxy, in debug mode, by giving the Proxy Minion process and ID that matches the config file name created in the Configuration section.
salt-proxy --proxyid='esxi-1' -l debug
1. Accept the esxi-1 Proxy Minion's key on the Salt Master:
# salt-key -L Accepted Keys: Denied Keys: Unaccepted Keys: esxi-1 Rejected Keys: # # salt-key -a esxi-1 The following keys are going to be accepted: Unaccepted Keys: esxi-1 Proceed? [n/Y] y Key for minion esxi-1 accepted.
1. Repeat for the second Salt Proxy, this time we'll run the proxy process as a daemon, as an example.
salt-proxy --proxyid='esxi-2' -d
1. Accept the esxi-2 Proxy Minion's key on the Salt Master:
# salt-key -L Accepted Keys: esxi-1 Denied Keys: Unaccepted Keys: esxi-2 Rejected Keys: # # salt-key -a esxi-1 The following keys are going to be accepted: Unaccepted Keys: esxi-2 Proceed? [n/Y] y Key for minion esxi-1 accepted.
1. Check and see if your Proxy Minions are responding:
# salt 'esxi-*' test.version esxi-1: True esxi-3: True
Executing Commands Now that you've configured your Proxy Minions and have them responding successfully to a test.version, we can start executing commands against the ESXi hosts via Salt.
It's important to understand how this particular proxy works, and there are a couple of important pieces to be aware of in order to start running remote execution and state commands against the ESXi host via a Proxy Minion: the vSphere Execution Module, the ESXi Execution Module, and the ESXi State Module.
vSphere Execution Module The Salt.modules.vsphere is a standard Salt execution module that does the bulk of the work for the ESXi Proxy Minion. If you pull up the docs for it you'll see that almost every function in the module takes credentials (username and password) and a target host argument. When credentials and a host aren't passed, Salt runs commands through pyVmomi or ESXCLI against the local machine. If you wanted, you could run functions from this module on any machine where an appropriate version of pyVmomi and ESXCLI are installed, and that machine would reach out over the network and communicate with the ESXi host.
You'll notice that most of the functions in the vSphere module require a host, username, and password. These parameters are contained in the Pillar files and passed through to the function via the proxy process that is already running. You don't need to provide these parameters when you execute the commands. See the Running Remote Execution Commands section below for an example.
ESXi Execution Module In order for the Pillar information set up in the Configuration section above to be passed to the function call in the vSphere Execution Module, the salt.modules.esxi execution module acts as a "shim" between the vSphere execution module functions and the proxy process.
The "shim" takes the authentication credentials specified in the Pillar files and passes them through to the host, username, password, and optional protocol and port options required by the vSphere Execution Module functions.
If the function takes more positional, or keyword, arguments you can append them to the call. It's this shim that speaks to the ESXi host through the proxy, arranging for the credentials and hostname to be pulled from the Pillar section for the ESXi Proxy Minion.
Because of the presence of the shim, to lookup documentation for what functions you can use to interface with the ESXi host, you'll want to look in salt.modules.vsphere instead of salt.modules.esxi.
Running Remote Execution Commands To run commands from the Salt Master to execute, via the ESXi Proxy Minion, against the ESXi host, you use the esxi.cmd <vsphere-function-name> syntax to call functions located in the vSphere Execution Module. Both args and kwargs needed for various vsphere execution module functions must be passed through in a kwarg- type manor. For example:
salt 'esxi-*' esxi.cmd system_info salt 'exsi-*' esxi.cmd get_service_running service_name='ssh'
ESXi State Module The ESXi State Module functions similarly to other state modules. The "shim" provided by the ESXi Execution Module passes the necessary host, username, and password credentials through, so those options don't need to be provided in the state. Other than that, state files are written and executed just like any other Salt state. See the salt.modules.esxi state for ESXi state functions.
The follow state file is an example of how to configure various pieces of an ESXi host including enabling SSH, uploading and SSH key, configuring a coredump network config, syslog, ntp, enabling VMotion, resetting a host password, and more.
# /srv/salt/configure-esxi.sls
configure-host-ssh: esxi.ssh_configured: - service_running: True - ssh_key_file: /etc/salt/ssh_keys/my_key.pub - service_policy: 'automatic' - service_restart: True - certificate_verify: True
configure-host-coredump: esxi.coredump_configured: - enabled: True - dump_ip: 'my-coredump-ip.example.com'
configure-host-syslog: esxi.syslog_configured: - syslog_configs: loghost: ssl://localhost:5432,tcp://10.1.0.1:1514 default-timeout: 120 - firewall: True - reset_service: True - reset_syslog_config: True - reset_configs: loghost,default-timeout
configure-host-ntp: esxi.ntp_configured: - service_running: True - ntp_servers: - 192.174.1.100 - 192.174.1.200 - service_policy: 'automatic' - service_restart: True
configure-vmotion: esxi.vmotion_configured: - enabled: True
configure-host-vsan: esxi.vsan_configured: - enabled: True - add_disks_to_vsan: True
configure-host-password: esxi.password_present: - password: 'new-bad-password'
States are called via the ESXi Proxy Minion just as they would on a regular minion.  For example:
salt 'esxi-*' state.sls configure-esxi test=true salt 'esxi-*' state.sls configure-esxi
Relevant Salt Files and Resources • ESXi Proxy Minion
• ESXi Execution Module
• ESXi State Module
• Salt Proxy Minion Docs
• Salt Proxy Minion End-to-End Example
• vSphere Execution Module
HTTP Modules This tutorial demonstrates using the various HTTP modules available in Salt.  These modules wrap the Python tornado, urllib2, and requests libraries, extending them in a manner that is more consistent with Salt workflows.
The salt.utils.http Library This library forms the core of the HTTP modules. Since it is designed to be used from the minion as an execution module, in addition to the master as a runner, it was abstracted into this multi-use library. This library can also be imported by 3rd-party programs wishing to take advantage of its extended functionality.
Core functionality of the execution, state, and runner modules is derived from this library, so common usages between them are described here. Documentation specific to each module is described below.
This library can be imported with:
import salt.utils.http
Configuring Libraries This library can make use of either tornado, which is required by Salt, urllib2, which ships with Python, or requests, which can be installed separately. By default, tornado will be used. In order to switch to urllib2, set the following variable:
backend: urllib2
In order to switch to requests, set the following variable:
backend: requests
This can be set in the master or minion configuration file, or passed as an option directly to any http.query() functions.
salt.utils.http.query() This function forms a basic query, but with some add-ons not present in the tornado, urllib2, and requests libraries. Not all functionality currently available in these libraries has been added, but can be in future iterations.
HTTPS Request Methods A basic query can be performed by calling this function with no more than a single URL:
salt.utils.http.query("http://example.com")
By default the query will be performed with a GET method. The method can be overridden with the method argument:
salt.utils.http.query("http://example.com/delete/url", "DELETE")
When using the POST method (and others, such as PUT), extra data is usually sent as well. This data can be sent directly (would be URL encoded when necessary), or in whatever format is required by the remote server (XML, JSON, plain text, etc).
salt.utils.http.query( "http://example.com/post/url", method="POST", data=json.dumps(mydict) )
Data Formatting and Templating Bear in mind that the data must be sent pre-formatted; this function will not format it for you. However, a templated file stored on the local system may be passed through, along with variables to populate it with. To pass through only the file (untemplated):
salt.utils.http.query( "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.xml" )
To pass through a file that contains jinja + yaml templating (the default):
salt.utils.http.query( "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.jinja", data_render=True, template_dict={"key1": "value1", "key2": "value2"}, )
To pass through a file that contains mako templating:
salt.utils.http.query( "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.mako", data_render=True, data_renderer="mako", template_dict={"key1": "value1", "key2": "value2"}, )
Because this function uses Salt's own rendering system, any Salt renderer can be used. Because Salt's renderer requires __opts__ to be set, an opts dictionary should be passed in. If it is not, then the default __opts__ values for the node type (master or minion) will be used. Because this library is intended primarily for use by minions, the default node type is minion.  However, this can be changed to master if necessary.
salt.utils.http.query( "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.jinja", data_render=True, template_dict={"key1": "value1", "key2": "value2"}, opts=__opts__, )
salt.utils.http.query( "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.jinja", data_render=True, template_dict={"key1": "value1", "key2": "value2"}, node="master", )
Headers Headers may also be passed through, either as a header_list, a header_dict, or as a header_file. As with the data_file, the header_file may also be templated. Take note that because HTTP headers are normally syntactically-correct YAML, they will automatically be imported as an a Python dict.
salt.utils.http.query( "http://example.com/delete/url", method="POST", header_file="/srv/salt/headers.jinja", header_render=True, header_renderer="jinja", template_dict={"key1": "value1", "key2": "value2"}, )
Because much of the data that would be templated between headers and data may be the same, the template_dict is the same for both. Correcting possible variable name collisions is up to the user.
Authentication The query() function supports basic HTTP authentication. A username and password may be passed in as username and password, respectively.
salt.utils.http.query("http://example.com", username="larry", password="5700g3543v4r")
Cookies and Sessions Cookies are also supported, using Python's built-in cookielib. However, they are turned off by default. To turn cookies on, set cookies to True.
salt.utils.http.query("http://example.com", cookies=True)
By default cookies are stored in Salt's cache directory, normally /var/cache/salt, as a file called cookies.txt. However, this location may be changed with the cookie_jar argument:
salt.utils.http.query( "http://example.com", cookies=True, cookie_jar="/path/to/cookie_jar.txt" )
By default, the format of the cookie jar is LWP (aka, lib-www-perl). This default was chosen because it is a human-readable text file. If desired, the format of the cookie jar can be set to Mozilla:
salt.utils.http.query( "http://example.com", cookies=True, cookie_jar="/path/to/cookie_jar.txt", cookie_format="mozilla", )
Because Salt commands are normally one-off commands that are piped together, this library cannot normally behave as a normal browser, with session cookies that persist across multiple HTTP requests. However, the session can be persisted in a separate cookie jar. The default filename for this file, inside Salt's cache directory, is cookies.session.p. This can also be changed.
salt.utils.http.query( "http://example.com", persist_session=True, session_cookie_jar="/path/to/jar.p" )
The format of this file is msgpack, which is consistent with much of the rest of Salt's internal structure. Historically, the extension for this file is .p. There are no current plans to make this configurable.
Proxy If the tornado backend is used (tornado is the default), proxy information configured in proxy_host, proxy_port, proxy_username, proxy_password and no_proxy from the __opts__ dictionary will be used. Normally these are set in the minion configuration file.
proxy_host: proxy.my-domain proxy_port: 31337 proxy_username: charon proxy_password: obolus no_proxy: ['127.0.0.1', 'localhost']
salt.utils.http.query("http://example.com", opts=__opts__, backend="tornado")
Return Data NOTE: Return data encoding
If decode is set to True, query() will attempt to decode the return data. decode_type defaults to auto.  Set it to a specific encoding, xml, for example, to override autodetection.
Because Salt's http library was designed to be used with REST interfaces, query() will attempt to decode the data received from the remote server when decode is set to True.  First it will check the Content-type header to try and find references to XML. If it does not find any, it will look for references to JSON. If it does not find any, it will fall back to plain text, which will not be decoded.
JSON data is translated into a dict using Python's built-in json library. XML is translated using salt.utils.xml_util, which will use Python's built-in XML libraries to attempt to convert the XML into a dict. In order to force either JSON or XML decoding, the decode_type may be set:
salt.utils.http.query("http://example.com", decode_type="xml")
Once translated, the return dict from query() will include a dict called dict.
If the data is not to be translated using one of these methods, decoding may be turned off.
salt.utils.http.query("http://example.com", decode=False)
If decoding is turned on, and references to JSON or XML cannot be found, then this module will default to plain text, and return the undecoded data as text (even if text is set to False; see below).
The query() function can return the HTTP status code, headers, and/or text as required. However, each must individually be turned on.
salt.utils.http.query("http://example.com", status=True, headers=True, text=True)
The return from these will be found in the return dict as status, headers and text, respectively.
Writing Return Data to Files It is possible to write either the return data or headers to files, as soon as the response is received from the server, but specifying file locations via the text_out or headers_out arguments. text and headers do not need to be returned to the user in order to do this.
salt.utils.http.query( "http://example.com", text=False, headers=False, text_out="/path/to/url_download.txt", headers_out="/path/to/headers_download.txt", )
SSL Verification By default, this function will verify SSL certificates. However, for testing or debugging purposes, SSL verification can be turned off.
salt.utils.http.query("https://example.com", verify_ssl=False)
CA Bundles The requests library has its own method of detecting which CA (certificate authority) bundle file to use. Usually this is implemented by the packager for the specific operating system distribution that you are using. However, urllib2 requires a little more work under the hood. By default, Salt will try to auto-detect the location of this file. However, if it is not in an expected location, or a different path needs to be specified, it may be done so using the ca_bundle variable.
salt.utils.http.query("https://example.com", ca_bundle="/path/to/ca_bundle.pem")
Updating CA Bundles The update_ca_bundle() function can be used to update the bundle file at a specified location. If the target location is not specified, then it will attempt to auto-detect the location of the bundle file. If the URL to download the bundle from does not exist, a bundle will be downloaded from the cURL website.
CAUTION: The target and the source should always be specified! Failure to specify the target may result in the file being written to the wrong location on the local system. Failure to specify the source may cause the upstream URL to receive excess unnecessary traffic, and may cause a file to be download which is hazardous or does not meet the needs of the user.
salt.utils.http.update_ca_bundle( target="/path/to/ca-bundle.crt", source="https://example.com/path/to/ca-bundle.crt", opts=__opts__, )
The opts parameter should also always be specified. If it is, then the target and the source may be specified in the relevant configuration file (master or minion) as ca_bundle and ca_bundle_url, respectively.
ca_bundle: /path/to/ca-bundle.crt ca_bundle_url: https://example.com/path/to/ca-bundle.crt
If Salt is unable to auto-detect the location of the CA bundle, it will raise an error.
The update_ca_bundle() function can also be passed a string or a list of strings which represent files on the local system, which should be appended (in the specified order) to the end of the CA bundle file. This is useful in environments where private certs need to be made available, and are not otherwise reasonable to add to the bundle file.
salt.utils.http.update_ca_bundle( opts=__opts__, merge_files=[ "/etc/ssl/private_cert_1.pem", "/etc/ssl/private_cert_2.pem", "/etc/ssl/private_cert_3.pem", ], )
Test Mode This function may be run in test mode. This mode will perform all work up until the actual HTTP request. By default, instead of performing the request, an empty dict will be returned. Using this function with TRACE logging turned on will reveal the contents of the headers and POST data to be sent.
Rather than returning an empty dict, an alternate test_url may be passed in.  If this is detected, then test mode will replace the url with the test_url, set test to True in the return data, and perform the rest of the requested operations as usual. This allows a custom, non-destructive URL to be used for testing when necessary.
Execution Module The http execution module is a very thin wrapper around the salt.utils.http library. The opts can be passed through as well, but if they are not specified, the minion defaults will be used as necessary.
Because passing complete data structures from the command line can be tricky at best and dangerous (in terms of execution injection attacks) at worse, the data_file, and header_file are likely to see more use here.
All methods for the library are available in the execution module, as kwargs.
salt myminion http.query http://example.com/restapi method=POST \ username='larry' password='5700g3543v4r' headers=True text=True \ status=True decode_type=xml data_render=True \ header_file=/tmp/headers.txt data_file=/tmp/data.txt \ header_render=True cookies=True persist_session=True
Runner Module Like the execution module, the http runner module is a very thin wrapper around the salt.utils.http library. The only significant difference is that because runners execute on the master instead of a minion, a target is not required, and default opts will be derived from the master config, rather than the minion config.
All methods for the library are available in the runner module, as kwargs.
salt-run http.query http://example.com/restapi method=POST \ username='larry' password='5700g3543v4r' headers=True text=True \ status=True decode_type=xml data_render=True \ header_file=/tmp/headers.txt data_file=/tmp/data.txt \ header_render=True cookies=True persist_session=True
State Module The state module is a wrapper around the runner module, which applies stateful logic to a query. All kwargs as listed above are specified as usual in state files, but two more kwargs are available to apply stateful logic. A required parameter is match, which specifies a pattern to look for in the return text. By default, this will perform a string comparison of looking for the value of match in the return text. In Python terms this looks like:
def myfunc(): if match in html_text: return True
If more complex pattern matching is required, a regular expression can be used by specifying a match_type. By default this is set to string, but it can be manually set to pcre instead. Please note that despite the name, this will use Python's re.search() rather than re.match().
Therefore, the following states are valid:
http://example.com/restapi: http.query: - match: 'SUCCESS' - username: 'larry' - password: '5700g3543v4r' - data_render: True - header_file: /tmp/headers.txt - data_file: /tmp/data.txt - header_render: True - cookies: True - persist_session: True
http://example.com/restapi: http.query: - match_type: pcre - match: '(?i)succe[ss|ed]' - username: 'larry' - password: '5700g3543v4r' - data_render: True - header_file: /tmp/headers.txt - data_file: /tmp/data.txt - header_render: True - cookies: True - persist_session: True
In addition to, or instead of a match pattern, the status code for a URL can be checked. This is done using the status argument:
http://example.com/: http.query: - status: 200
If both are specified, both will be checked, but if only one is True and the other is False, then False will be returned. In this case, the comments in the return data will contain information for troubleshooting.