neulab/tldr huggingface dataset:
The provider parameter in cloud provider configuration was renamed to driver.  This change was made to avoid confusion with the provider parameter that is used in cloud profile configuration. Cloud provider configuration now uses driver to refer to the salt-cloud driver that provides the underlying functionality to connect to a cloud provider, while cloud profile configuration continues to use provider to refer to the cloud provider configuration that you define.
Profiles Set up an initial profile at /etc/salt/cloud.profiles or /etc/salt/cloud.profiles.d/vmware.conf:
vmware-centos6.5: provider: vcenter01 clonefrom: test-vm
## Optional arguments num_cpus: 4 memory: 8GB devices: cd: CD/DVD drive 1: device_type: datastore_iso_file iso_path: "[nap004-1] vmimages/tools-isoimages/linux.iso" CD/DVD drive 2: device_type: client_device mode: atapi controller: IDE 2 CD/DVD drive 3: device_type: client_device mode: passthrough controller: IDE 3 disk: Hard disk 1: size: 30 Hard disk 2: size: 20 controller: SCSI controller 2 Hard disk 3: size: 5 controller: SCSI controller 3 datastore: smalldiskdatastore network: Network adapter 1: name: 10.20.30-400-Test switch_type: standard ip: 10.20.30.123 gateway: [10.20.30.110] subnet_mask: 255.255.255.128 domain: example.com Network adapter 2: name: 10.30.40-500-Dev-DHCP adapter_type: e1000 switch_type: distributed mac: '00:16:3e:e8:19:0f' Network adapter 3: name: 10.40.50-600-Prod adapter_type: vmxnet3 switch_type: distributed ip: 10.40.50.123 gateway: [10.40.50.110] subnet_mask: 255.255.255.128 domain: example.com scsi: SCSI controller 1: type: lsilogic SCSI controller 2: type: lsilogic_sas bus_sharing: virtual SCSI controller 3: type: paravirtual bus_sharing: physical ide: IDE 2: {} IDE 3: {}
domain: example.com dns_servers: - 123.127.255.240 - 123.127.255.241 - 123.127.255.242
resourcepool: Resources cluster: Prod
datastore: HUGE-DATASTORE-Cluster folder: Development datacenter: DC1 host: c4212n-002.domain.com template: False power_on: True extra_config: mem.hotadd: 'yes' guestinfo.foo: bar guestinfo.domain: foobar.com guestinfo.customVariable: customValue annotation: Created by Salt-Cloud
deploy: True customization: True private_key: /root/.ssh/mykey.pem ssh_username: cloud-user password: veryVeryBadPassword minion: master: 123.127.193.105
file_map: /path/to/local/custom/script: /path/to/remote/script /path/to/local/file: /path/to/remote/file /srv/salt/yum/epel.repo: /etc/yum.repos.d/epel.repo
hardware_version: 10 image: centos64Guest
#For Windows VM win_username: Administrator win_password: administrator win_organization_name: ABC-Corp plain_text: True win_installer: /root/Salt-Minion-2015.8.4-AMD64-Setup.exe win_user_fullname: Windows User
provider Enter the name that was specified when the cloud provider config was created.
clonefrom Enter the name of the VM/template to clone from. If not specified, the VM will be created without cloning.
num_cpus Enter the number of vCPUS that you want the VM/template to have. If not specified, the current VM/template's vCPU count is used.
cores_per_socket Enter the number of cores per vCPU that you want the VM/template to have. If not specified, this will default to 1.
NOTE: Cores per socket should be less than or equal to the total number of vCPUs assigned to the VM/template.
memory Enter the memory size (in MB or GB) that you want the VM/template to have. If not specified, the current VM/template's memory size is used. Example memory: 8GB or memory: 8192MB.
devices Enter the device specifications here. Currently, the following devices can be created or reconfigured:
cd     Enter the CD/DVD drive specification here. If the CD/DVD drive doesn't exist, it will be created with the specified configuration. If the CD/DVD drive already exists, it will be reconfigured with the specifications. The following options can be specified per CD/DVD drive:
device_type Specify how the CD/DVD drive should be used. Currently supported types are client_device and datastore_iso_file. Default is device_type: client_device
iso_path Enter the path to the iso file present on the datastore only if device_type: datastore_iso_file. The syntax to specify this is iso_path: "[datastoreName] vmimages/tools-isoimages/linux.iso". This field is ignored if device_type: client_device
mode   Enter the mode of connection only if device_type: client_device. Currently supported modes are passthrough and atapi. This field is ignored if device_type: datastore_iso_file. Default is mode: passthrough
controller Specify the IDE controller label to which this drive should be attached.  This should be specified only when creating both the specified IDE controller as well as the CD/DVD drive at the same time.
disk   Enter the disk specification here. If the hard disk doesn't exist, it will be created with the provided size. If the hard disk already exists, it will be expanded if the provided size is greater than the current size of the disk.
size   Enter the size of disk in GB
thin_provision Specifies whether the disk should be thin provisioned or not. Default is thin_provision: False.  .. versionadded:: 2016.3.0
eagerly_scrub Specifies whether the disk should be rewrite with zeros during thick provisioning or not.  Default is eagerly_scrub: False.  .. versionadded:: 2018.3.0
controller Specify the SCSI controller label to which this disk should be attached.  This should be specified only when creating both the specified SCSI controller as well as the hard disk at the same time.
datastore The name of a valid datastore should you wish the new disk to be in a datastore other than the default for the VM.
network Enter the network adapter specification here. If the network adapter doesn't exist, a new network adapter will be created with the specified network name, type and other configuration. If the network adapter already exists, it will be reconfigured with the specifications. The following additional options can be specified per network adapter (See example above):
name   Enter the network name you want the network adapter to be mapped to.
adapter_type Enter the network adapter type you want to create. Currently supported types are vmxnet, vmxnet2, vmxnet3, e1000 and e1000e.  If no type is specified, by default vmxnet3 will be used.
switch_type Enter the type of switch to use. This decides whether to use a standard switch network or a distributed virtual portgroup. Currently supported types are standard for standard portgroups and distributed for distributed virtual portgroups.
ip     Enter the static IP you want the network adapter to be mapped to. If the network specified is DHCP enabled, you do not have to specify this.
gateway Enter the gateway for the network as a list. If the network specified is DHCP enabled, you do not have to specify this.
subnet_mask Enter the subnet mask for the network. If the network specified is DHCP enabled, you do not have to specify this.
domain Enter the domain to be used with the network adapter. If the network specified is DHCP enabled, you do not have to specify this.
mac    Enter the MAC for this network adapter. If not specified an address will be selected automatically.
scsi   Enter the SCSI controller specification here. If the SCSI controller doesn't exist, a new SCSI controller will be created of the specified type. If the SCSI controller already exists, it will be reconfigured with the specifications. The following additional options can be specified per SCSI controller:
type   Enter the SCSI controller type you want to create. Currently supported types are lsilogic, lsilogic_sas and paravirtual. Type must be specified when creating a new SCSI controller.
bus_sharing Specify this if sharing of virtual disks between virtual machines is desired.  The following can be specified:
virtual Virtual disks can be shared between virtual machines on the same server.
physical Virtual disks can be shared between virtual machines on any server.
no     Virtual disks cannot be shared between virtual machines.
ide    Enter the IDE controller specification here. If the IDE controller doesn't exist, a new IDE controller is created. If the IDE controller already exists, no further changes to it are made. The IDE controller specification is a dictionary.
ide: IDE 2: {}
domain Enter the global domain name to be used for DNS. If not specified and if the VM name is a FQDN, domain is set to the domain from the VM name. Default is local.
dns_servers Enter the list of DNS servers to use in order of priority.
resourcepool Enter the name of the resourcepool to which the new virtual machine should be attached. This determines what compute resources will be available to the clone.
• For a clone operation from a virtual machine, it will use the same resourcepool as the original virtual machine unless specified.
• For a clone operation from a template to a virtual machine, specifying either this or cluster is required. If both are specified, the resourcepool value will be used.
• For a clone operation to a template, this argument is ignored.
cluster Enter the name of the cluster whose resource pool the new virtual machine should be attached to.
• For a clone operation from a virtual machine, it will use the same cluster's resourcepool as the original virtual machine unless specified.
• For a clone operation from a template to a virtual machine, specifying either this or resourcepool is required. If both are specified, the resourcepool value will be used.
datastore Enter the name of the datastore or the datastore cluster where the virtual machine should be located on physical storage. If not specified, the current datastore is used.
• If you specify a datastore cluster name, DRS Storage recommendation is automatically applied.
• If you specify a datastore name, DRS Storage recommendation is disabled.
folder Enter the name of the folder that will contain the new virtual machine.
• For a clone operation from a VM/template, the new VM/template will be added to the same folder that the original VM/template belongs to unless specified.
• If both folder and datacenter are specified, the folder value will be used.
datacenter Enter the name of the datacenter that will contain the new virtual machine.
host   Enter the name of the target host where the virtual machine should be registered.
If not specified:
• If resource pool is not specified, current host is used.
• If resource pool is specified, and the target pool represents a stand-alone host, the host is used.
• If resource pool is specified, and the target pool represents a DRS-enabled cluster, a host selected by DRS is used.
• If resource pool is specified and the target pool represents a cluster without DRS enabled, an InvalidArgument exception be thrown.
template Specifies whether the new virtual machine should be marked as a template or not.  Default is template: False.
power_on Specifies whether the new virtual machine should be powered on or not. If template: True is set, this field is ignored. Default is power_on: True.
extra_config Specifies the additional configuration information for the virtual machine. This describes a set of modifications to the additional options. If the key is already present, it will be reset with the new value provided. Otherwise, a new option is added. Keys with empty values will be removed.
annotation User-provided description of the virtual machine. This will store a message in the vSphere interface, under the annotations section in the Summary view of the virtual machine.
deploy Specifies if salt should be installed on the newly created VM. Default is True so salt will be installed using the bootstrap script. If template: True or power_on: False is set, this field is ignored and salt will not be installed.
wait_for_ip_timeout When deploy: True, this timeout determines the maximum time to wait for VMware tools to be installed on the virtual machine. If this timeout is reached, an attempt to determine the client's IP will be made by resolving the VM's name.  By lowering this value a salt bootstrap can be fully automated for systems that are not built with VMware tools.  Default is wait_for_ip_timeout: 1200.
customization Specify whether the new virtual machine should be customized or not. If customization: False is set, the new virtual machine will not be customized.  Default is customization: True.
private_key Specify the path to the private key to use to be able to ssh to the VM.
ssh_username Specify the username to use in order to ssh to the VM. Default is root
password Specify a password to use in order to ssh to the VM. If private_key is specified, you do not need to specify this.
minion Specify custom minion configuration you want the salt minion to have. A good example would be to specify the master as the IP/DNS name of the master.
file_map Specify file/files you want to copy to the VM before the bootstrap script is run and salt is installed. A good example of using this would be if you need to put custom repo files on the server in case your server will be in a private network and cannot reach external networks.
hardware_version Specify the virtual hardware version for the vm/template that is supported by the host.
image  Specify the guest id of the VM. For a full list of supported values see the VMware vSphere documentation:
https://code.vmware.com/apis?pid=com.vmware.wssdk.apiref.doc&release=vsphere-60&topic=vim.vm.GuestOsDescriptor.GuestOsIdentifier.html
NOTE: For a clone operation, this argument is ignored.
win_username Specify windows vm administrator account.
NOTE: Windows template should have "administrator" account.
win_password Specify windows vm administrator account password.
NOTE: During network configuration (if network specified), it is used to specify new administrator password for the machine.
win_organization_name
Specify windows vm user's organization. Default organization name is Organization VMware vSphere documentation:
https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.UserData.html
win_user_fullname
Specify windows vm user's fullname. Default fullname is Windows User VMware vSphere documentation:
plain_text Flag to specify whether or not the password is in plain text, rather than encrypted.  VMware vSphere documentation:
https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.Password.html
win_installer Specify windows minion client installer path
win_run_once Specify a list of commands to run on first login to a windows minion
https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.GuiRunOnce.html
Cloning a VM Cloning VMs/templates is the easiest and the preferred way to work with VMs using the VMware driver.
NOTE: Cloning operations are unsupported on standalone ESXi hosts, a vCenter server will be required.
Example of a minimal profile:
my-minimal-clone: provider: vcenter01 clonefrom: 'test-vm'
When cloning a VM, all the profile configuration parameters are optional and the configuration gets inherited from the clone.
Example to add/resize a disk:
my-disk-example: provider: vcenter01 clonefrom: 'test-vm'
devices: disk: Hard disk 1: size: 30
Depending on the configuration of the VM that is getting cloned, the disk in the resulting clone will differ.
• If the VM has no disk named 'Hard disk 1' an empty disk with the specified size will be added to the clone.
• If the VM has a disk named 'Hard disk 1' and the size specified is larger than the original disk, an empty disk with the specified size will be added to the clone.
• If the VM has a disk named 'Hard disk 1' and the size specified is smaller than the original disk, an empty disk with the original size will be added to the clone.
Example to reconfigure the memory and number of vCPUs:
memory: 16GB num_cpus: 8
Cloning a Template Cloning a template works similar to cloning a VM except for the fact that a resource pool or cluster must be specified additionally in the profile.
my-template-clone: provider: vcenter01 clonefrom: 'test-template' cluster: 'Prod'
Cloning from a Snapshot New in version 2016.3.5.
Cloning from a snapshot requires that one of the supported options be set in the cloud profile.
Supported options are createNewChildDiskBacking, moveChildMostDiskBacking, moveAllDiskBackingsAndAllowSharing and moveAllDiskBackingsAndDisallowSharing.
my-template-clone: provider: vcenter01 clonefrom: 'salt_vm' snapshot: disk_move_type: createNewChildDiskBacking # these types are also supported # disk_move_type: moveChildMostDiskBacking # disk_move_type: moveAllDiskBackingsAndAllowSharing # disk_move_type: moveAllDiskBackingsAndDisallowSharing
Creating a VM New in version 2016.3.0.
Creating a VM from scratch means that more configuration has to be specified in the profile because there is no place to inherit configuration from.
NOTE: Unlike most cloud drivers that use prepared images, creating VMs using VMware cloud driver needs an installation method that requires no human interaction.  For Example: preseeded ISO, kickstart URL or network PXE boot.
my-minimal-profile: provider: esx01 datastore: esx01-datastore resourcepool: Resources folder: vm
NOTE: The example above contains the minimum required configuration needed to create a VM from scratch. The resulting VM will only have 1 VCPU, 32MB of RAM and will not have any storage or networking.
Example of a complete profile:
my-complete-example: provider: esx01 datastore: esx01-datastore resourcepool: Resources folder: vm
num_cpus: 2 memory: 8GB
image: debian7_64Guest
devices: scsi: SCSI controller 0: type: lsilogic_sas ide: IDE 0: {} IDE 1: {} disk: Hard disk 0: controller: 'SCSI controller 0' size: 20 mode: 'independent_nonpersistent' cd: CD/DVD drive 0: controller: 'IDE 0' device_type: datastore_iso_file iso_path: '[esx01-datastore] debian-8-with-preseed.iso' network: Network adapter 0: name: 'VM Network' swith_type: standard
NOTE: Depending on VMware ESX/ESXi version, an exact match for image might not be available. In such cases, the closest match to another image should be used. In the example above, a Debian 8 VM is created using the image debian7_64Guest which is for a Debian 7 guest.
Specifying disk backing mode New in version 2016.3.5.
Disk backing mode can now be specified when cloning a VM. This option can be set in the cloud profile as shown in example below:
my-vm: provider: esx01 datastore: esx01-datastore resourcepool: Resources folder: vm
devices: disk: Hard disk 1: mode: 'independent_nonpersistent' size: 42 Hard disk 2: mode: 'independent_nonpersistent'
Getting Started With Xen The Xen cloud driver works with Citrix XenServer.
It can be used with a single XenServer or a XenServer resource pool.
Setup Dependencies This driver requires a copy of the freely available XenAPI.py Python module.
Information about the Xen API Python module in the XenServer SDK can be found at https://pypi.org/project/XenAPI/
Place a copy of this module on your system. For example, it can be placed in the site packages location on your system.
The location of site packages can be determined by running:
python -m site --user-site
Provider Configuration Xen requires login credentials to a XenServer.
# /etc/salt/cloud.providers.d/myxen.conf myxen: driver: xen url: https://10.0.0.120 user: root password: p@ssw0rd
url:   The url option supports both http and https uri prefixes.
user:  A valid user id to login to the XenServer host.
password: The associated password for the user.
Profile Configuration Xen profiles require a provider and  image.
provider: This will be the name of your defined provider.
image: The name of the VM template used to clone or copy.
clone: The default behavior is to clone a template or VM. This is very fast, but requires the source template or VM to be in the same storage repository of the new target system. If the source and target are in different storage repositories then you must copy the source and not clone it by setting clone: False.
deploy: The provisioning process will attempt to install the Salt minion service on the new target system by default. This will require login credentials for Salt cloud to login via ssh to it.  The user and password options are required.  If deploy is set to False then these options are not needed.
resource_pool: The name of the resource pool used for this profile.
storage_repo: The name of the storage repository for the target system.
ipv4_cidr: If template is Windows, and running guest tools then a static ip address can be set.
ipv4_gw: If template is Windows, and running guest tools then a gateway can be set.
Set up an initial profile at /etc/salt/cloud.profiles or in the /etc/salt/cloud.profiles.d/ directory:
# file: /etc/salt/cloud.profiles.d/xenprofiles.conf sles: provider: myxen deploy: False image: sles12sp2-template
suse: user: root password: p@ssw0rd provider: myxen image: opensuseleap42_2-template storage_repo: 'Local storage' clone: False minion: master: 10.0.0.20
w2k12: provider: myxen image: w2k12svr-template clone: True userdata_file: /srv/salt/win/files/windows-firewall.ps1 win_installer: /srv/salt/win/files/Salt-Minion-2016.11.3-AMD64-Setup.exe win_username: Administrator win_password: p@ssw0rd use_winrm: False ipv4_cidr: 10.0.0.215/24 ipv4_gw: 10.0.0.1 minion: master: 10.0.0.21
The first example will create a clone of the sles12sp2-template in the same storage repository without deploying the Salt minion.
The second example will make a copy of the image and deploy a new suse VM with the Salt minion installed.
The third example will create a clone of the Windows 2012 template and deploy the Salt minion.
The profile can be used with a salt command:
salt-cloud -p suse  xenvm02
This will create an salt minion instance named xenvm02 in Xen. If the command was executed on the salt-master, its Salt key will automatically be signed on the master.
salt xenvm02 test.version
Listing Sizes Sizes can be obtained using the --list-sizes option for the salt-cloud command:
# salt-cloud --list-sizes myxen
NOTE: Since size information is build in a template this command is not implemented.
Listing Images Images can be obtained using the --list-images option for the salt-cloud command:
# salt-cloud --list-images myxen
This command will return a list of templates with details.
Listing Locations Locations can be obtained using the --list-locations option for the salt-cloud command:
# salt-cloud --list-locations myxen
Returns a list of resource pools.
Miscellaneous Options Miscellaneous Salt Cloud Options This page describes various miscellaneous options available in Salt Cloud
Deploy Script Arguments Custom deploy scripts are unlikely to need custom arguments to be passed to them, but salt-bootstrap has been extended quite a bit, and this may be necessary. script_args can be specified in either the profile or the map file, to pass arguments to the deploy script:
ec2-amazon: provider: my-ec2-config image: ami-1624987f size: t1.micro ssh_username: ec2-user script: bootstrap-salt script_args: -c /tmp/
This has also been tested to work with pipes, if needed:
script_args: '| head'
Selecting the File Transport By default, Salt Cloud uses SFTP to transfer files to Linux hosts. However, if SFTP is not available, or specific SCP functionality is needed, Salt Cloud can be configured to use SCP instead.
file_transport: sftp file_transport: scp
Sync After Install Salt allows users to create custom modules, grains, and states which can be synchronised to minions to extend Salt with further functionality.
This option will inform Salt Cloud to synchronise your custom modules, grains, states or all these to the minion just after it has been created. For this to happen, the following line needs to be added to the main cloud configuration file:
sync_after_install: all
The available options for this setting are:
modules grains states all
Setting Up New Salt Masters It has become increasingly common for users to set up multi-hierarchal infrastructures using Salt Cloud. This sometimes involves setting up an instance to be a master in addition to a minion. With that in mind, you can now lay down master configuration on a machine by specifying master options in the profile or map file.
make_master: True
This will cause Salt Cloud to generate master keys for the instance, and tell salt-bootstrap to install the salt-master package, in addition to the salt-minion package.
The default master configuration is usually appropriate for most users, and will not be changed unless specific master configuration has been added to the profile or map:
master: user: root interface: 0.0.0.0
Setting Up a Salt Syndic with Salt Cloud In addition to setting up new Salt Masters, syndics can also be provisioned using Salt Cloud. In order to set up a Salt Syndic via Salt Cloud, a Salt Master needs to be installed on the new machine and a master configuration file needs to be set up using the make_master setting. This setting can be defined either in a profile config file or in a map file:
To install the Salt Syndic, the only other specification that needs to be configured is the syndic_master key to specify the location of the master that the syndic will be reporting to. This modification needs to be placed in the master setting, which can be configured either in the profile, provider, or /etc/salt/cloud config file:
master: syndic_master: 123.456.789  # may be either an IP address or a hostname
Many other Salt Syndic configuration settings and specifications can be passed through to the new syndic machine via the master configuration setting.  See the syndic documentation for more information.
SSH Port By default ssh port is set to port 22. If you want to use a custom port in provider, profile, or map blocks use ssh_port option.
ssh_port: 2222
Delete SSH Keys When Salt Cloud deploys an instance, the SSH pub key for the instance is added to the known_hosts file for the user that ran the salt-cloud command. When an instance is deployed, a cloud host generally recycles the IP address for the instance.  When Salt Cloud attempts to deploy an instance using a recycled IP address that has previously been accessed from the same machine, the old key in the known_hosts file will cause a conflict.
In order to mitigate this issue, Salt Cloud can be configured to remove old keys from the known_hosts file when destroying the node. In order to do this, the following line needs to be added to the main cloud configuration file:
delete_sshkeys: True
Keeping /tmp/ Files When Salt Cloud deploys an instance, it uploads temporary files to /tmp/ for salt-bootstrap to put in place. After the script has run, they are deleted. To keep these files around (mostly for debugging purposes), the --keep-tmp option can be added:
salt-cloud -p myprofile mymachine --keep-tmp
For those wondering why /tmp/ was used instead of /root/, this had to be done for images which require the use of sudo, and therefore do not allow remote root logins, even for file transfers (which makes /root/ unavailable).
Hide Output From Minion Install By default Salt Cloud will stream the output from the minion deploy script directly to STDOUT. Although this can been very useful, in certain cases you may wish to switch this off. The following config option is there to enable or disable this output:
display_ssh_output: False
Connection Timeout There are several stages when deploying Salt where Salt Cloud needs to wait for something to happen. The VM getting its IP address, the VM's SSH port is available, etc.
If you find that the Salt Cloud defaults are not enough and your deployment fails because Salt Cloud did not wait log enough, there are some settings you can tweak.
All settings should be provided in lowercase All values should be provided in seconds
You can tweak these settings globally, per cloud provider, or event per profile definition.
wait_for_ip_timeout The amount of time Salt Cloud should wait for a VM to start and get an IP back from the cloud host.  Default: varies by cloud provider ( between 5 and 25 minutes)
wait_for_ip_interval The amount of time Salt Cloud should sleep while querying for the VM's IP.  Default: varies by cloud provider ( between .5 and 10 seconds)
ssh_connect_timeout The amount of time Salt Cloud should wait for a successful SSH connection to the VM.  Default: varies by cloud provider  (between 5 and 15 minutes)
wait_for_passwd_timeout The amount of time until an ssh connection can be established via password or ssh key.  Default: varies by cloud provider (mostly 15 seconds)
wait_for_passwd_maxtries The number of attempts to connect to the VM until we abandon.  Default: 15 attempts
wait_for_fun_timeout Some cloud drivers check for an available IP or a successful SSH connection using a function, namely, SoftLayer, and SoftLayer-HW. So, the amount of time Salt Cloud should retry such functions before failing. Default: 15 minutes.
wait_for_spot_timeout The amount of time Salt Cloud should wait before an EC2 Spot instance is available. This setting is only available for the EC2 cloud driver. Default: 10  minutes
Salt Cloud Cache Salt Cloud can maintain a cache of node data, for supported providers. The following options manage this functionality.
update_cachedir On supported cloud providers, whether or not to maintain a cache of nodes returned from a --full-query. The data will be stored in msgpack format under <SALT_CACHEDIR>/cloud/active/<DRIVER>/<PROVIDER>/<NODE_NAME>.p. This setting can be True or False.
diff_cache_events When the cloud cachedir is being managed, if differences are encountered between the data that is returned live from the cloud host and the data in the cache, fire events which describe the changes. This setting can be True or False.
Some of these events will contain data which describe a node. Because some of the fields returned may contain sensitive data, the cache_event_strip_fields configuration option exists to strip those fields from the event return.
cache_event_strip_fields: - password - priv_key
The following are events that can be fired based on this data.
salt/cloud/minionid/cache_node_new A new node was found on the cloud host which was not listed in the cloud cachedir. A dict describing the new node will be contained in the event.
salt/cloud/minionid/cache_node_missing A node that was previously listed in the cloud cachedir is no longer available on the cloud host.
salt/cloud/minionid/cache_node_diff One or more pieces of data in the cloud cachedir has changed on the cloud host. A dict containing both the old and the new data will be contained in the event.
SSH Known Hosts Normally when bootstrapping a VM, salt-cloud will ignore the SSH host key. This is because it does not know what the host key is before starting (because it doesn't exist yet). If strict host key checking is turned on without the key in the known_hosts file, then the host will never be available, and cannot be bootstrapped.
If a provider is able to determine the host key before trying to bootstrap it, that provider's driver can add it to the known_hosts file, and then turn on strict host key checking. This can be set up in the main cloud configuration file (normally /etc/salt/cloud) or in the provider-specific configuration file:
known_hosts_file: /path/to/.ssh/known_hosts
If this is not set, it will default to /dev/null, and strict host key checking will be turned off.
It is highly recommended that this option is not set, unless the user has verified that the provider supports this functionality, and that the image being used is capable of providing the necessary information. At this time, only the EC2 driver supports this functionality.
SSH Agent New in version 2015.5.0.
If the ssh key is not stored on the server salt-cloud is being run on, set ssh_agent, and salt-cloud will use the forwarded ssh-agent to authenticate.
ssh_agent: True
File Map Upload New in version 2014.7.0.
The file_map option allows an arbitrary group of files to be uploaded to the target system before running the deploy script. This functionality requires a provider uses salt.utils.cloud.bootstrap(), which is currently limited to the ec2, gce, openstack and nova drivers.
The file_map can be configured globally in /etc/salt/cloud, or in any cloud provider or profile file. For example, to upload an extra package or a custom deploy script, a cloud profile using file_map might look like:
ubuntu14: provider: ec2-config image: ami-98aa1cf0 size: t1.micro ssh_username: root securitygroup: default file_map: /local/path/to/custom/script: /remote/path/to/use/custom/script /local/path/to/package: /remote/path/to/store/package
Running Pre-Flight Commands New in version 2018.3.0.
To execute specified preflight shell commands on a VM before the deploy script is run, use the preflight_cmds option. These must be defined as a list in a cloud configuration file. For example:
my-cloud-profile: provider: linode-config image: Ubuntu 16.04 LTS size: Linode 2048 preflight_cmds: - whoami - echo 'hello world!'
These commands will run in sequence before the bootstrap script is executed.
Force Minion Config New in version 2018.3.0.
The force_minion_config option requests the bootstrap process to overwrite an existing minion configuration file and public/private key files.  Default: False
This might be important for drivers (such as saltify) which are expected to take over a connection from a former salt master.
my_saltify_provider: driver: saltify force_minion_config: true
Troubleshooting Steps Troubleshooting Salt Cloud This page describes various steps for troubleshooting problems that may arise while using Salt Cloud.
Virtual Machines Are Created, But Do Not Respond Are TCP ports 4505 and 4506 open on the master? This is easy to overlook on new masters. Information on how to open firewall ports on various platforms can be found here.
Generic Troubleshooting Steps This section describes a set of instructions that are useful to a large number of situations, and are likely to solve most issues that arise.
Debug Mode Frequently, running Salt Cloud in debug mode will reveal information about a deployment which would otherwise not be obvious:
salt-cloud -p myprofile myinstance -l debug
Keep in mind that a number of messages will appear that look at first like errors, but are in fact intended to give developers factual information to assist in debugging. A number of messages that appear will be for cloud providers that you do not have configured; in these cases, the message usually is intended to confirm that they are not configured.
Salt Bootstrap By default, Salt Cloud uses the Salt Bootstrap script to provision instances:
This script is packaged with Salt Cloud, but may be updated without updating the Salt package:
salt-cloud -u
The Bootstrap Log If the default deploy script was used, there should be a file in the /tmp/ directory called bootstrap-salt.log. This file contains the full output from the deployment, including any errors that may have occurred.
Keeping Temp Files Salt Cloud uploads minion-specific files to instances once they are available via SSH, and then executes a deploy script to put them into the correct place and install Salt. The --keep-tmp option will instruct Salt Cloud not to remove those files when finished with them, so that the user may inspect them for problems:
salt-cloud -p myprofile myinstance --keep-tmp
By default, Salt Cloud will create a directory on the target instance called /tmp/.saltcloud/. This directory should be owned by the user that is to execute the deploy script, and should have permissions of 0700.
Most cloud hosts are configured to use root as the default initial user for deployment, and as such, this directory and all files in it should be owned by the root user.
The /tmp/.saltcloud/ directory should the following files:
• A deploy.sh script. This script should have permissions of 0755.
• A .pem and .pub key named after the minion. The .pem file should have permissions of 0600. Ensure that the .pem and .pub files have been properly copied to the /etc/salt/pki/minion/ directory.
• A file called minion. This file should have been copied to the /etc/salt/ directory.
• Optionally, a file called grains. This file, if present, should have been copied to the /etc/salt/ directory.
Unprivileged Primary Users Some cloud hosts, most notably EC2, are configured with a different primary user.  Some common examples are ec2-user, ubuntu, fedora, and bitnami.  In these cases, the /tmp/.saltcloud/ directory and all files in it should be owned by this user.
Some cloud hosts, such as EC2, are configured to not require these users to provide a password when using the sudo command. Because it is more secure to require sudo users to provide a password, other hosts are configured that way.
If this instance is required to provide a password, it needs to be configured in Salt Cloud. A password for sudo to use may be added to either the provider configuration or the profile configuration:
sudo_password: mypassword
/tmp/ is Mounted as noexec It is more secure to mount the /tmp/ directory with a noexec option. This is uncommon on most cloud hosts, but very common in private environments. To see if the /tmp/ directory is mounted this way, run the following command:
mount | grep tmp
The if the output of this command includes a line that looks like this, then the /tmp/ directory is mounted as noexec:
tmpfs on /tmp type tmpfs (rw,noexec)
If this is the case, then the deploy_command will need to be changed in order to run the deploy script through the sh command, rather than trying to execute it directly. This may be specified in either the provider or the profile config:
deploy_command: sh /tmp/.saltcloud/deploy.sh
Please note that by default, Salt Cloud will place its files in a directory called /tmp/.saltcloud/. This may be also be changed in the provider or profile configuration:
tmp_dir: /tmp/.saltcloud/
If this directory is changed, then the deploy_command need to be changed in order to reflect the tmp_dir configuration.
Executing the Deploy Script Manually If all of the files needed for deployment were successfully uploaded to the correct locations, and contain the correct permissions and ownerships, the deploy script may be executed manually in order to check for other issues:
cd /tmp/.saltcloud/ ./deploy.sh
Extending Salt Cloud Writing Cloud Driver Modules Salt Cloud runs on a module system similar to the main Salt project. The modules inside saltcloud exist in the salt/cloud/clouds directory of the salt source.
There are two basic types of cloud modules. If a cloud host is supported by libcloud, then using it is the fastest route to getting a module written. The Apache Libcloud project is located at:
http://libcloud.apache.org/
Not every cloud host is supported by libcloud. Additionally, not every feature in a supported cloud host is necessarily supported by libcloud. In either of these cases, a module can be created which does not rely on libcloud.
All Driver Modules The following functions are required by all driver modules, whether or not they are based on libcloud.
The __virtual__() Function This function determines whether or not to make this cloud module available upon execution. Most often, it uses get_configured_provider() to determine if the necessary configuration has been set up. It may also check for necessary imports, to decide whether to load the module. In most cases, it will return a True or False value. If the name of the driver used does not match the filename, then that name should be returned instead of True. An example of this may be seen in the Azure module:
https://github.com/saltstack/salt/tree/master/salt/cloud/clouds/msazure.py
The get_configured_provider() Function This function uses config.is_provider_configured() to determine whether all required information for this driver has been configured. The last value in the list of required settings should be followed by a comma.
Libcloud Based Modules Writing a cloud module based on libcloud has two major advantages. First of all, much of the work has already been done by the libcloud project. Second, most of the functions necessary to Salt have already been added to the Salt Cloud project.
The create() Function The most important function that does need to be manually written is the create() function. This is what is used to request a virtual machine to be created by the cloud host, wait for it to become available, and then (optionally) log in and install Salt on it.
A good example to follow for writing a cloud driver module based on libcloud is the module provided for Linode:
https://github.com/saltstack/salt/tree/master/salt/cloud/clouds/linode.py
The basic flow of a create() function is as follows:
• Send a request to the cloud host to create a virtual machine.
• Wait for the virtual machine to become available.
• Generate kwargs to be used to deploy Salt.
• Log into the virtual machine and deploy Salt.
• Return a data structure that describes the newly-created virtual machine.
At various points throughout this function, events may be fired on the Salt event bus. Four of these events, which are described below, are required. Other events may be added by the user, where appropriate.
When the create() function is called, it is passed a data structure called vm_. This dict contains a composite of information describing the virtual machine to be created. A dict called __opts__ is also provided by Salt, which contains the options used to run Salt Cloud, as well as a set of configuration and environment variables.
The first thing the create() function must do is fire an event stating that it has started the create process. This event is tagged salt/cloud/<vm name>/creating. The payload contains the names of the VM, profile, and provider.
A set of kwargs is then usually created, to describe the parameters required by the cloud host to request the virtual machine.
An event is then fired to state that a virtual machine is about to be requested.  It is tagged as salt/cloud/<vm name>/requesting. The payload contains most or all of the parameters that will be sent to the cloud host. Any private information (such as passwords) should not be sent in the event.
After a request is made, a set of deploy kwargs will be generated. These will be used to install Salt on the target machine. Windows options are supported at this point, and should be generated, even if the cloud host does not currently support Windows. This will save time in the future if the host does eventually decide to support Windows.
An event is then fired to state that the deploy process is about to begin. This event is tagged salt/cloud/<vm name>/deploying. The payload for the event will contain a set of deploy kwargs, useful for debugging purposed. Any private data, including passwords and keys (including public keys) should be stripped from the deploy kwargs before the event is fired.
If any Windows options have been passed in, the salt.utils.cloud.deploy_windows() function will be called. Otherwise, it will be assumed that the target is a Linux or Unix machine, and the salt.utils.cloud.deploy_script() will be called.
Both of these functions will wait for the target machine to become available, then the necessary port to log in, then a successful login that can be used to install Salt. Minion configuration and keys will then be uploaded to a temporary directory on the target by the appropriate function. On a Windows target, the Windows Minion Installer will be run in silent mode. On a Linux/Unix target, a deploy script (bootstrap-salt.sh, by default) will be run, which will auto-detect the operating system, and install Salt using its native package manager. These do not need to be handled by the developer in the cloud module.
The salt.utils.cloud.validate_windows_cred() function has been extended to take the number of retries and retry_delay parameters in case a specific cloud host has a delay between providing the Windows credentials and the credentials being available for use.  In their create() function, or as a sub-function called during the creation process, developers should use the win_deploy_auth_retries and win_deploy_auth_retry_delay parameters from the provider configuration to allow the end-user the ability to customize the number of tries and delay between tries for their particular host.
After the appropriate deploy function completes, a final event is fired which describes the virtual machine that has just been created. This event is tagged salt/cloud/<vm name>/created. The payload contains the names of the VM, profile, and provider.
Finally, a dict (queried from the provider) which describes the new virtual machine is returned to the user. Because this data is not fired on the event bus it can, and should, return any passwords that were returned by the cloud host. In some cases (for example, Rackspace), this is the only time that the password can be queried by the user; post-creation queries may not contain password information (depending upon the host).
The libcloudfuncs Functions A number of other functions are required for all cloud hosts. However, with libcloud-based modules, these are all provided for free by the libcloudfuncs library. The following two lines set up the imports:
from salt.cloud.libcloudfuncs import *  # pylint: disable=W0614,W0401 import salt.utils.functools
And then a series of declarations will make the necessary functions available within the cloud module.
get_size = salt.utils.functools.namespaced_function(get_size, globals()) get_image = salt.utils.functools.namespaced_function(get_image, globals()) avail_locations = salt.utils.functools.namespaced_function(avail_locations, globals()) avail_images = salt.utils.functools.namespaced_function(avail_images, globals()) avail_sizes = salt.utils.functools.namespaced_function(avail_sizes, globals()) script = salt.utils.functools.namespaced_function(script, globals()) destroy = salt.utils.functools.namespaced_function(destroy, globals()) list_nodes = salt.utils.functools.namespaced_function(list_nodes, globals()) list_nodes_full = salt.utils.functools.namespaced_function(list_nodes_full, globals()) list_nodes_select = salt.utils.functools.namespaced_function( list_nodes_select, globals() ) show_instance = salt.utils.functools.namespaced_function(show_instance, globals())
If necessary, these functions may be replaced by removing the appropriate declaration line, and then adding the function as normal.
These functions are required for all cloud modules, and are described in detail in the next section.
Non-Libcloud Based Modules In some cases, using libcloud is not an option. This may be because libcloud has not yet included the necessary driver itself, or it may be that the driver that is included with libcloud does not contain all of the necessary features required by the developer. When this is the case, some or all of the functions in libcloudfuncs may be replaced. If they are all replaced, the libcloud imports should be absent from the Salt Cloud module.
A good example of a non-libcloud driver is the DigitalOcean driver:
https://github.com/saltstack/salt/tree/master/salt/cloud/clouds/digitalocean.py
The create() Function The create() function must be created as described in the libcloud-based module documentation.
The get_size() Function This function is only necessary for libcloud-based modules, and does not need to exist otherwise.
The get_image() Function This function is only necessary for libcloud-based modules, and does not need to exist otherwise.
The avail_locations() Function This function returns a list of locations available, if the cloud host uses multiple data centers. It is not necessary if the cloud host uses only one data center. It is normally called using the --list-locations option.
salt-cloud --list-locations my-cloud-provider
The avail_images() Function This function returns a list of images available for this cloud provider. There are not currently any known cloud providers that do not provide this functionality, though they may refer to images by a different name (for example, "templates"). It is normally called using the --list-images option.
salt-cloud --list-images my-cloud-provider
The avail_sizes() Function This function returns a list of sizes available for this cloud provider. Generally, this refers to a combination of RAM, CPU, and/or disk space. This functionality may not be present on some cloud providers. For example, the Parallels module breaks down RAM, CPU, and disk space into separate options, whereas in other providers, these options are baked into the image. It is normally called using the --list-sizes option.
salt-cloud --list-sizes my-cloud-provider
The script() Function This function builds the deploy script to be used on the remote machine. It is likely to be moved into the salt.utils.cloud library in the near future, as it is very generic and can usually be copied wholesale from another module. An excellent example is in the Azure driver.