neulab/tldr huggingface dataset:
# salt-cloud --list-images my-dimensiondata-config my-dimensiondata-config: ---------- dimensiondata: ---------- CSfM SharePoint 2013 Trial: ---------- driver: extra: ---------- OS_displayName: WIN2012R2S/64 OS_type: None cpu: created: 2015-03-19T18:36:06.000Z description: Windows 2012 R2 Standard 64-bit installed with SharePoint 2013 and Visual Studio 2013 Pro (Trial Version) location: memoryGb: 12 osImageKey: T-WIN-2012R2-STD-SP2013-VS2013-64-4-12-100 get_uuid: id: 0df4677e-d380-4e9b-9469-b529ee0214c5 name: CSfM SharePoint 2013 Trial uuid: 28c077f1be970ee904541407b377e3ff87a9ac69 CentOS 5 32-bit 2 CPU: ---------- driver: extra: ---------- OS_displayName: CENTOS5/32 OS_type: None cpu: created: 2015-10-21T14:52:29.000Z description: CentOS Release 5.11 32-bit location: memoryGb: 4 osImageKey: T-CENT-5-32-2-4-10 get_uuid: id: a8046bd1-04ea-4668-bf32-bf8d5540faed name: CentOS 5 32-bit 2 CPU uuid: 4d7dd59929fed6f4228db861b609da64997773a7
my-dimensiondata-config: ---------- dimensiondata: ---------- Australia - Melbourne: ---------- country: Australia driver: id: AU2 name: Australia - Melbourne Australia - Melbourne MCP2: ---------- country: Australia driver: id: AU10 name: Australia - Melbourne MCP2 Australia - Sydney: ---------- country: Australia driver: id: AU1 name: Australia - Sydney Australia - Sydney MCP2: ---------- country: Australia driver: id: AU9 name: Australia - Sydney MCP2 New Zealand: ---------- country: New Zealand driver: id: AU8 name: New Zealand New_Zealand: ---------- country: New Zealand driver: id: AU11 name: New_Zealand
NOTE: Dimension Data Cloud REST API documentation is available from Dimension Data MCP 2.
Getting Started With AWS EC2 Amazon EC2 is a very widely used public cloud platform and one of the core platforms Salt Cloud has been built to support.
Previously, the suggested driver for AWS EC2 was the aws driver. This has been deprecated in favor of the ec2 driver. Configuration using the old aws driver will still function, but that driver is no longer in active development.
Configuration The following example illustrates some of the options that can be set. These parameters are discussed in more detail below.
my-ec2-southeast-public-ips: # Set up the location of the salt master # minion: master: saltmaster.example.com
# Set up grains information, which will be common for all nodes # using this provider grains: node_type: broker release: 1.0.1
# Specify whether to use public or private IP for deploy script. # # Valid options are: #     private_ips - The salt-cloud command is run inside the EC2 #     public_ips - The salt-cloud command is run outside of EC2 # ssh_interface: public_ips
# Optionally configure the Windows credential validation number of # retries and delay between retries.  This defaults to 10 retries # with a one second delay betwee retries win_deploy_auth_retries: 10 win_deploy_auth_retry_delay: 1
# Set the EC2 access credentials (see below) # id: 'use-instance-role-credentials' key: 'use-instance-role-credentials'
# If 'role_arn' is specified the above credentials are used to # to assume to the role. By default, role_arn is set to None. role_arn: arn:aws:iam::012345678910:role/SomeRoleName
# Make sure this key is owned by corresponding user (default 'salt') with permissions 0400. # private_key: /etc/salt/my_test_key.pem keyname: my_test_key securitygroup: default
# Optionally configure default region # Use salt-cloud --list-locations <provider> to obtain valid regions # location: ap-southeast-1 availability_zone: ap-southeast-1b
# Configure which user to use to run the deploy script. This setting is # dependent upon the AMI that is used to deploy. It is usually safer to # configure this individually in a profile, than globally. Typical users # are: # # Amazon Linux -> ec2-user # RHEL         -> ec2-user # CentOS       -> ec2-user # Ubuntu       -> ubuntu # Debian       -> admin # ssh_username: ec2-user
# Optionally add an IAM profile iam_profile: 'arn:aws:iam::123456789012:instance-profile/ExampleInstanceProfile'
driver: ec2
my-ec2-southeast-private-ips: # Set up the location of the salt master # minion: master: saltmaster.example.com
# Specify whether to use public or private IP for deploy script. # # Valid options are: #     private_ips - The salt-master is also hosted with EC2 #     public_ips - The salt-master is hosted outside of EC2 # ssh_interface: private_ips
# Make sure this key is owned by root with permissions 0400. # private_key: /etc/salt/my_test_key.pem keyname: my_test_key
# This one should NOT be specified if VPC was not configured in AWS to be # the default. It might cause an error message which says that network # interfaces and an instance-level security groups may not be specified # on the same request. # securitygroup: default
# Optionally configure default region # location: ap-southeast-1 availability_zone: ap-southeast-1b
# Configure which user to use to run the deploy script. This setting is # dependent upon the AMI that is used to deploy. It is usually safer to # configure this individually in a profile, than globally. Typical users # are: # # Amazon Linux -> ec2-user # RHEL         -> ec2-user # CentOS       -> ec2-user # Ubuntu       -> ubuntu # ssh_username: ec2-user
# Optionally add an IAM profile iam_profile: 'my other profile name'
Access Credentials The id and key settings may be found in the Security Credentials area of the AWS Account page:
https://portal.aws.amazon.com/gp/aws/securityCredentials
Both are located in the Access Credentials area of the page, under the Access Keys tab. The id setting is labeled Access Key ID, and the key setting is labeled Secret Access Key.
Note: if either id or key is set to 'use-instance-role-credentials' it is assumed that Salt is running on an AWS instance, and the instance role credentials will be retrieved and used.  Since both the id and key are required parameters for the AWS ec2 provider, it is recommended to set both to 'use-instance-role-credentials' for this functionality.
A "static" and "permanent" Access Key ID and Secret Key can be specified, but this is not recommended.  Instance role keys are rotated on a regular basis, and are the recommended method of specifying AWS credentials.
Windows Deploy Timeouts For Windows instances, it may take longer than normal for the instance to be ready.  In these circumstances, the provider configuration can be configured with a win_deploy_auth_retries and/or a win_deploy_auth_retry_delay setting, which default to 10 retries and a one second delay between retries.  These retries and timeouts relate to validating the Administrator password once AWS provides the credentials via the AWS API.
Key Pairs In order to create an instance with Salt installed and configured, a key pair will need to be created. This can be done in the EC2 Management Console, in the Key Pairs area. These key pairs are unique to a specific region. Keys in the us-east-1 region can be configured at:
https://console.aws.amazon.com/ec2/home?region=us-east-1#s=KeyPairs
Keys in the us-west-1 region can be configured at
https://console.aws.amazon.com/ec2/home?region=us-west-1#s=KeyPairs
...and so on. When creating a key pair, the browser will prompt to download a pem file. This file must be placed in a directory accessible by Salt Cloud, with permissions set to either 0400 or 0600.
Security Groups An instance on EC2 needs to belong to a security group. Like key pairs, these are unique to a specific region. These are also configured in the EC2 Management Console. Security groups for the us-east-1 region can be configured at:
https://console.aws.amazon.com/ec2/home?region=us-east-1#s=SecurityGroups
...and so on.
A security group defines firewall rules which an instance will adhere to. If the salt-master is configured outside of EC2, the security group must open the SSH port (usually port 22) in order for Salt Cloud to install Salt.
IAM Profile Amazon EC2 instances support the concept of an instance profile, which is a logical container for the IAM role. At the time that you launch an EC2 instance, you can associate the instance with an instance profile, which in turn corresponds to the IAM role. Any software that runs on the EC2 instance is able to access AWS using the permissions associated with the IAM role.
Scaffolding the profile is a 2-step configuration process:
1. Configure an IAM Role from the IAM Management Console.
2. Attach this role to a new profile. It can be done with the AWS CLI:
> aws iam create-instance-profile --instance-profile-name PROFILE_NAME > aws iam add-role-to-instance-profile --instance-profile-name PROFILE_NAME --role-name ROLE_NAME
Once the profile is created, you can use the PROFILE_NAME to configure your cloud profiles.
base_ec2_private: provider: my-ec2-southeast-private-ips image: ami-e565ba8c size: t2.micro ssh_username: ec2-user
base_ec2_public: provider: my-ec2-southeast-public-ips image: ami-e565ba8c size: t2.micro ssh_username: ec2-user
base_ec2_db: provider: my-ec2-southeast-public-ips image: ami-e565ba8c size: m1.xlarge ssh_username: ec2-user volumes: - { size: 10, device: /dev/sdf } - { size: 10, device: /dev/sdg, type: io1, iops: 1000 } - { size: 10, device: /dev/sdh, type: io1, iops: 1000 } - { size: 10, device: /dev/sdi, tags: {"Environment": "production"} } # optionally add tags to profile: tag: {'Environment': 'production', 'Role': 'database'} # force grains to sync after install sync_after_install: grains
base_ec2_vpc: provider: my-ec2-southeast-public-ips image: ami-a73264ce size: m1.xlarge ssh_username: ec2-user script:  /etc/salt/cloud.deploy.d/user_data.sh network_interfaces: - DeviceIndex: 0 PrivateIpAddresses: - Primary: True #auto assign public ip (not EIP) AssociatePublicIpAddress: True SubnetId: subnet-813d4bbf SecurityGroupId: - sg-750af413 del_root_vol_on_destroy: True del_all_vols_on_destroy: True volumes: - { size: 10, device: /dev/sdf } - { size: 10, device: /dev/sdg, type: io1, iops: 1000 } - { size: 10, device: /dev/sdh, type: io1, iops: 1000 } tag: {'Environment': 'production', 'Role': 'database'} sync_after_install: grains
The profile can now be realized with a salt command:
# salt-cloud -p base_ec2 ami.example.com # salt-cloud -p base_ec2_public ami.example.com # salt-cloud -p base_ec2_private ami.example.com
This will create an instance named ami.example.com in EC2. The minion that is installed on this instance will have an id of ami.example.com. If the command was executed on the salt-master, its Salt key will automatically be signed on the master.
# salt 'ami.example.com' test.version
Required Settings The following settings are always required for EC2:
# Set the EC2 login data my-ec2-config: id: HJGRYCILJLKJYG key: 'kdjgfsgm;woormgl/aserigjksjdhasdfgn' keyname: test securitygroup: quick-start private_key: /root/test.pem driver: ec2
Optional Settings EC2 allows a userdata file to be passed to the instance to be created. This functionality was added to Salt in the 2015.5.0 release.
my-ec2-config: # Pass userdata to the instance to be created userdata_file: /etc/salt/my-userdata-file
my-ec2-config: # Pass userdata to the instance to be created userdata_file: /etc/salt/my-userdata-file userdata_template: jinja
my-ec2-config: # Pass userdata to the instance to be created userdata_file: /etc/salt/my-userdata-file userdata_template: False
EC2 allows a location to be set for servers to be deployed in. Availability zones exist inside regions, and may be added to increase specificity.
my-ec2-config: # Optionally configure default region location: ap-southeast-1 availability_zone: ap-southeast-1b
EC2 instances can have a public or private IP, or both. When an instance is deployed, Salt Cloud needs to log into it via SSH to run the deploy script.  By default, the public IP will be used for this. If the salt-cloud command is run from another EC2 instance, the private IP should be used.
my-ec2-config: # Specify whether to use public or private IP for deploy script # private_ips or public_ips ssh_interface: public_ips
Many EC2 instances do not allow remote access to the root user by default.  Instead, another user must be used to run the deploy script using sudo. Some common usernames include ec2-user (for Amazon Linux), ubuntu (for Ubuntu instances), admin (official Debian) and bitnami (for images provided by Bitnami).
my-ec2-config: # Configure which user to use to run the deploy script ssh_username: ec2-user
Multiple usernames can be provided, in which case Salt Cloud will attempt to guess the correct username. This is mostly useful in the main configuration file:
my-ec2-config: ssh_username: - ec2-user - ubuntu - admin - bitnami
Multiple security groups can also be specified in the same fashion:
my-ec2-config: securitygroup: - default - extra
EC2 instances can be added to an AWS Placement Group by specifying the placementgroup option:
my-ec2-config: placementgroup: my-aws-placement-group
Your instances may optionally make use of EC2 Spot Instances. The following example will request that spot instances be used and your maximum bid will be $0.10. Keep in mind that different spot prices may be needed based on the current value of the various EC2 instance sizes. You can check current and past spot instance pricing via the EC2 API or AWS Console.
my-ec2-config: spot_config: spot_price: 0.10
You can optionally specify tags to apply to the EC2 spot instance request.  A spot instance request itself is an object in AWS. The following example will set two tags on the spot instance request.
my-ec2-config: spot_config: spot_price: 0.10 tag: tag0: value tag1: value
By default, the spot instance type is set to 'one-time', meaning it will be launched and, if it's ever terminated for whatever reason, it will not be recreated. If you would like your spot instances to be relaunched after a termination (by you or AWS), set the type to 'persistent'.
NOTE: Spot instances are a great way to save a bit of money, but you do run the risk of losing your spot instances if the current price for the instance size goes above your maximum bid.
The following parameters may be set in the cloud configuration file to control various aspects of the spot instance launching:
• wait_for_spot_timeout: seconds to wait before giving up on spot instance launch (default=600)
• wait_for_spot_interval: seconds to wait in between polling requests to determine if a spot instance is available (default=30)
• wait_for_spot_interval_multiplier: a multiplier to add to the interval in between requests, which is useful if AWS is throttling your requests (default=1)
• wait_for_spot_max_failures: maximum number of failures before giving up on launching your spot instance (default=10)
If you find that you're being throttled by AWS while polling for spot instances, you can set the following in your core cloud configuration file that will double the polling interval after each request to AWS.
wait_for_spot_interval: 1 wait_for_spot_interval_multiplier: 2
See the AWS Spot Instances documentation for more information.
Block device mappings enable you to specify additional EBS volumes or instance store volumes when the instance is launched. This setting is also available on each cloud profile. Note that the number of instance stores varies by instance type.  If more mappings are provided than are supported by the instance type, mappings will be created in the order provided and additional mappings will be ignored. Consult the AWS documentation for a listing of the available instance stores, and device names.
my-ec2-config: block_device_mappings: - DeviceName: /dev/sdb VirtualName: ephemeral0 - DeviceName: /dev/sdc VirtualName: ephemeral1
You can also use block device mappings to change the size of the root device at the provisioning time. For example, assuming the root device is '/dev/sda', you can set its size to 100G by using the following configuration.
my-ec2-config: block_device_mappings: - DeviceName: /dev/sda Ebs.VolumeSize: 100 Ebs.VolumeType: gp2 Ebs.SnapshotId: dummy0 - DeviceName: /dev/sdb # required for devices > 2TB Ebs.VolumeType: gp2 Ebs.VolumeSize: 3001
Tagging of block devices can be set on a per device basis. For example, you may have multiple devices defined in your block_device_mappings structure. You have the option to set tags on any of one device or all of them as shown in the following configuration.
my-ec2-config: block_device_mappings: - DeviceName: /dev/sda Ebs.VolumeSize: 100 Ebs.VolumeType: gp2 tag: tag0: myserver tag1: value - DeviceName: /dev/sdb Ebs.VolumeType: gp2 Ebs.VolumeSize: 3001 tag: tagX: value tagY: value
You can configure any AWS valid tag name as shown in the above example, including 'Name'. If you do not configure the tag 'Name', it will be automatically created with a value set to the virtual machine name. If you configure the tag 'Name', the value you configure will be used rather than defaulting to the virtual machine name as shown in the following configuration.
my-ec2-config: block_device_mappings: - DeviceName: /dev/sda Ebs.VolumeSize: 100 Ebs.VolumeType: gp2 tag: Name: myserver tag0: value tag1: value - DeviceName: /dev/sdb Ebs.VolumeType: gp2 Ebs.VolumeSize: 3001 tag: Name: customvalue tagX: value tagY: value
Existing EBS volumes may also be attached (not created) to your instances or you can create new EBS volumes based on EBS snapshots. To simply attach an existing volume use the volume_id parameter.
device: /dev/xvdj volume_id: vol-12345abcd
Or, to create a volume from an EBS snapshot, use the snapshot parameter.
device: /dev/xvdj snapshot: snap-abcd12345
Note that volume_id will take precedence over the snapshot parameter.
Tags can be set once an instance has been launched.
my-ec2-config: tag: tag0: value tag1: value
Setting up a Master inside EC2 Salt Cloud can configure Salt Masters as well as Minions. Use the make_master setting to use this functionality.
my-ec2-config: # Optionally install a Salt Master in addition to the Salt Minion make_master: True
When creating a Salt Master inside EC2 with make_master: True, or when the Salt Master is already located and configured inside EC2, by default, minions connect to the master's public IP address during Salt Cloud's provisioning process. Depending on how your security groups are defined, the minions may or may not be able to communicate with the master. In order to use the master's private IP in EC2 instead of the public IP, set the salt_interface to private_ips.
my-ec2-config: # Optionally set the IP configuration to private_ips salt_interface: private_ips
Modify EC2 Tags One of the features of EC2 is the ability to tag resources. In fact, under the hood, the names given to EC2 instances by salt-cloud are actually just stored as a tag called Name. Salt Cloud has the ability to manage these tags:
salt-cloud -a get_tags mymachine salt-cloud -a set_tags mymachine tag1=somestuff tag2='Other stuff' salt-cloud -a del_tags mymachine tag1,tag2,tag3
It is possible to manage tags on any resource in EC2 with a Resource ID, not just instances:
salt-cloud -f get_tags my_ec2 resource_id=af5467ba salt-cloud -f set_tags my_ec2 resource_id=af5467ba tag1=somestuff salt-cloud -f del_tags my_ec2 resource_id=af5467ba tags=tag1,tag2,tag3
Rename EC2 Instances As mentioned above, EC2 instances are named via a tag. However, renaming an instance by renaming its tag will cause the salt keys to mismatch. A rename function exists which renames both the instance, and the salt keys.
salt-cloud -a rename mymachine newname=yourmachine
Rename on Destroy When instances on EC2 are destroyed, there will be a lag between the time that the action is sent, and the time that Amazon cleans up the instance. During this time, the instance still retains a Name tag, which will cause a collision if the creation of an instance with the same name is attempted before the cleanup occurs. In order to avoid such collisions, Salt Cloud can be configured to rename instances when they are destroyed. The new name will look something like:
myinstance-DEL20f5b8ad4eb64ed88f2c428df80a1a0c
In order to enable this, add rename_on_destroy line to the main configuration file:
my-ec2-config: rename_on_destroy: True
Listing Images Normally, images can be queried on a cloud provider by passing the --list-images argument to Salt Cloud. This still holds true for EC2:
salt-cloud --list-images my-ec2-config
However, the full list of images on EC2 is extremely large, and querying all of the available images may cause Salt Cloud to behave as if frozen. Therefore, the default behavior of this option may be modified, by adding an owner argument to the provider configuration:
owner: aws-marketplace
The possible values for this setting are amazon, aws-marketplace, self, <AWS account ID> or all. The default setting is amazon.  Take note that all and aws-marketplace may cause Salt Cloud to appear as if it is freezing, as it tries to handle the large amount of data.
It is also possible to perform this query using different settings without modifying the configuration files. To do this, call the avail_images function directly:
salt-cloud -f avail_images my-ec2-config owner=aws-marketplace
EC2 Images The following are lists of available AMI images, generally sorted by OS. These lists are on 3rd-party websites, are not managed by Salt Stack in any way. They are provided here as a reference for those who are interested, and contain no warranty (express or implied) from anyone affiliated with Salt Stack. Most of them have never been used, much less tested, by the Salt Stack team.
• FreeBSD
• CentOS
• Ubuntu
• Debian
• OmniOS
• All Images on Amazon
NOTE: If image of a profile does not start with ami-, latest image with that name will be used. For example, to create a CentOS 7 profile, instead of using the AMI like image: ami-1caef165, we can use its name like image: 'CentOS Linux 7 x86_64 HVM EBS ENA 1803_01'.  We can also use a pattern like below to get the latest CentOS 7:
profile-id: provider: provider-name subnetid: subnet-XXXXXXXX image: 'CentOS Linux 7 x86_64 HVM EBS *' size: m1.medium ssh_username: centos securitygroupid: - sg-XXXXXXXX securitygroupname: - AnotherSecurityGroup - AndThirdSecurityGroup
show_image This is a function that describes an AMI on EC2. This will give insight as to the defaults that will be applied to an instance using a particular AMI.
show_instance This action is a thin wrapper around --full-query, which displays details on a single instance only. In an environment with several machines, this will save a user from having to sort through all instance data, just to examine a single instance.
$ salt-cloud -a show_instance myinstance
ebs_optimized This argument enables switching of the EbsOptimized setting which default to 'false'. Indicates whether the instance is optimized for EBS I/O. This optimization provides dedicated throughput to Amazon EBS and an optimized configuration stack to provide optimal Amazon EBS I/O performance. This optimization isn't available with all instance types. Additional usage charges apply when using an EBS-optimized instance.
This setting can be added to the profile or map file for an instance.
If set to True, this setting will enable an instance to be EbsOptimized
ebs_optimized: True
This can also be set as a cloud provider setting in the EC2 cloud configuration:
my-ec2-config: ebs_optimized: True
del_root_vol_on_destroy This argument overrides the default DeleteOnTermination setting in the AMI for the EBS root volumes for an instance. Many AMIs contain 'false' as a default, resulting in orphaned volumes in the EC2 account, which may unknowingly be charged to the account. This setting can be added to the profile or map file for an instance.
If set, this setting will apply to the root EBS volume
del_root_vol_on_destroy: True
my-ec2-config: del_root_vol_on_destroy: True
del_all_vols_on_destroy This argument overrides the default DeleteOnTermination setting in the AMI for the not-root EBS volumes for an instance. Many AMIs contain 'false' as a default, resulting in orphaned volumes in the EC2 account, which may unknowingly be charged to the account. This setting can be added to the profile or map file for an instance.
If set, this setting will apply to any (non-root) volumes that were created by salt-cloud using the 'volumes' setting.
The volumes will not be deleted under the following conditions * If a volume is detached before terminating the instance * If a volume is created without this setting and attached to the instance
del_all_vols_on_destroy: True
my-ec2-config: del_all_vols_on_destroy: True
The setting for this may be changed on all volumes of an existing instance using one of the following commands:
salt-cloud -a delvol_on_destroy myinstance salt-cloud -a keepvol_on_destroy myinstance salt-cloud -a show_delvol_on_destroy myinstance
The setting for this may be changed on a volume on an existing instance using one of the following commands:
salt-cloud -a delvol_on_destroy myinstance device=/dev/sda1 salt-cloud -a delvol_on_destroy myinstance volume_id=vol-1a2b3c4d salt-cloud -a keepvol_on_destroy myinstance device=/dev/sda1 salt-cloud -a keepvol_on_destroy myinstance volume_id=vol-1a2b3c4d salt-cloud -a show_delvol_on_destroy myinstance device=/dev/sda1 salt-cloud -a show_delvol_on_destroy myinstance volume_id=vol-1a2b3c4d
EC2 Termination Protection EC2 allows the user to enable and disable termination protection on a specific instance. An instance with this protection enabled cannot be destroyed. The EC2 driver adds a show_term_protect action to the regular EC2 functionality.
salt-cloud -a show_term_protect mymachine salt-cloud -a enable_term_protect mymachine salt-cloud -a disable_term_protect mymachine
Alternate Endpoint Normally, EC2 endpoints are build using the region and the service_url. The resulting endpoint would follow this pattern:
ec2.<region>.<service_url>
This results in an endpoint that looks like:
There are other projects that support an EC2 compatibility layer, which this scheme does not account for. This can be overridden by specifying the endpoint directly in the main cloud configuration file:
my-ec2-config: endpoint: myendpoint.example.com:1138/services/Cloud
Volume Management The EC2 driver has several functions and actions for management of EBS volumes.
Creating Volumes A volume may be created, independent of an instance. A zone must be specified.  A size or a snapshot may be specified (in GiB). If neither is given, a default size of 10 GiB will be used. If a snapshot is given, the size of the snapshot will be used.
The following parameters may also be set (when providing a snapshot OR size):
• type: choose between standard (magnetic disk), gp2 (SSD), or io1 (provisioned IOPS).  (default=standard)
• iops: the number of IOPS (only applicable to io1 volumes) (default varies on volume size)
• encrypted: enable encryption on the volume (default=false)
salt-cloud -f create_volume ec2 zone=us-east-1b salt-cloud -f create_volume ec2 zone=us-east-1b size=10 salt-cloud -f create_volume ec2 zone=us-east-1b snapshot=snap12345678 salt-cloud -f create_volume ec2 size=10 type=standard salt-cloud -f create_volume ec2 size=10 type=gp2 salt-cloud -f create_volume ec2 size=10 type=io1 iops=1000
Attaching Volumes Unattached volumes may be attached to an instance. The following values are required; name or instance_id, volume_id, and device.
salt-cloud -a attach_volume myinstance volume_id=vol-12345 device=/dev/sdb1
Show a Volume The details about an existing volume may be retrieved.
salt-cloud -a show_volume myinstance volume_id=vol-12345 salt-cloud -f show_volume ec2 volume_id=vol-12345
Detaching Volumes An existing volume may be detached from an instance.
salt-cloud -a detach_volume myinstance volume_id=vol-12345
Deleting Volumes A volume that is not attached to an instance may be deleted.
salt-cloud -f delete_volume ec2 volume_id=vol-12345
Managing Key Pairs The EC2 driver has the ability to manage key pairs.
Creating a Key Pair A key pair is required in order to create an instance. When creating a key pair with this function, the return data will contain a copy of the private key.  This private key is not stored by Amazon, will not be obtainable past this point, and should be stored immediately.
salt-cloud -f create_keypair ec2 keyname=mykeypair
Importing a Key Pair salt-cloud -f import_keypair ec2 keyname=mykeypair file=/path/to/id_rsa.pub
Show a Key Pair This function will show the details related to a key pair, not including the private key itself (which is not stored by Amazon).
salt-cloud -f show_keypair ec2 keyname=mykeypair
Delete a Key Pair This function removes the key pair from Amazon.
salt-cloud -f delete_keypair ec2 keyname=mykeypair
Launching instances into a VPC Simple launching into a VPC In the amazon web interface, identify the id or the name of the subnet into which your image should be created. Then, edit your cloud.profiles file like so:-
profile-id: provider: provider-name subnetid: subnet-XXXXXXXX image: ami-XXXXXXXX size: m1.medium ssh_username: ubuntu securitygroupid: - sg-XXXXXXXX securitygroupname: - AnotherSecurityGroup - AndThirdSecurityGroup
Note that 'subnetid' takes precedence over 'subnetname', but 'securitygroupid' and 'securitygroupname' are merged together to generate a single list for SecurityGroups of instances.
Specifying interface properties New in version 2014.7.0.
Launching into a VPC allows you to specify more complex configurations for the network interfaces of your virtual machines, for example:-
profile-id: provider: provider-name image: ami-XXXXXXXX size: m1.medium ssh_username: ubuntu
# Do not include either 'subnetid', 'subnetname', 'securitygroupid' or # 'securitygroupname' here if you are going to manually specify # interface configuration # network_interfaces: - DeviceIndex: 0 SubnetId: subnet-XXXXXXXX SecurityGroupId: - sg-XXXXXXXX
# Uncomment this line if you would like to set an explicit private # IP address for the ec2 instance # # PrivateIpAddress: 192.168.1.66
# Uncomment this to associate an existing Elastic IP Address with # this network interface: # # associate_eip: eipalloc-XXXXXXXX
# You can allocate more than one IP address to an interface. Use the # 'ip addr list' command to see them. # # SecondaryPrivateIpAddressCount: 2
# Uncomment this to allocate a new Elastic IP Address to this # interface (will be associated with the primary private ip address # of the interface # # allocate_new_eip: True
# Uncomment this instead to allocate a new Elastic IP Address to # both the primary private ip address and each of the secondary ones # allocate_new_eips: True
# Uncomment this if you're creating NAT instances. Allows an instance # to accept IP packets with destinations other than itself. # SourceDestCheck: False
- DeviceIndex: 1 subnetname: XXXXXXXX-Subnet securitygroupname: - XXXXXXXX-SecurityGroup - YYYYYYYY-SecurityGroup
Note that it is an error to assign a 'subnetid', 'subnetname', 'securitygroupid' or 'securitygroupname' to a profile where the interfaces are manually configured like this. These are both really properties of each network interface, not of the machine itself.
Getting Started With GoGrid GoGrid is a public cloud host that supports Linux and Windows.
Configuration To use Salt Cloud with GoGrid log into the GoGrid web interface and create an API key. Do this by clicking on "My Account" and then going to the API Keys tab.
my-gogrid-config: driver: gogrid apikey: asdff7896asdh789 sharedsecret: saltybacon
NOTE: A Note about using Map files with GoGrid:
Due to limitations in the GoGrid API, instances cannot be provisioned in parallel with the GoGrid driver. Map files will work with GoGrid, but the -P argument should not be used on maps referencing GoGrid instances.
gogrid_512: provider: my-gogrid-config size: 512MB image: CentOS 6.2 (64-bit) w/ None
# salt-cloud --list-sizes my-gogrid-config my-gogrid-config: ---------- gogrid: ---------- 512MB: ---------- bandwidth: None disk: 30 driver: get_uuid: id: 512MB name: 512MB price: 0.095 ram: 512 uuid: bde1e4d7c3a643536e42a35142c7caac34b060e9 ...SNIP...
# salt-cloud --list-images my-gogrid-config my-gogrid-config: ---------- gogrid: ---------- CentOS 6.4 (64-bit) w/ None: ---------- driver: extra: ---------- get_uuid: id: 18094 name: CentOS 6.4 (64-bit) w/ None uuid: bfd4055389919e01aa6261828a96cf54c8dcc2c4 ...SNIP...
Assigning IPs New in version 2015.8.0.
The GoGrid API allows IP addresses to be manually assigned. Salt Cloud supports this functionality by allowing an IP address to be specified using the assign_public_ip argument. This likely makes the most sense inside a map file, but it may also be used inside a profile.
gogrid_512: provider: my-gogrid-config size: 512MB image: CentOS 6.2 (64-bit) w/ None assign_public_ip: 11.38.257.42
Getting Started With Google Compute Engine Google Compute Engine (GCE) is Google-infrastructure as a service that lets you run your large-scale computing workloads on virtual machines. This document covers how to use Salt Cloud to provision and manage your virtual machines hosted within Google's infrastructure.
You can find out more about GCE and other Google Cloud Platform services at https://cloud.google.com.
Dependencies • LibCloud >= 1.0.0
• A Google Cloud Platform account with Compute Engine enabled
• A registered Service Account for authorization
• Oh, and obviously you'll need salt
Google Compute Engine Setup 1. Sign up for Google Cloud Platform
Go to https://cloud.google.com and use your Google account to sign up for Google Cloud Platform and complete the guided instructions.
2. Create a Project
Next, go to the console at https://cloud.google.com/console and create a new Project.  Make sure to select your new Project if you are not automatically directed to the Project.
Projects are a way of grouping together related users, services, and billing.  You may opt to create multiple Projects and the remaining instructions will need to be completed for each Project if you wish to use GCE and Salt Cloud to manage your virtual machines.
3. Enable the Google Compute Engine service
In your Project, either just click Compute Engine to the left, or go to the APIs & auth section and APIs link and enable the Google Compute Engine service.
4. Create a Service Account
To set up authorization, navigate to APIs & auth section and then the Credentials link and click the CREATE NEW CLIENT ID button. Select Service Account and click the Create Client ID button. This will automatically download a .json file, which may or may not be used in later steps, depending on your version of libcloud.
Look for a new Service Account section in the page and record the generated email address for the matching key/fingerprint. The email address will be used in the service_account_email_address of the /etc/salt/cloud.providers or the /etc/salt/cloud.providers.d/*.conf file.
5. Key Format
NOTE: If you are using libcloud >= 0.17.0 it is recommended that you use the JSON format file you downloaded above and skip to the Provider Configuration section below, using the JSON file in place of 'NEW.pem' in the documentation.
If you are using an older version of libcloud or are unsure of the version you have, please follow the instructions below to generate and format a new P12 key.
In the new Service Account section, click Generate new P12 key, which will automatically download a .p12 private key file. The .p12 private key needs to be converted to a format compatible with libcloud.  This new Google-generated private key was encrypted using notasecret as a passphrase. Use the following command and record the location of the converted private key and record the location for use in the service_account_private_key of the /etc/salt/cloud file:
openssl pkcs12 -in ORIG.p12 -passin pass:notasecret \ -nodes -nocerts | openssl rsa -out NEW.pem
Provider Configuration Set up the provider cloud config at /etc/salt/cloud.providers or /etc/salt/cloud.providers.d/*.conf:
gce-config: # Set up the Project name and Service Account authorization project: "your-project-id" service_account_email_address: "123-a5gt@developer.gserviceaccount.com" service_account_private_key: "/path/to/your/NEW.pem"
# Set up the location of the salt master minion: master: saltmaster.example.com
driver: gce
NOTE: Empty strings as values for service_account_private_key and service_account_email_address can be used on GCE instances. This will result in the service account assigned to the GCE instance being used.
NOTE: The value provided for project must not contain underscores or spaces and is labeled as "Project ID" on the Google Developers Console.
Profile Configuration Set up an initial profile at /etc/salt/cloud.profiles or /etc/salt/cloud.profiles.d/*.conf:
my-gce-profile: image: centos-6 size: n1-standard-1 location: europe-west1-b network: default subnetwork: default tags: '["one", "two", "three"]' metadata: '{"one": "1", "2": "two"}' use_persistent_disk: True delete_boot_pd: False deploy: True make_master: False provider: gce-config
The profile can be realized now with a salt command:
salt-cloud -p my-gce-profile gce-instance
This will create an salt minion instance named gce-instance in GCE.  If the command was executed on the salt-master, its Salt key will automatically be signed on the master.
Once the instance has been created with a salt-minion installed, connectivity to it can be verified with Salt:
salt gce-instance test.version
GCE Specific Settings Consult the sample profile below for more information about GCE specific settings. Some of them are mandatory and are properly labeled below but typically also include a hard-coded default.
Initial Profile Set up an initial profile at /etc/salt/cloud.profiles or /etc/salt/cloud.profiles.d/gce.conf:
my-gce-profile: image: centos-6 size: n1-standard-1 location: europe-west1-b network: default subnetwork: default tags: '["one", "two", "three"]' metadata: '{"one": "1", "2": "two"}' use_persistent_disk: True delete_boot_pd: False ssh_interface: public_ips external_ip: "ephemeral"
image Image is used to define what Operating System image should be used to for the instance. Examples are Debian 7 (wheezy) and CentOS 6. Required.
size A 'size', in GCE terms, refers to the instance's 'machine type'. See the on-line documentation for a complete list of GCE machine types. Required.
location A 'location', in GCE terms, refers to the instance's 'zone'. GCE has the notion of both Regions (e.g. us-central1, europe-west1, etc) and Zones (e.g. us-central1-a, us-central1-b, etc). Required.
network Use this setting to define the network resource for the instance.  All GCE projects contain a network named 'default' but it's possible to use this setting to create instances belonging to a different network resource.
subnetwork Use this setting to define the subnetwork an instance will be created in. This requires that the network your instance is created under has a mode of 'custom' or 'auto'.  Additionally, the subnetwork your instance is created under is associated with the location you provide.
tags GCE supports instance/network tags and this setting allows you to set custom tags. It should be a list of strings and must be parse-able by the python ast.literal_eval() function to convert it to a python list.
metadata GCE supports instance metadata and this setting allows you to set custom metadata. It should be a hash of key/value strings and parse-able by the python ast.literal_eval() function to convert it to a python dictionary.
use_persistent_disk Use this setting to ensure that when new instances are created, they will use a persistent disk to preserve data between instance terminations and re-creations.
delete_boot_pd In the event that you wish the boot persistent disk to be permanently deleted when you destroy an instance, set delete_boot_pd to True.
ssh_interface New in version 2015.5.0.
Specify whether to use public or private IP for deploy script.
Valid options are:
• private_ips: The salt-master is also hosted with GCE
• public_ips: The salt-master is hosted outside of GCE
external_ip Per instance setting: Used a named fixed IP address to this host.
• ephemeral: The host will use a GCE ephemeral IP
• None: No external IP will be configured on this host.
Optionally, pass the name of a GCE address to use a fixed IP address.  If the address does not already exist, it will be created.
ex_disk_type GCE supports two different disk types, pd-standard and pd-ssd.  The default disk type setting is pd-standard. To specify using an SSD disk, set pd-ssd as the value.
ip_forwarding GCE instances can be enabled to use IP Forwarding. When set to True, this options allows the instance to send/receive non-matching src/dst packets. Default is False.
Profile with scopes Scopes can be specified by setting the optional ex_service_accounts key in your cloud profile. The following example enables the bigquery scope.
my-gce-profile: image: centos-6 ssh_username: salt size: f1-micro location: us-central1-a network: default subnetwork: default tags: '["one", "two", "three"]' metadata: '{"one": "1", "2": "two", "sshKeys": ""}' use_persistent_disk: True delete_boot_pd: False deploy: False make_master: False provider: gce-config ex_service_accounts: - scopes: - bigquery
Email can also be specified as an (optional) parameter.
my-gce-profile: ...snip ex_service_accounts: - scopes: - bigquery email: default
There can be multiple entries for scopes since ex-service_accounts accepts a list of dictionaries. For more information refer to the libcloud documentation on specifying service account scopes.
SSH Remote Access GCE instances do not allow remote access to the root user by default. Instead, another user must be used to run the deploy script using sudo. Append something like this to /etc/salt/cloud.profiles or /etc/salt/cloud.profiles.d/*.conf:
my-gce-profile: ...
# SSH to GCE instances as gceuser ssh_username: gceuser
# Use the local private SSH key file located here ssh_keyfile: /etc/cloud/google_compute_engine
If you have not already used this SSH key to login to instances in this GCE project you will also need to add the public key to your projects metadata at https://cloud.google.com/console. You could also add it via the metadata setting too:
metadata: '{"one": "1", "2": "two", "sshKeys": "gceuser:ssh-rsa <Your SSH Public Key> gceuser@host"}'
Single instance details This action is a thin wrapper around --full-query, which displays details on a single instance only. In an environment with several machines, this will save a user from having to sort through all instance data, just to examine a single instance.
Destroy, persistent disks, and metadata As noted in the provider configuration, it's possible to force the boot persistent disk to be deleted when you destroy the instance.  The way that this has been implemented is to use the instance metadata to record the cloud profile used when creating the instance.  When destroy is called, if the instance contains a salt-cloud-profile key, it's value is used to reference the matching profile to determine if delete_boot_pd is set to True.
Be aware that any GCE instances created with salt cloud will contain this custom salt-cloud-profile metadata entry.
List various resources It's also possible to list several GCE resources similar to what can be done with other providers.  The following commands can be used to list GCE zones (locations), machine types (sizes), and images.
salt-cloud --list-locations gce salt-cloud --list-sizes gce salt-cloud --list-images gce
Persistent Disk The Compute Engine provider provides functions via salt-cloud to manage your Persistent Disks. You can create and destroy disks as well as attach and detach them from running instances.
Create When creating a disk, you can create an empty disk and specify its size (in GB), or specify either an 'image' or 'snapshot'.
salt-cloud -f create_disk gce disk_name=pd location=us-central1-b size=200
Delete Deleting a disk only requires the name of the disk to delete
salt-cloud -f delete_disk gce disk_name=old-backup
Attach Attaching a disk to an existing instance is really an 'action' and requires both an instance name and disk name. It's possible to use this ation to create bootable persistent disks if necessary. Compute Engine also supports attaching a persistent disk in READ_ONLY mode to multiple instances at the same time (but then cannot be attached in READ_WRITE to any instance).
salt-cloud -a attach_disk myinstance disk_name=pd mode=READ_WRITE boot=yes
Detach Detaching a disk is also an action against an instance and only requires the name of the disk. Note that this does not safely sync and umount the disk from the instance. To ensure no data loss, you must first make sure the disk is unmounted from the instance.
salt-cloud -a detach_disk myinstance disk_name=pd
Show disk It's also possible to look up the details for an existing disk with either a function or an action.
salt-cloud -a show_disk myinstance disk_name=pd salt-cloud -f show_disk gce disk_name=pd
Create snapshot You can take a snapshot of an existing disk's content. The snapshot can then in turn be used to create other persistent disks. Note that to prevent data corruption, it is strongly suggested that you unmount the disk prior to taking a snapshot. You must name the snapshot and provide the name of the disk.
salt-cloud -f create_snapshot gce name=backup-20140226 disk_name=pd
Delete snapshot You can delete a snapshot when it's no longer needed by specifying the name of the snapshot.
salt-cloud -f delete_snapshot gce name=backup-20140226
Show snapshot Use this function to look up information about the snapshot.
salt-cloud -f show_snapshot gce name=backup-20140226
Networking Compute Engine supports multiple private networks per project. Instances within a private network can easily communicate with each other by an internal DNS service that resolves instance names. Instances within a private network can also communicate with either directly without needing special routing or firewall rules even if they span different regions/zones.
Networks also support custom firewall rules. By default, traffic between instances on the same private network is open to all ports and protocols. Inbound SSH traffic (port 22) is also allowed but all other inbound traffic is blocked.
Create network New networks require a name and CIDR range if they don't have a 'mode'. Optionally, 'mode' can be provided. Supported modes are 'auto', 'custom', 'legacy'.  Optionally, 'description' can be provided to add an extra note to your network.  New instances can be created and added to this network by setting the network name during create. It is not possible to add/remove existing instances to a network.
salt-cloud -f create_network gce name=mynet cidr=10.10.10.0/24 salt-cloud -f create_network gce name=mynet mode=auto description=some optional info.
Destroy network Destroy a network by specifying the name. If a resource is currently using the target network an exception will be raised.
salt-cloud -f delete_network gce name=mynet
Show network Specify the network name to view information about the network.
salt-cloud -f show_network gce name=mynet
Create subnetwork New subnetworks require a name, region, and CIDR range.  Optionally, 'description' can be provided to add an extra note to your subnetwork. New instances can be created and added to this subnetwork by setting the subnetwork name during create. It is not possible to add/remove existing instances to a subnetwork.
salt-cloud -f create_subnetwork gce name=mynet network=mynet region=us-central1 cidr=10.0.10.0/24 salt-cloud -f create_subnetwork gce name=mynet network=mynet region=us-central1 cidr=10.10.10.0/24 description=some info about my subnet.
Destroy subnetwork Destroy a subnetwork by specifying the name and region. If a resource is currently using the target subnetwork an exception will be raised.
salt-cloud -f delete_subnetwork gce name=mynet region=us-central1
Show subnetwork Specify the subnetwork name to view information about the subnetwork.
salt-cloud -f show_subnetwork gce name=mynet
Create address Create a new named static IP address in a region.
salt-cloud -f create_address gce name=my-fixed-ip region=us-central1
Delete address Delete an existing named fixed IP address.
salt-cloud -f delete_address gce name=my-fixed-ip region=us-central1
Show address View details on a named address.
salt-cloud -f show_address gce name=my-fixed-ip region=us-central1
Create firewall You'll need to create custom firewall rules if you want to allow other traffic than what is described above. For instance, if you run a web service on your instances, you'll need to explicitly allow HTTP and/or SSL traffic.  The firewall rule must have a name and it will use the 'default' network unless otherwise specified with a 'network' attribute. Firewalls also support instance tags for source/destination
salt-cloud -f create_fwrule gce name=web allow=tcp:80,tcp:443,icmp