Package: simplec2snap
Simple EC2 Snapshot===================Simple solution to backup ec2 instances using snapshotsWith Simple EC2 Snapshot supports:* Hot snapshots (by default) and Cold snapshots* Multiple instances snapshot in one line* Detection of doubles* Filters by tags (allowing wildcards) or by instance IDs* Credentials file multiple with profiles* Limit the number of snapshots* Restrict snapshots to data disks only* Snapshot retention over time or for a given number## InstallationTo install the tool, the simplest solution is to use pip:```pip install simplec2snap```## FiltersYou can first decide to choose what to backup with Instance ID or Tags. You can set multiple tags and/or multiple instance ID at the same time.For example, if I want to snapshot 2 instances in the same command line, it should looks like this:```> ./simplec2snap.py -i i-ad0fcc4b -i i-56489db2```To snapshot multiple instances by selecting multiple tags. Here is an example with 2 tagsi, so it should match both:```> ./simplec2snap.py -t Name 'instance*' -t env prod```If you want to add an instance in addition of the previous tags:```> ./simplec2snap.py -t Name 'instance*' -t env prod -i i-ad0fcc4b```## Credentials fileYou can use a credentials file with several profiles. It should looks like this:```ini[default]aws_region = <region>aws_access_key_id = <key_id>aws_secret_access_key = <access_key>#[profile profile1]#aws_region = <region>#aws_access_key_id = <key_id>#aws_secret_access_key = <access_key>```The default one should be located in '~/.aws_cred'. You can override this with '-c' argument and '-p' to specify the profile fulfill into brackets.## Dry Run modeUse the dry run mode (enabled by default) to see what actions will be performed when selecting a tag Name or an instance:```> ./simplec2snap.py -t Name "instance-name*"2015-01-26 17:05:25,954 [INFO] == Launching dry run mode ==2015-01-26 17:05:25,954 [INFO] Connecting to AWS2015-01-26 17:05:25,955 [INFO] Getting instances information2015-01-26 17:05:28,341 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:05:28,341 [INFO] Hot snapshot made for vol-fa415bfd(/dev/sdb)2015-01-26 17:05:28,341 [INFO] Hot snapshot made for vol-22465c25(/dev/sda)2015-01-26 17:05:28,341 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:05:28,341 [INFO] Hot snapshot made for vol-9d465c9a(/dev/sda)2015-01-26 17:05:28,341 [INFO] Hot snapshot made for vol-9c465c9b(/dev/sdb)```## Run modeIf you're ok with the previous dry run, then add '-u' for run mode:```> ./simplec2snap.py -t Name "instance-name*" -u2015-01-26 17:06:19,163 [INFO] == Launching run mode ==2015-01-26 17:06:19,163 [INFO] Connecting to AWS2015-01-26 17:06:19,163 [INFO] Getting instances information2015-01-26 17:06:21,083 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:06:21,352 [INFO] Hot snapshot made for vol-fa415bfd(/dev/sdb) - snap-35adb8c42015-01-26 17:06:21,587 [INFO] Hot snapshot made for vol-22465c25(/dev/sda) - snap-36adb8c72015-01-26 17:06:21,587 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:06:21,832 [INFO] Hot snapshot made for vol-9d465c9a(/dev/sda) - snap-3cadb8cd2015-01-26 17:06:22,087 [INFO] Hot snapshot made for vol-9c465c9b(/dev/sdb) - snap-21adb8d0```## Hot vs Cold snapshotBy default Hot mode is selected to perform snapshot without stopping instances. However, this may not be the best choice in some case, like for database purpose. To get a full consistent snapshot of your EC2 with attached EBS, you have to make a Cold snapshot which involves to shutdown, snapshot and start instance.To do so, you have to add '-H' option:```> ./simplec2snap.py -t Name "instance-name*" -u -H2015-01-26 17:07:10,281 [INFO] == Launching run mode ==2015-01-26 17:07:10,281 [INFO] Connecting to AWS2015-01-26 17:07:10,282 [INFO] Getting instances information2015-01-26 17:07:12,490 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:07:12,490 [INFO] Instance is going to be shutdown2015-01-26 17:07:48,871 [INFO] Instance i-e16cc205 now stopped !2015-01-26 17:07:49,134 [INFO] Cold snapshot made for vol-fa415bfd(/dev/sdb) - snap-a8afba592015-01-26 17:07:49,379 [INFO] Cold snapshot made for vol-22465c25(/dev/sda) - snap-adafba5c2015-01-26 17:07:49,379 [INFO] Instance is going to be started2015-01-26 17:08:20,565 [INFO] Instance i-e16cc205 now running !2015-01-26 17:08:20,565 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:08:20,565 [INFO] Instance is going to be shutdown2015-01-26 17:08:51,617 [INFO] Instance i-6f6ec08b now stopped !2015-01-26 17:08:51,853 [INFO] Cold snapshot made for vol-9d465c9a(/dev/sda) - snap-b1aebb402015-01-26 17:08:52,098 [INFO] Cold snapshot made for vol-9c465c9b(/dev/sdb) - snap-b2aebb432015-01-26 17:08:52,098 [INFO] Instance is going to be started2015-01-26 17:09:09,467 [INFO] Instance i-6f6ec08b now running !```## Limit snapshots for auto-scaling groupIn auto-scaling groups, you normally have x time the same running intance. Snapshoting a huge number of time the same instance may not be very interesting. That's why you can limit the number of snapshot by using '-l' command followed by the number of desired snapshot. If I only want one:```> ./simplec2snap.py -t Name "instance-name*" -l 12015-01-26 17:11:27,561 [INFO] == Launching dry run mode ==2015-01-26 17:11:27,561 [INFO] Connecting to AWS2015-01-26 17:11:27,562 [INFO] Getting instances information2015-01-26 17:11:29,659 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:11:29,659 [INFO] Hot snapshot made for vol-fa415bfd(/dev/sdb)2015-01-26 17:11:29,659 [INFO] Hot snapshot made for vol-22465c25(/dev/sda)2015-01-26 17:11:29,659 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:11:29,660 [INFO] The requested limit of snapshots has been reached: 1```## Remove root device from snapshotsStill for auto-scaling groups, your root device may not be required to snapshot. Generally because it may be builded from a configuration manager and you just don't care of it. So the goal is to remove it from the snapshot list, you can so use '-o' option:```> ./simplec2snap.py -t Name "instance-name*" -o2015-01-26 17:11:50,757 [INFO] == Launching dry run mode ==2015-01-26 17:11:50,757 [INFO] Connecting to AWS2015-01-26 17:11:50,758 [INFO] Getting instances information2015-01-26 17:11:52,708 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:11:52,708 [INFO] Hot snapshot made for vol-fa415bfd(/dev/sdb)2015-01-26 17:11:52,708 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:11:52,708 [INFO] Hot snapshot made for vol-9c465c9b(/dev/sdb)```## Snapshot retentionThere are 2 methods for snapshot retention. Choose the best one for your needs.### Keep snapshots over timeYou can define the retention of your backups. You need to specify 2 args:* Number: sepcify a number for day, week... which is defined in the second arg* Time element: specify s(second), m(min), h(hour), d(day), w(week), M(month), y(year)So for example, if you want to keep snapshots for 3 weeks and delete the old ones, you have to set: 3 w.Here is a basic example where I want to delete snapshots older than 10 days:```> ./simplec2snap.py -t Name "instance-name*" -n -g 10 d2015-01-28 10:12:11,216 [INFO] == Launching dry run mode ==2015-01-28 10:12:11,217 [INFO] Connecting to AWS2015-01-28 10:12:11,217 [INFO] Getting instances information2015-01-28 10:12:13,080 [INFO] Working on instance i-e16cc205 (pmavro-test2)2015-01-28 10:12:14,109 [INFO] Deleting snapshot snap-b427c144 (vol-fa415bfd|/dev/sdb)2015-01-28 10:12:14,205 [INFO] Deleting snapshot snap-a327c153 (vol-22465c25|/dev/sda)2015-01-28 10:12:14,205 [INFO] Working on instance i-6f6ec08b (pmavro-test1)2015-01-28 10:12:14,321 [INFO] Deleting snapshot snap-a627c156 (vol-9d465c9a|/dev/sda)2015-01-28 10:12:14,440 [INFO] Deleting snapshot snap-a927c159 (vol-9c465c9b|/dev/sdb)```Here -n is used to not make snapshots, only delete olds. But you can ask on the same line to make snapshots AND remove old ones:```> ./simplec2snap.py -t Name "instance-name*" -g 10 m2015-01-26 17:22:43,263 [INFO] == Launching dry run mode ==2015-01-26 17:22:43,263 [INFO] Connecting to AWS2015-01-26 17:22:43,264 [INFO] Getting instances information2015-01-26 17:22:46,217 [INFO] Working on instance i-e16cc205 (instance-name2)2015-01-26 17:22:46,218 [INFO] Hot snapshot made for vol-fa415bfd(/dev/sdb)2015-01-26 17:22:46,218 [INFO] Hot snapshot made for vol-22465c25(/dev/sda)2015-01-26 17:22:47,328 [INFO] Deleting snapshot snap-a8afba59 (vol-fa415bfd|/dev/sdb)2015-01-26 17:22:47,491 [INFO] Deleting snapshot snap-adafba5c (vol-22465c25|/dev/sda)2015-01-26 17:22:47,491 [INFO] Working on instance i-6f6ec08b (instance-name1)2015-01-26 17:22:47,491 [INFO] Hot snapshot made for vol-9d465c9a(/dev/sda)2015-01-26 17:22:47,492 [INFO] Hot snapshot made for vol-9c465c9b(/dev/sdb)2015-01-26 17:22:47,669 [INFO] Deleting snapshot snap-b1aebb40 (vol-9d465c9a|/dev/sda)2015-01-26 17:22:47,842 [INFO] Deleting snapshot snap-b2aebb43 (vol-9c465c9b|/dev/sdb)```### Keep at least a given number of snapshotsAnother solution to manage the retention of your snapshots is to specify how many snapshots you want to keep. For example, if I have 5 snapshots per device of an instance and want to keep the last 4 ones:```> ./simplec2snap.py -t Name "instance-name*" -n -d 42015-01-28 10:14:02,713 [INFO] == Launching dry run mode ==2015-01-28 10:14:02,713 [INFO] Connecting to AWS2015-01-28 10:14:02,713 [INFO] Getting instances information2015-01-28 10:14:04,328 [INFO] Working on instance i-e16cc205 (pmavro-test2)2015-01-28 10:14:05,316 [INFO] Deleting snapshot snap-b427c144 (vol-fa415bfd|/dev/sdb)2015-01-28 10:14:05,444 [INFO] Deleting snapshot snap-a327c153 (vol-22465c25|/dev/sda)2015-01-28 10:14:05,444 [INFO] Working on instance i-6f6ec08b (pmavro-test1)2015-01-28 10:14:05,540 [INFO] Deleting snapshot snap-a627c156 (vol-9d465c9a|/dev/sda)2015-01-28 10:14:05,654 [INFO] Deleting snapshot snap-a927c159 (vol-9c465c9b|/dev/sdb)```## HelpHere is the help with the complete list of options:```> ./simplec2snap.pyusage: simplec2snap.py [-h] [-r REGION] [-k KEY_ID] [-a ACCESS_KEY][-c CREDENTIALS] [-p CRED_PROFILE] [-i INSTANCE_ID][-t ARG ARG] [-u] [-l LIMIT] [-H] [-m COLDSNAP_TIMEOUT][-o] [-g ARG ARG] [-d KEEP_LAST_SNAPSHOTS] [-n][-f FILE] [-s] [-v LEVEL] [-V]Simple EC2 Snapshot utilityoptional arguments:-h, --help            show this help message and exit-r REGION, --region REGIONSet AWS region (ex: eu-west-1) (default: None)-k KEY_ID, --key_id KEY_IDSet AWS Key ID (default: None)-a ACCESS_KEY, --access_key ACCESS_KEYSet AWS Access Key (default: None)-c CREDENTIALS, --credentials CREDENTIALSCredentials file path (default:/home/pmavro/.aws_cred)-p CRED_PROFILE, --profile CRED_PROFILECredentials profile file defined in credentials file(default: default)-i INSTANCE_ID, --instance INSTANCE_IDInstance ID (ex: i-00000000 or all) (default: [])-t ARG ARG, --tags ARG ARGSelect tags with values (ex: tagname value) (default:[])-u, --dry_run         Define if it should make snapshot or just dry run(default: True)-l LIMIT, --limit LIMITLimit the number of snapshot (can be usefull withauto-scaling groups) (default: -1)-H, --cold_snap       Make cold snapshot for a better consistency(Recommended) (default: False)-m COLDSNAP_TIMEOUT, --timeout COLDSNAP_TIMEOUTInstance timeout (in seconds) for stop and startduring a cold snapshot (default: 600)-o, --no_root_device  Do not snapshot root device (default: False)-g ARG ARG, --max_age ARG ARGMaximum snapshot age to keep (<int> <s/m/h/d/w/M/y>)(ex: 1 h for one hour) (default: [])-d KEEP_LAST_SNAPSHOTS, --keep_last_snapshots KEEP_LAST_SNAPSHOTSKeep the x last snapshots (default: 0)-n, --no_snap         Do not make snapshot (useful when combien to -goption) (default: False)-f FILE, --file_output FILESet an output file (default: None)-s, --stdout          Log output to console (stdout) (default: True)-v LEVEL, --verbosity LEVELVerbosity level: DEBUG/INFO/ERROR/CRITICAL (default:INFO)-V, --version         Print version number```