Package: awstracer
AWS CLI Tracer"A poor man's Cloud Formation utility" --meIntroductionawstracerconsists of two small utitilities which hook into theawscommand-line interface internal event mechanism. Using it you can record a sequence ofawscommands to a trace-file. The player will allow you to replay that sequence of commands under for example a different configured AWS profile or against a different AWS region. Think of it as a set of poor man's Cloud Formation utilities. It's useful for when you have to re-run a set of commands or quickly test a bunch of things without having to switch back to the console the entire time. And it's also a whole lot quicker than having to write your ownawscliandbotocorelogic.Under normal circumstances these utilities would barely be more useful than using a simple shell-script which wraps aroundawscommands. However, the player features some logic that allows it to derive relationships between subsequent calls. This way it can automatically determine that the return value of one command should be supplied to the next request. Think for example of using a command which creates a resource and returns an ARN and the next command then using that specific ARN again. The player has the ability to do a dry run of the sequence of commands in the trace file. It will then colorize the output and highlight the replaced variables to show where the substitutions in a sequence of requests will appear.LimitationsVery complex traces with a set of similar commands might end up yielding unpredictable results. If one for example does anaws iam create-usercall twice then the second call's parameters will be automatically substituted for the ones of the first one. To inspect what would happen it's advisable to look at the output of a dryrun first. The examples below should make clearer what appropriate use-cases are and how one can useawstracer.InstallationPlease note that this tool requires at least Python>= 3.6. To install from source simply clone the repository and run:$ python3 setup.py sdist
$ pip3 install dist/awstracer-<version>.tar.gzUsageFirst runawstrace-recto run a recording of aws commands. There is the ability, enabled via a command-line switch, to also support arbitrary shell commands. These willNOTbe replayed or even captured in the trace. However this can be useful when you want to make sure a call to AWS IAM for example has settled. You can then execute for example a sleep command before executing the next call toaws.After the trace has been recorded you can replay it withawstrace-play. There are several switches to enable debugging, force a continuation of a trace execution when one intermediate command fails and so on. For more information simply runawstrace-play -h. To replay a trace against a different region or profile simply use the--profileor--regionswitches.Please note that bothawstrace-playandawstrace-recare very light wrappers around the standard aws cli. This means that it will automatically import your profiles from~/.aws/credentialsor load IAM access keys from the environment. From that perspective everything works exactly like usual.Overriding request parameters can be done via-por--param. Request parameters tend to be similarly cased to how theaws clistyles them. That means that even if the request and response use for exampleUserNameyou specify it on the commandline with--user-name. Forawstrace-playthat means you would use something like-p user-name test-userto override the value in the trace.Usage Example 1: Creating a DynamoDB table and adding data to itThis is the example that can also be seen in the recorded terminal session above. Let's create a tracefile that creates a DynamoDB table namedMusicand inserts a song in the table. The commands are taken directly from theAmazon DynamoDB Developer Guide. This looks something like this. Please note that the output is truncated.$ awstrace-rec --trace-file create_table.trace
(rec) aws dynamodb create-table \
>     --table-name Music \
>     --attribute-definitions \
>         AttributeName=Artist,AttributeType=S \
>         AttributeName=SongTitle,AttributeType=S \
>     --key-schema \
>         AttributeName=Artist,KeyType=HASH \
>         AttributeName=SongTitle,KeyType=RANGE \
> --provisioned-throughput \
>         ReadCapacityUnits=10,WriteCapacityUnits=5
[...]
Add command to trace cache? [y/N]: y
(rec) aws dynamodb put-item \
> --table-name Music  \
> --item \
>     '{"Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Call Me Today"}, "AlbumTitle": {"S": "Somewhat Famous"}, "Awards": {"N": "1"}}'
Add command to trace cache? [y/N]: y
(rec)
Save cached trace to create_table.trace? [y/N]: yNow we can simply replay our trace. If we play it we get the following error as the table already exists:$ awstrace-play --trace-file create_table.trace
(play) aws dynamodb create-table --attribute-definitions '[{"AttributeName": "Artist", "AttributeType": "S"}, {"AttributeName": "SongTitle", "AttributeType": "S"}]' --key-schema '[{"AttributeName": "Artist", "KeyType": "HASH"}, {"AttributeName": "SongTitle", "KeyType": "RANGE"}]' --provisioned-throughput '{"ReadCapacityUnits": 10, "WriteCapacityUnits": 5}' --table-name Music

An error occurred (ResourceInUseException) when calling the CreateTable operation: Table already exists: MusicWe can immediately run the trace against a different region, profile or endpoint. To do so we simply use the--region,--profileor--endpointparameters similarly to how this works with the AWS CLI. For example to run the same trace under theadmin-profileagainst the endpointhttps://beta-service-url.comwe can do the following:$ awstrace-play --trace-file create_table.trace --profile admin-profile --endpoint https://beta-service-url.comThe player has the ability to automatically derive relationships between subsequent commands. This means that we have the ability to override a parameter inside a trace file without having to edit the trace file or edit any of the commands themselves. For example if we want to create a table with a different name but still want to get the data inserted to that new table we simply do this:$ awstrace-play --trace-file create_table.trace -p table-name test-table
[...]
$ aws dynamodb list-tables
{
    "TableNames": [
        "Music",
        "test-table",
    ]
}We can see thattest-tablewas created. To check if the data actually got inserted we can use the following to see if we can get an item returned from the table. If that works it means the player did its job correctly and it replayed the trace with the overridden parameters properly.$ aws dynamodb get-item --consistent-read --table-name test-table --key '{ "Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Call Me Today"}}
{
    "Item": {
        "AlbumTitle": {
            "S": "Somewhat Famous"
        },
        "Awards": {
            "N": "1"
        },
        "Artist": {
            "S": "No One You Know"
        },
        "SongTitle": {
            "S": "Call Me Today"
        }
    }
}Usage Example 2: Creating AWS users and policiesWe start a recording session. First we create usertest-user1. We then create a policy fromfile://policy.jsonnamedtest-policy1and subsequently attach the policy to the user.$ awstrace-rec --trace-file create_user.trace
(rec) aws iam create-user --user-name test-user-1
[...]
(rec) aws iam create-policy --policy-name test-policy-1 --policy-document file://policy.json
{
    "Policy": {
    [...]
        "Arn": "arn:aws:iam::111111111111:policy/policy1",
    [...]
    }
}
(rec) aws iam attach-user-policy --user-name test-user-1 --policy-arn arn:aws:iam::111111111111:policy/policy1
[...]
aws iam list-attached-user-policies --user-name test-user-1
{
    "AttachedPolicies": [
        {
            "PolicyName": "policy1",
            "PolicyArn": "arn:aws:iam::111111111111:policy/policy1"
        }
    ]
}To now create a different user with a different policy all we have to do is edit thepolicy.jsonfile and then rerun the trace.
We can do a dry-run first by specifying--dryrunto see if the appropiate relations between the traces has been properly derived.
This will look something like this:$ awstrace-play --trace-file create-user.trace -p user-name tu -p policy-name tp policy-document file://policy.json --dryrun
(play) aws iam create-user --user-name tu
(play) aws iam create-policy --policy-document file://policy.json --policy-name tp
(play) aws iam attach-user-policy --policy-arn arn:aws:iam::111111111111:policy/tp --user-name tu
(play) aws iam list-attached-user-policies --user-name tuIf we now run it without the--dryrunoption we will ultimately see the output of the last request which was the call tolist-attached-user-policies.$ awstrace-play --trace-file create-user.trace -p user-name tu -p policy-name tp -p policy-document file://policy.json
[...]
(play) aws iam list-attached-user-policies --user-name tu
{
    "AttachedPolicies": [
        {
            "PolicyName": "tp",
            "PolicyArn": "arn:aws:iam::111111111111:policy/tp"
        }
    ]
}
[...]Bugs, comments, suggestionsShoot in a pull-request via github, post an issue in the issue tracker or simply shoot an email togvb@anvilventures.com.