Package: skew
skewSkewis a package for identifying and enumerating cloud resources.
The name is a homonym for SKU (Stock Keeping Unit).  Skew allows you to
define different SKUschemeswhich are a particular encoding of a
SKU.  Skew then allows you to use this scheme pattern and regular expressions
based on the scheme pattern to identify and enumerate a resource or set
of resources.At the moment, the the only availableschemeis theARNscheme.
TheARNscheme uses the basic structure ofAmazon Resource Names(ARNs) to assign a unique identifier to every AWS
resource.An example ARN pattern would be:arn:aws:ec2:us-west-2:123456789012:instance/i-12345678This pattern identifies a specific EC2 instance running in theus-west-2region under the account ID123456789012.  The account ID is the 12-digit
unique identifier for a specific AWS account as describedhere.
To allowskewto find your account number, you need to create askewYAML config file.  By default,skewwill look for your config file in~/.skewbut you can use theSKEW_CONFIGenvironment variable to tellskewwhere to find your config file if you choose to put it somewhere else.  The
basic format of theskewconfig file is:---accounts:"123456789012":profile:dev"234567890123":profile:prodWithin theaccountssection, you create keys named after your 12-digit
account ID (as a string).  Within that, you must have an entry calledprofilethat lists the profile name this account maps to within your AWS credential
file.The main purpose of skew is to identify resources or sets of resources
across services, regions, and accounts and to quickly and easily return the
data associated with those resources. For example, if you wanted to return
the data associated with the example ARN above:fromskewimportscanarn=scan('arn:aws:ec2:us-west-2:123456789012:instance/i-12345678')forresourceinarn:print(resource.data)The call toscanreturns an ARN object which implements theiterator patternand returns aResourceobject for each AWS resource that matches the
ARN pattern provided.  TheResourceobject contains all of the data
associated with the AWS resource in dictionary under thedataattribute.Any of the elements of the ARN can be replaced with a regular expression.
The simplest regular expression is*which means all available choices.
So, for example:arn=scan('arn:aws:ec2:us-east-1:*:instance/*')would return an iterator for all EC2 instances in theus-east-1region
found in all accounts defined in the config file.To find all DynamoDB tables in all US regions for the account ID 234567890123
you would use:arn=scan('arn:aws:dynamodb:us-.*:234567890123:table/*')CloudWatch MetricsIn addition to making the metadata about a particular AWS resource available
to you,skewalso tries to make it easy to access the available CloudWatch
metrics for a given resource.For example, assume that you had did ascanon the original ARN above
and had the resource associated with that instance available as the variableinstance.  You could do the following:>>>instance.metric_names['CPUUtilization','NetworkOut','StatusCheckFailed','StatusCheckFailed_System','NetworkIn','DiskWriteOps','DiskReadBytes','DiskReadOps','StatusCheckFailed_Instance','DiskWriteBytes']>>>Themetric_namesattribute returns the list of available CloudWatch metrics
for this resource.  The retrieve the metric data for one of these:>>>instance.get_metric_data('CPUUtilization')[{'Average':0.134,'Timestamp':'2014-09-29T14:04:00Z','Unit':'Percent'},{'Average':0.066,'Timestamp':'2014-09-29T13:54:00Z','Unit':'Percent'},{'Average':0.066,'Timestamp':'2014-09-29T14:09:00Z','Unit':'Percent'},{'Average':0.134,'Timestamp':'2014-09-29T13:34:00Z','Unit':'Percent'},{'Average':0.066,'Timestamp':'2014-09-29T14:19:00Z','Unit':'Percent'},{'Average':0.068,'Timestamp':'2014-09-29T13:44:00Z','Unit':'Percent'},{'Average':0.134,'Timestamp':'2014-09-29T14:14:00Z','Unit':'Percent'},{'Average':0.066,'Timestamp':'2014-09-29T13:29:00Z','Unit':'Percent'},{'Average':0.132,'Timestamp':'2014-09-29T13:59:00Z','Unit':'Percent'},{'Average':0.134,'Timestamp':'2014-09-29T13:49:00Z','Unit':'Percent'},{'Average':0.134,'Timestamp':'2014-09-29T13:39:00Z','Unit':'Percent'}]>>>You can also customize the data returned rather than using the default settings:>>>instance.get_metric_data('CPUUtilization',hours=8,statistics=['Average','Minimum','Maximum'])[{'Average':0.132,'Maximum':0.33,'Minimum':0.0,'Timestamp':'2014-09-29T10:54:00Z','Unit':'Percent'},{'Average':0.134,'Maximum':0.34,'Minimum':0.0,'Timestamp':'2014-09-29T14:04:00Z','Unit':'Percent'},...,{'Average':0.066,'Maximum':0.33,'Minimum':0.0,'Timestamp':'2014-09-29T08:34:00Z','Unit':'Percent'},{'Average':0.134,'Maximum':0.34,'Minimum':0.0,'Timestamp':'2014-09-29T08:04:00Z','Unit':'Percent'}]>>>Filtering DataEach resource that is retrieved is a Python dictionary.  Some of these (e.g.
an EC2 Instance) can be quite large and complex.  Skew allows you to filter
the data returned by applying ajmespathquery to
the resulting data.  If you aren't familiar with jmespath, check it out.
Its a very powerful query language for JSON data and has full support in
Python as well as a number of other languages such as Ruby, PHP, and
Javascript.  It is also the query language used in theAWSCLIso if you are familiar with the--queryoption there, you can use the same thing with skew.To specify a query to be applied to results of a scan, simply append
the query to the end of the ARN, separated by a|(pipe) character.
For example:arn:aws:ec2:us-west-2:123456789012:instance/i-12345678|InstanceTypeWould retrieve the data for this particular EC2 instance and would then
filter the returned data through the (very) simple jmespath query to which
retrieves the value of the attributeInstanceTypewithin the data.  The
filtered data is available as thefiltered_dataattribute of the
Resource object.  The full, unfiltered data is still available as thedataattribute.Multithreaded UsageSkew is single-threaded by default, like most Python libraries. In order to
speed up the enumeration of matching resources, you can use multiple threads:importskewclassWorker(Thread):def__init__(self,arn):Thread.__init__(self)self.arn=arnself.name=arndefrun(self):foriinskew.scan(self.arn):# now do something with the stuffarn=skew.ARN()forserviceinarn.service.choices():uri='arn:aws:'+service+':*:*:*/*'worker=Worker(uri);worker.start()(thanks to @alFReD-NSH for the snippet)More ExamplesFind Unattached VolumesAudit Security GroupsFind Untagged Instances