AWS Service Quotas: Discovering Where you Stand
Tags: aws, intelligentautomatio, otherposts, servicehealth
Published: 2023-04-19 20:17:48

---
title: AWS Service Quotas: Discovering Where you Stand
published: true
date: 2023-02-17 19:31:01 UTC
tags: AWS,IntelligentAutomatio,OtherPosts,ServiceHealth
canonical_url: https://unskript.com/aws-service-quotas-discovering-where-you-stand/
---

We’ve written a few posts in the last week about AWS Service Quotas.  These are restrictions on services that are set by AWS (but can often be increased).

If our [first post, we looked at New Actions in unSkript](https://unskript.com/aws-service-quotas-what-are-they-and-how-can-i-increase-them/) that can be used to determine quota values and request a quota increase.  In this post, we’ll take the Actions a step further, and build an Action that compares the AWS quota to actual usage – generating an alert when a threshold is met.

 

## Getting Started

To begin, we will need to consider how the Action will work.  For any given service, we’ll need to query AWS at least twice:

1. Get the Quota Limit.
2. Determine the usage of a service.

Every query requires one call to complete Step 1.  However, Step two can require many queries to complete the usage query.  In the simplest case, we can do just one query:

> Example: _Client VPN Endpoints per Region. _ If we query AWS for the list of endpoints in a region, we can simply get the length of the response to know how many endpoints exist.

However, there are times where there will be multiple queries:

> Example: _Routes per Client VPN Endpoint_. In the first query, we get the list of VPN endpoints.  In step 2, we must query every VPN endpoint to get the count of Routes.  If there are 4 VPN endpoints, there will be a total of 5 calls made (On call to get the list of 4 VPN endpoints, and then one call to each of the four endpoints).

To account for these two options, we create an input Dictionary.

## The Simple, one pass Dictionary

For the _Describe AMIs_ call (only one Usage query is required), the Dict looks like this:

```
{'QuotaName':'AMIs','ServiceCode':'ec2','QuotaCode': 'L-B665C33B',
'ApiName': 'describe_images', 'ApiFilter' : '[]','ApiParam': 'Images',
'initialQuery': ''},

```

To get the Quota, we need the ServiceCode and the QuotaCode (If you need to obtain these variables, you can use the unSkript Action, or you can refer to the table in the [unSkript Docs)](https://docs.unskript.com/unskript-product-documentation/lists/test).  The one usage API call will be made to the _describe\_images_ endpoint, and retrieve a list of _Images_.  Counting this length gives us our usage.

## The Two Pass Dictionary

 

To determine the _Attachments per transit gateway_, we must again get the quota from the Service Code and Quota Code.  To get the count of attachments per transit gateway, we us the _initalQuery_ array to make a first query.

The first query probes the _describe\_transit\_gateways_ endpoint, to give a list of _TransitGateways. _ In the second set of calls, we call the _describe\_transit\_gateway\_attachments_ endpoint for each transit gateway. The filter has a string VARIABLE that is replaced with the _TransitGatewayId_ for each gateway -ensuring that each call is made to a different transit gateway.  We can then count the length of the response to find out how many attachments are in each transit gateway.  If we have 12 transit gateways. we will have 12 usage reports.

```
{'QuotaName':'Attachments per transit gateway','ServiceCode':'ec2','QuotaCode': 'L-E0233F82',
'ApiName': 'describe_transit_gateway_attachments', 'ApiFilter' : '[{"Name": "transit-gateway-id","Values": ["VARIABLE"]}]',
'ApiParam': 'TransitGatewayAttachments',
'initialQuery': '["describe_transit_gateways","TransitGateways", "TransitGatewayId"]'},

```

## Outliers

For most of our quota measurements, these two approaches work well.  However, with over 2600 different quotas inside AWS, not all of them fit neatly into these two buckets. For example _Multicast Network Interfaces per transit gateway_ requires 3 calls: Transit gateways -> Multicast Domains – > Domain attachments.

For others, there is custom code to iterate over.  These require an extra if statement in the code to properly account for their usage.

## Action Format

We can differentiate between the two types of query by looking at the ‘initialQuery’ parameter. If it is empty, we can do the Simple query, otherwise, do the double query (with a for loop that queries each initial result).  For outliers, we can add specific code inside the if/else:

(this is simplified a bit from what actually runs):

```
for i in table: 
    #get quota 
    sq = sqClient.get_service_quota(ServiceCode=i.get('ServiceCode'),QuotaCode=i.get('QuotaCode')) 
    quotaValue =sq['Quota']['Value'] 

    #get usage     
    if i.get('initialQuery') = '': 
       res = aws_get_paginator(ec2Client, i.get('ApiName'), i.get('ApiParam'), Filters=filterList) 
       count = len(res) 
       percentage = count/quotaValue 
       combinedData = {'Quota Name': i.get('QuotaName'), 'Limit':quotaValue, 'used': count, 'percentage':percentage} 
       result.append( combinedData) 
       print(combinedData) else: res = aws_get_paginator(ec2Client, i.get('ApiName'), i.get('ApiParam'), Filters=filterList) 
       for j in res: #build the filter query with some simple substitutions res2 = aws_get_paginator(ec2Client, i.get('ApiName'), i.get('ApiParam'), Filters=filterList) count = len(res2) percentage = count/quotaValue objectResult = {j[initialQueryFilter] : count} quotaName = f"{i.get('QuotaName')} for {j[initialQueryFilter]}" combinedData = {'Quota Name': quotaName, 'Limit':quotaValue, 'used': count, 'percentage':percentage} result.append(combinedData)

```

 

## Action Output

Once all of the values have been collected, the percentage utilized is compared to the warning percentage input. If the utilization is over the requested percentage, the Service data will be added to the output of the Action. With this information, the SRE responsible can decide the correct Action to take – either prune away some usage, or request an increase from AWS.

For example, testing all VPC Service quotas with a earning of 50% utilization gives the following data:

```
{'Instances': [{'Limit': 20.0,
                'Quota Name': 'VPCs Per Region',
                'percentage': 0.65,
                'used': 13},
               {'Limit': 20.0,
                'Quota Name': 'Internet gateways per Region',
                'percentage': 0.6,
                'used': 12},
               {'Limit': 5.0,
                'Quota Name': 'NAT gateways per Availability Zone',
                'percentage': 0.8,
                'used': 4},
               {'Limit': 50.0,
                'Quota Name': 'Routes per route table',
                'percentage': 0.5,
                'used': 25},
               {'Limit': 20.0,
                'Quota Name': 'Rules per network ACL',
                'percentage': 0.65,
                'used': 13}]}
```

## Availability Today

As we publish this article, we have 2 Actions heading into unSkript:

1. A general _AWS\_ServiceQuota Compare_ Action that has the basic framework described above. This will likely require customization for each Quota you wish to test against.
2. _AWS VPC Service Quota Warning_. This Action takes all of the VPC service quotas (as of February 2023) and tests them against your infrastructure.

Coming Soon:

1. _AWS EC2 Service Quota Warning_. This Action will test your infrastructure against all EC2 Service Quotas, and warn you if you are approaching the quota threshold.

WE’re really excited to see how people use these Service Quota alerts in their infrastructure.  If you have questions – feel free to reach out in our [Slack Community](https://communityinviter.com/apps/cloud-ops-community/awesome-cloud-automation).  If you haven’t tried unSkript – try our [OSS Docker Container](https://github.com/unskript/Awesome-CloudOps-Automation), or use our [free trial online!](https://us.app.unskript.io/)