AWS Budget Killswitch: Disable AWS Services When Budget Is Exceeded
Tags: aws
Published: 2020-07-06 03:47:39

I like creating small projects and hosting them on AWS. AWS is what I use at work and using it for my personal projects makes sense because it's convenient and it's what I know. 
However, my personal projects don't make me any money so a huge increase in traffic doesn't mean an increase in revenue to cover the cost of an AWS bill. 
Sometimes I just want to put something out there and see if people like it. Call me paranoid, but I'm afraid of a situation when I wake up one day and discover that I have a [$2700 bill](https://mobile.twitter.com/ChrisShort/status/1279406322837082114) in my AWS account for something that doesn't make any money. 

I've always wanted to have a hard limit on AWS spending on my personal account. Something like: "Stop everything when my bill reaches $100 a month". There is no such functionality on AWS and I appreciate how hard would it be to implement it. There would not be a solution which works for everyone. Some people would only want to stop services which are the main culprits, some would like to stop everything. Would you want to have your s3 buckets emptied if you have a sudden spike in your bill? Also, there is a matter of production services. Very few companies would be ok with terminating their infrastructure when the bill goes above a certain budget. 
For people who just like to tinker and are ok with their services being down in an emergency such solution might be useful. This is where "AWS Budget Killswitch" comes in.

## AWS Budget Killswitch
The solution is very simple:
1. Create an emergency budget, a certain value which would constitute an unacceptable spending
2. When budget is reached send an email and disable public-facing AWS services, currently only CloudFront and ApiGateway are supported

The app is implemented as a [SAM](https://aws.amazon.com/serverless/sam/) template:  
```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: ''

Parameters:
  NotificationEmail:
    Type: String
  BudgetLimit:
    Type: Number

Resources:
  KillswitchBudgetTopic:
    Type: AWS::SNS::Topic
    Properties:
      Subscription:
        - Endpoint: !GetAtt DisableServicesFunction.Arn
          Protocol: "lambda"

  KillswitchBudget:
    Type: AWS::Budgets::Budget
    Properties: 
      Budget:
        BudgetLimit:
          Amount: !Ref BudgetLimit
          Unit: USD
        TimeUnit: MONTHLY
        BudgetType: COST
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 99
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref NotificationEmail
            - SubscriptionType: SNS
              Address: !Ref KillswitchBudgetTopic

  SnsInvokeLambdaPermission:            
    Type: AWS::Lambda::Permission
    Properties: 
      Action: lambda:InvokeFunction
      FunctionName: !Ref DisableServicesFunction
      Principal: 'sns.amazonaws.com'
      SourceArn: !Ref KillswitchBudgetTopic              

  DisableServicesFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: disable-services
      Handler: app.handler
      Runtime: nodejs12.x
      MemorySize: 256  
      Policies:
      - AWSLambdaExecute
      - Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Action:
            - cloudfront:ListDistributions
            - cloudfront:GetDistributionConfig
            - cloudfront:UpdateDistribution
            - 'apigateway:*'
            Resource: '*'  
```

It creates a budget with 2 subscriptions, email, and an SNS topic.  
SNS topic is configured to invoke a Lambda which contains the logic to disable services. AWS doesn't have a big red "Disable" button for each service, each one is unique, so disabling them is different per service.  
You can disable a CloudFront distribution, but ApiGateway stage can only be deleted. Luckily you can set throttling limits to 0, so it can also be made unreachable.  

Here is how the Lambda looks like:  
```javascript
const AWS = require('aws-sdk')

const cloudfront = new AWS.CloudFront()
const apigateway = new AWS.APIGateway()

const disableDistribution = async(id) => {
  const config = await cloudfront.getDistributionConfig({ Id: id }).promise()
  const disabledConfig = { ...config.DistributionConfig, Enabled: false }

  await cloudfront.updateDistribution({
    Id: id,
    DistributionConfig: disabledConfig,
    IfMatch: config.ETag
  }).promise()
}

const disableCloudfront = async() => {
  const distributions = await cloudfront.listDistributions().promise()
  await Promise.all(distributions.DistributionList.Items.map(d => disableDistribution(d.Id)))
}

const setStageLimits = async(restApiId, stageName) => {
  const stage = await apigateway.getStage({
    restApiId,
    stageName,
  }).promise()

  await apigateway.updateStage({
    restApiId,
    stageName,
    patchOperations: [{
        op: 'replace',
        path: '/*/*/throttling/burstLimit',
        value: '0'
      },
      {
        op: 'replace',
        path: '/*/*/throttling/rateLimit',
        value: '0'
      },
    ]
  }).promise()

  console.log(stage)
}

const disableApiGateways = async() => {
  const apiIds = (await apigateway.getRestApis().promise()).items.map(a => a.id)

  await Promise.all(apiIds.map(async(restApiId) => {
    const stages = (await apigateway.getStages({
      restApiId
    }).promise()).item

    await Promise.all(stages.map(stage => setStageLimits(restApiId, stage.stageName)))
  }))

}

exports.handler = async(event) => {
  await disableCloudfront()
  await disableApiGateways()
};
```

There is nothing else to it, it's a very blunt instrument, but it will make sure that if you have a huge spike in traffic that you don't expect you'll end up with your app down instead of paying an exorbitant bill.  
This is definitely something only for tinkerers like me, not for a production use.  
The proper solution would be using an [AWS WAF](https://aws.amazon.com/waf/) but I'm too cheap to spend $6 a month for a very unlickely event of a DDoS attack on one of my small projects.  

[Here](https://github.com/Leonti/aws-budget-killswitch) is a Github repository ready to be deployed.