CloudCycle - Set lifecycle for your cloud resources to avoid surprising costs
Tags: aws, go, serverless, lambda
Published: 2024-05-25 14:48:41

## The Problem

I have recently forgot to Turn off / Terminate two ec2 instances running **m5x.large** and **t3.medium** that runs about 1 month and I just received a notification of my AWS monthly bill and shocked me! 

At the first place I thought my account was hacked, But turns out that I just left two AWS EC2 Instances running. 🥲😔

## The Solution (CloudCycle)

So, that's why I came up with a basic solution and share it. So now, whether I forgot to turn it off / terminate a running AWS Resources, I don't have to worry about it anymore, if I have properly set the desired lifecycle of my cloud resources.

I have created a lambda function which get's executed every **15 minutes** to check if the supported resources are due to termination or not. 

Now, if you or your team forgot to terminate cloud resources, the **CloudCycle** will do the job for you.

## Tools & Language used

1. **Golang** - Since lambda function bills will be based on the duration of execution. At first, I developed this by using python and I have realized that I need to consider it's performance and execution time.

2. **Terraform** - Used only for deployment and examples.

## Architecture


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7g5em2k86yenw694t8sk.png)

This setup will utilized the schedule expression of event bridge, In which event bridge get's executed every 15 minutes. In doing so, lambda function get validate if the supported resources is valid for termination.

But why termination? Instead of turning it off?

Of course there's a free tool available called **Cloud-Custodian**. But still, sometimes turning it off isn't enough and it can lead to lots of unused resources and can still occur minimal cost.

## Sample Code for EC2 Instance Service

Below are the sample codes for EC2 Instance service.

```
package services

import (
	"context"
	"github.com/aws/aws-sdk-go-v2/service/ec2"
	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/service/ec2/types"
	"github.com/redopsbay/cloudcycle/internal"
	"github.com/redopsbay/cloudcycle/internal/schedule"
	"fmt"
)

type EC2 struct {
	InstanceId         string
	CloudCycle         string
	MarkForTermination bool
}

type EC2Instances struct {
	Instances []EC2
}

func GetEC2Instances(ctx context.Context, client *ec2.Client) ([]types.Reservation, error) {
	filters := []types.Filter{{
		Name:   aws.String("tag-key"),
		Values: []string{internal.TagKey},
	},
	}

	DescribeInputs := ec2.DescribeInstancesInput{
		Filters: filters,
	}

	instances, err := client.DescribeInstances(ctx, &DescribeInputs)

	if err != nil {
		panic(err)
		return []types.Reservation{}, err
	}

	return instances.Reservations, nil
}

func MarkInstancesForTermination(reservations []types.Reservation) (EC2Instances, error) {
	var instances EC2Instances

	for _, reservation := range reservations {
		for _, instance := range reservation.Instances {
			for _, tag := range instance.Tags {
				if *tag.Key == internal.TagKey {
					Lifecycle, err := schedule.GetLifeCycle(instance.LaunchTime, *tag.Value)
					if err != nil {
						return EC2Instances{}, err
					}

					if schedule.ValidForTermination(Lifecycle) {
						ec2instance := EC2{
							InstanceId:         *instance.InstanceId,
							CloudCycle:         *tag.Value,
							MarkForTermination: true,
						}
						instances.Instances = append(instances.Instances, ec2instance)

					} else {
						ec2instance := EC2{
							InstanceId:         *instance.InstanceId,
							CloudCycle:         *tag.Value,
							MarkForTermination: false,
						}
						instances.Instances = append(instances.Instances, ec2instance)
					}
				}
			}
		}
	}
	return instances, nil
}

func StartEC2InstanceTermination(ctx context.Context, client *ec2.Client) error {
	var instanceIds []string

	reservations, err := GetEC2Instances(ctx, client)
	if err != nil {
		fmt.Println("Unable to get instances.")
		return err
	}

	instances, err := MarkInstancesForTermination(reservations)
	if err != nil {
		fmt.Println("Unable to mark instances for termination.")
		return err
	}

	for _, instance := range instances.Instances {
		if instance.MarkForTermination {
			instanceIds = append(instanceIds, instance.InstanceId)
			fmt.Printf("\nInstanceID: %s, ForTermination: %t, CloudCycle: %s\n",
				instance.InstanceId,
				instance.MarkForTermination,
				instance.CloudCycle)
		}
	}

	TerminatedOutput, err := client.TerminateInstances(ctx, &ec2.TerminateInstancesInput{
		InstanceIds: instanceIds,
	})

	for _, state := range TerminatedOutput.TerminatingInstances {
		if *state.CurrentState.Code == 0 {
			fmt.Printf("InstanceID: %s, State: Pending for Termination", *state.InstanceId)
		} else if *state.CurrentState.Code == 32 {
			fmt.Printf("InstanceID: %s, State: Shutting down", *state.InstanceId)
		} else if *state.CurrentState.Code == 48 {
			fmt.Printf("InstanceID: %s, State: Shutting down", *state.InstanceId)
		} else if *state.CurrentState.Code == 16 {
			fmt.Printf("InstanceID: %s, State: Still running", *state.InstanceId)
		} else if *state.CurrentState.Code == 64 {
			fmt.Printf("InstanceID: %s, State: Stopping", *state.InstanceId)
		} else if *state.CurrentState.Code == 80 {
			fmt.Printf("InstanceID: %s, State: Stopped", *state.InstanceId)
		} else {
			fmt.Printf("InstanceID: %s, State: Unknown", *state.InstanceId)
		}
	}

	return nil

}
```

## Objective

My objective with **CloudCycle** is to automatically cleanup supported resources based on the specified **duration** or **lifecycle** through resource tagging, And to support the commonly used resources that causes AWS Bills to grow even though the cloud resources is not needed anymore.

No more story telling! 

For complete documentation and project link, you can proceed directly to my GitHub repo below.

[https://github.com/redopsbay/cloudcycle](https://github.com/redopsbay/cloudcycle)

This repo is open for contributors!!! Some documents and cloud resources are currently **WORK-IN-PROGRESS**

## Usage

Just specify that tag and set your desired lifecycle for supported resources with **CloudCycle** Key. Below are the supported duration.

| Suffixes | Detail  | Sample Value |
| -------- | ------- | ------------ |
| `m`      | Minutes | `60m`        |
| `h`      | Hours   | `2h`         |
| `d`      | Days    | `7d`         |

## How does it works?

CloudCycle will get all the supported resources with a tagged key ***CloudCycle*** and it will simply compare the ***current time*** vs ***launch time*** of the supported resources with the specified `key/value` pair resource tag if the supported resources are valid for termination.


Below are the sample terraform code.


Specify ec2 lifecycle by 24 hours from it's launch date.

```hcl
data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"

  tags = {
    CloudCycle = "1d" // This ec2 instance will be terminated within 24 hours from it's launch date.
  }
}
```

## Sample Terraform Deployment Usage

For deployment, you can refer to the github repo [Deployment Page](https://github.com/redopsbay/cloudcycle/blob/master/deploy/README.md)

## Benefit's

Sometimes it's better to let go than stay strong. Leaving up unused resources will incur costs.

And lastly, No nightmare's, No poverty! 😂🤣

## Reference

- [https://github.com/redopsbay/cloudcycle](https://github.com/redopsbay/cloudcycle)