Scheduling EC2 start and stop using Eventbridge and Lambda (part 1 of 2)
Tags: aws, serverless, lambda, cloud
Published: 2023-09-13 23:17:48

## Introduction:

By now we understand very well that:
**Cloud == Pay for what you use**
One of the biggest factor that attracts users to the cloud, is the ability to provision resources within minutes. You are able to setup a virtual server in the Cloud, with just a few clicks, in a matter of minutes.

## Use Case:
Your organization's development team works from 08:00AM to 05:00PM Monday to Friday. The development team's Cloud Virtual Machines (Amazon EC2 instances) are running idle in their out of office hours. It is your responsibility to ensure that the dev environment stops at 05:30PM everyday, and resumes the next morning. The environment should remain down on the weekends. Automate this task.

## Services :

#### Amazon [Eventbridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-what-is.html) (formerly called Amazon CloudWatch Events):
EventBridge is a serverless service that uses events to connect application components together, making it easier for you to build scalable event-driven applications. Event-driven architecture is a style of building loosely-coupled software systems that work together by emitting and responding to events. Event-driven architecture can help you boost agility and build reliable, scalable applications.


#### AWS [Lambda](https://aws.amazon.com/lambda/):
AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications, and only pay for what you use.

#### AWS [IAM](https://aws.amazon.com/iam/):
AWS Identity and Access Management (IAM), allows us to specify who or what can access services and resources in AWS, centrally manage fine-grained permissions, and analyze access to refine permissions across AWS.

#### Amazon [EC2](https://aws.amazon.com/pm/ec2/?trk=32f4fbd0-ffda-4695-a60c-8857fab7d0dd&sc_channel=ps&ef_id=CjwKCAjwu4WoBhBkEiwAojNdXp4MetAwaCyPq-D5I0LQt7u2WyBh9WPlC4jKRDKTLbvdUnpUvdinSRoCm50QAvD_BwE:G:s&s_kwcid=AL!4422!3!476942909971!e!!g!!amazon%20ec2!11539707735!118057053088):
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.

## Goal: 

1. Setup a scheduler using the Amazon Eventbridge service which will trigger a Lambda function.

2. Lambda function will temporarily _assume_ the defined IAM role. This IAM role will have an IAM policy associated with it. The IAM Policy is what grants the Role permissions to access our EC2 instances. 

3. The Lambda function will stop/start our Amazon EC2 instances as per the schedule we define in Eventbridge.

![arch_diagram](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rhfp1aixqlna5u0dx1dr.png)

This is part-1 of the blog, where we will :
i.   Create the IAM Policy and associate it with an IAM Role.
ii.  Create our Lambda functions: one to stop the EC2 instances and another to start the EC2 instance.
iii. Test the Lambda functions by triggering them manually, to check if they have the desired effect on our EC2 instances  

Part-2 of the blog will tie everything together, where we will be configuring the Amazon Eventbridge rules and defining the Cron entry that will trigger our Lambda functions to stop/start our EC2 instances.


## Pre-requisites :
1. AWS IAM account (do not use root account) having:
- admin privileges
- access to AWS Management Console
2. Understanding of AWS Identity based policies
In case you are not familiar with IAM Policies or need a quick refresher, please refer my [blog](https://medium.com/@aggarwal.tanushree/breaking-down-your-cloud-identity-and-access-249b506116d5)
3. Atleast one EC2 instance created in the account (this is required for testing purpose)


## Cost :
- None (if you have an AWS free-tier eligible account)

## Implementation:

Let's begin! Login to the AWS Management Console as IAM admin user

## 1. Create IAM policy and IAM role
1.1	From the AWS Management Console navigate to **IAM**
 
![iam_dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rtinf6nft82x36xbfxj3.png)

1.2 From the left navigation panel, select  `Policies` and then click the `Create Policy` button

![create_policy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/68ubhnabnauyz3nk9rop.png)

1.3	Select `JSON`
 
![json](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xy43j55yf91opmai40sh.png)

1.4	Paste the below policy into the policy editor (delete all the pre-filled content) and click `Next`

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*",
      "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "eu-central-1"
                }
            }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "*",
      "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "eu-central-1"
                }
            }
    }
  ]
}
```

1.5	Give the policy a meaningful name and description. Review the permissions and then click `Click Policy`
 
![policy_name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m9kb0w5p07nr0tfax4ld.png)
![policy_save_1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s7ka63b0dif19qoqnwj6.png)
![policy_save_2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l6vvuxtx4ah3a7mxjvej.png) 

1.6	From the left navigation panel, select `Roles` and then click the `Create Role` button
 
![iam_role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zwj9028qojyr4mut711b.png)

1.7	Select `AWS Service` as the `Trusted entity type`

1.8	From the `Service or use case` dropdown list, select `Lambda`

1.9	Click `Next` to move to the next screen

![trusted_entity](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8ze21fz72yxtjfzw8i4n.png)

1.10	From the policy dropdown list, select the policy we created in the previous step and click `Next`
 
![add_permission](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ki62lxw2oww7vhfiktzc.png)

1.11	Give the role a name and description
 
![role_name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f0ens541xxrfjbpmwgqb.png)

1.12	Verify the trusted entity (should be Lambda) and the associated customer managed IAM Policy. Click `Create Role`

![create_role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8362y2eotw68rf6fhi2g.png)

![role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p9pz6q75eu8fx366a5oe.png)

> Note the maximum duration this role can be assumed. Since we did not specify a duration, the default duration of **1 hour** was assigned

![duration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntk9v42hn15od5snzrzq.png)


 


## 2. Create Lambda Function to stop EC2 instances
2.1	From the AWS Management Console navigate to **Lambda**
 
![lambda](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rede6al50rqa88xh446t.png)

_Make sure you are in the desired AWS region. Change the region in case not._

2.2	Click `Create Function`
 
![create_function](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xub9828onyufc7jgyrhf.png)

2.3	Select `Author from scratch`
 
![author](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0xvcrurvnn0yj0isf99j.png)

2.4	Give the function a `name` and select the desired `runtime`.

![function_name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ltgcusguzbfvsyq6dfc3.png)

> In this demo we will be using `Python 3.11` as the runtime.

 
2.5	In the `Permissions` tab, change the _default execution role_ to `Use an existing role`.
From the dropdown list, select the lambda assume role we created in the previous step.
 
![lambda_role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ddc579c30978j9nwg5zh.png)


![role_save](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9iw1hlkzsua5utr2ch0o.png)

2.6	Click `Create Function`
 
![created_lambda](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pz2aewelqv616r23euzi.png)


Good job keeping up so far! You are making good progress!


2.7	Scroll down a bit to reach the `Code` section. You will notice a pre-filled `Code source` section.
 
![lambda_code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hgd4d8ycfsnc5l3cqnl3.png)

2.8	Delete the pre-filled contents and paste the below code in its place:

```py
import boto3

region = 'eu-central-1'


def lambda_handler(event, context):
    ec2_resource = boto3.resource("ec2", region_name=region)
    
    # Apply filter to check for EC2 instances in 'running' state
    instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])

    print("EC2 region is: ", region)
    print("instances= ", instances)
    
    # Count running instances
    count = len(list(instances))
    print(f"{count} instances running")
    
    # Stopping EC2 instances        
    for instance in instances:
        instance.stop()
        print(instance, ": Stopping!!! .")
        
```

2.9	Click `Deploy`
 
![deploy_lambda](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dextctkeziri478qcjon.png)

2.10	In the `Configuration` tab, select `General configuration` and click `Edit`

![configure_timeout](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8bb2yzks3hsrjd09kg1f.png)

2.11	Change the `Timeout` value to `10 seconds` and click `Save`

![timeout_valur](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9qfnjo4lfjm90z1g8td6.png)

_Note: The timeout value may differ for each use case, so choose one according to your requirement_

2.12	Click `Test` to test if our lambda function is working as expected
Configure the test event. 

_Just an `event name` will suffice. Our Lambda does not require any user input._

![configure_test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4oha4840eu5aver5yk4e.png)

Click `Save`
 
![save_test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hudzfxrmo0uwdqu0gozx.png)


Click `Test` to execute the newly created Lambda function.
Review the output logged by the stop EC2 Lambda.

**Important: For this to work you need to have atleast one EC2 instance running in eu-central-1 region**

![output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ueyewt6yvepmu51racz6.png)

- Two EC2 instances were found to be in `running` state in the region defined within our Lambda function `eu-central-1`
- Our Lambda attempts stopping each of these one-by-one (as coded in our `for` loop)

> Notice the time duration of our Lambda execution - it is roughly _4 seconds_. If we left the timeout value as default _3 seconds_, our Lambda function would most likely have timed-out before completing the execution.


2.13	Go to the EC2 dashboard in the AWS Management Console (in the same region defined in your Lambda code) and check if the instances really stopped.

It worked!
 

![ec2_dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ia0qu9xggun6ayfhmn06.png)



## 3. Create the Start EC2 instance Lambda Function
3.1 Repeat steps to create another Lambda function and this time use the below Python code, which will start our EC2 instances. 
 
 
![start_lambda](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tzsouv9nxsfwsbookgcy.png)

![lambda_exec](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w05dtv7t2mno0bchxdwd.png)

![lambda_saved_1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lpxqwpyvoudfg36jfhcg.png)
 


```py
import boto3

region = 'eu-central-1'


def lambda_handler(event, context):
    ec2_resource = boto3.resource("ec2", region_name=region)
    
    # Apply filter to check for EC2 instances in 'stopped' state
    instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['stopped']}])

    print("EC2 region is: ", region)
    print("instances= ", instances)
    
    # Count running instances
    count = len(list(instances))
    print(f"{count} instances in stopped state")
    
    # Starting EC2 instances        
    for instance in instances:
        instance.start()
        print(instance, ": Starting!!! .")
        
    
```

 
![lambda_2_deployed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rdqx7ret580yi28bvbu6.png)

3.2 Test the Lambda
 
![lambda_2_op](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oc5q83s6rmacmccqqysq.png)

Review the execution result of the Lambda function.
- 2 EC2 instances were found in `running` state in the defined region `eu-central-1`
- Both are stopped one after the other, with their respective EC2 instance IDs logged.
- The execution took ~4 seconds.

3.3 Verify if the EC2 instances started in the EC2 dashboard

![ec2_2_verify](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/osw8m3hhtze4j4yeu9bm.png)


 
Pheewww!! 
That was a lot of hand's on work! 
I hope this gave you good exposure on how to navigate the AWS Management Console and helped grow your understanding on how multiple services can be used together to build a simple solution ! 

## Conclusion: 
In this blog we discussed our use case and associated architecture. We created an IAM execution role which allows our Lambda functions to access our EC2 instances. We even created the EC2 stop and EC2 start Lambda functions and tested them in isolation.

> Stay tuned for the part-2 of this blog, where we complete our architecture by configuring Eventbridge to trigger our Lambdas.


## Bonus:
Now that you have understood how to create the Lambda functions, feel free to play around with these and modify as per your desired use case. In our demo we restricted the EC2 task to a single region, this can easily be extended to include multiple AWS regions.
You can even enhance the instance filters on the available EC2 instance states!
Additionally, you make the code more dynamic, by filtering the EC2 instances based on  defined `Tags`! 

**Filter based on instance state and defined tags:**
```py
 instances = ec2_resource.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']},{'Name': 'tag:environment','Values':['qa']}])
```

**The valid values EC2 instance states are:**

- pending
- running
- shutting-down
- terminated
- stopping
- stopped

**EC2 instance life cycle:**
 
![EC2-lifecycle](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rffcsw3zom6r7122mpjs.png)





## References:

1. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-lifecycle.html

2. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2/client/describe_instance_status.html
 
3. https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-run-lambda-schedule.html

4. https://www.youtube.com/watch?v=UibU8g503G0&t=1229s