Challenge: Display EC2 instances from all regions using Python and Boto3
Tags: aws, boto3, ec2, lambda
Published: 2022-12-18 01:38:37

This was an idea from the AWS Certified Global Community to create challenges by breaking a certain part of code, so the users can learn by fixing the code.

I have written this second challenge for the AWS Certified Global Community and have also used it to mentor some of my mentees.
I thought that it might be a good idea to share this with the broader community as well.

**Boto3** is the name of the Python SDK for AWS. It allows you to directly create, update, and delete AWS resources from your Python scripts.

I agree that we have **EC2 Global View**, which shows what we are trying to achieve. However this is more of a programming challenge on how to code using Python and Boto3.

Let's start the challenge!

**Pre-Requisites:**

* Lambda with Python 3.9
* Lambda Execution role with access to EC2 and CloudWatch access (IAM)
* It is preferable to have at least one EC2 instance in any of the regions (not mandatory though)

----

**Step 1. Create an IAM Policy:**

You could either let Lambda create an execution role and add the **AmazonEC2ReadOnlyAccess** policy or you could create the following IAM policy and a role.

> Note: Replace [Region], [LambdaFunctionName] and [AccountID] in the Policy

IAM Policy with the following access:

**AWSLambdaBasicExecution**

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:[Region]:[AccountID]:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
```

**AmazonEC2ReadOnlyAccess**

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:ListMetrics",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:Describe*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:Describe*",
            "Resource": "*"
        }
    ]
}
```

----

**Step 2. Create a role for Lambda and attach the policy created in Step 1**

```json
Trusted entities:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
```

----

**Step 3. Broken Code:** Create a Lambda function with the following code and execute the function.

> Note: Please increase the Lambda Function **Timeout** under **General Configuration** from **3 sec** to **2 min** or more as required.

```python
import json

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:
    
        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']
    
        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()
        
        for reservation in response["Instances"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }	
```

----

> Note: **DO NOT** forget to deploy the Lambda function every time you make a change.

**First Error:** We get the following error when we execute the code:question:

```json
Response
{
  "errorMessage": "name 'boto3' is not defined",
  "errorType": "NameError",
  "requestId": "6926f216-e0e8-4f25-bb16-761ada972e3f",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 8, in lambda_handler\n    ec2 = boto3.client('ec2')\n"
  ]
}
```

**Hint 1:**
The first one is easy to solve by executing the code in Lambda, the hint appears in the error message.

-----

**Second Error:** We get the following error after fixing the first one:question:

```json
Response
{
  "errorMessage": "'Instances'",
  "errorType": "KeyError",
  "requestId": "c78a6211-1d73-4d72-b83a-4c51bfd4e181",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 21, in lambda_handler\n    for reservation in response[\"Instances\"]:\n"
  ]
}
```

**Hint 2:**

Please refer to the Response Syntax of describe_instances(**kwargs) at [Boto3 Documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html?highlight=describe_instances#EC2.Client.describe_instances)

----

**Step 4. Final Code:** :white_check_mark:

```python
import json
import boto3

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:
    
        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']
    
        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()
        
        for reservation in response["Reservations"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }	
```

**Output**
```json
{
  "statusCode": 200,
  "body": [
    "I-[instanceid]"
  ]
}

START RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Version: $LATEST
* Checking region  --   eu-north-1
* Checking region  --   ap-south-1
* Checking region  --   eu-west-3
* Checking region  --   eu-west-2
* Checking region  --   eu-west-1
* Checking region  --   ap-northeast-3
* Checking region  --   ap-northeast-2
* Checking region  --   ap-northeast-1
* Checking region  --   sa-east-1
* Checking region  --   ca-central-1
* Checking region  --   ap-southeast-1
* Checking region  --   ap-southeast-2
* Checking region  --   eu-central-1
* Checking region  --   us-east-1
---- Instance [instanceid] in us-east-1
* Checking region  --   us-east-2
* Checking region  --   us-west-1
* Checking region  --   us-west-2
END RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd
REPORT RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Duration: 15667.85 ms Billed Duration: 15668 ms Memory Size: 128 MB Max Memory Used: 90 MB Init Duration: 246.67 ms
```
> Note: The actual instance id is replaced with [instanceid] in the output.

So what have I done to break the code

- I removed `import boto3` at `line no 2`
- Replaced `response["Reservations"]:` with `response["Instances"]:` at `line no 21`

Hope you liked the challenge! Please let me know your thoughts in the comments section.

> Tip: Please do check out the reference link as you will learn something about global services too with Boto3. :point_down:

See you next time :wave:

-----

### References: 
I learnt this from stackoverflow when I asked a question about [Retrieve list of buckets from all regions using boto3](https://stackoverflow.com/questions/72837445/retrieve-list-of-buckets-from-all-regions-using-boto3)

Hope you have also learnt something about retrieving S3 buckets from all regions using Boto3. 

-----