AWS Project using SHELL SCRIPTING for DevOps
Tags: aws, shell, devops, kcdchennai
Published: 2023-05-31 10:00:31

## Introduction

In the real-world DevOps scenario, The **AWS Resource Tracker** script is widely used to provide an overview of the AWS resources being utilised within an environment.

It aims to help organisations monitor and manage their AWS resources effectively. The script utilises the AWS Command Line Interface (**CLI**) to fetch information about different AWS services, such as **S3 buckets, EC2 instances, Lambda functions, and IAM users**.

## What does this project do?

By running the AWS Resource Tracker script, users can quickly obtain a list of S3 buckets, EC2 instances, Lambda functions, and IAM users associated with their AWS account.

This information can be valuable for various purposes, including auditing, inventory management, resource optimisation, and security assessment.

## Build the project

### Create EC2 Instance

**Step 1**. Go to your **AWS** account and log in, then search for **EC2** Instances in the search bar. Or you can click on the services button that is on the top left corner of your dashboard, from there also you can search for the same.


![Imageaws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/45bp44kldk9i1yhnl8dk.png)

Then click on the **Launch Instance** button.

**Step 2**. Give it a **name** of your choice, **ubuntu** as a machine image, select your **key-pair** or create one if not have any. Leave the instance type **t2.micro** ie free tier as the same and click again on the **Launch Instance** button.

![Image aws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/resa6qttug4lqtl7l48r.png)

Now you can see your **instance** up and **running** like this.

![Image aws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/12bnduj2ggqlebdjn8lg.png)

**Step 3**. Now click on the instance **id** which will give you detailed information about the running instance, there copy the public **IP address**.

![Image aws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/750rec8gy31u2876xc5a.png)

Run the Instance

**Step 4**. Now open up your terminal and run the below command.

`ssh -i /Users/poonampawar/Downloads/my-key-pair.pem ubuntu@ip_add`

We have to go inside the directory where you have downloaded your key-pair and in my case, it is in `/Downloads` folder.

Check yours and give a path accordingly. And don't forget to replace `ip_add` with your copied one. This will take you to the virtual machine which we have created in AWS.

![Image aws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3t67paiubn1xyvyxafm5.png)

### Create the Script

**Step 5**. Now create a shell script file named `aws_resource_tracker.sh` and copy and paste the below script.

```
#########################
# Author: Your Name
# Date: 28/05/23
# Version: v1
#
# This script will report the AWS resource usage
########################

set -x

# AWS S3
# AWS EC2
# AWS Lambda
# AWS IAM Users

# list s3 buckets
echo "Print list of s3 buckets"
aws s3 ls

# list EC2 Instances
echo "Print list of ec2 instances"
aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId'

# list lambda
echo "Print list of lambda functions"
aws lambda list-functions

# list IAM Users
echo "Print list of iam users"
aws iam list-users
```

![Image aws](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b1q4n733hq6o8ybgsxc4.png)

- It is a good convention to always provide your details so that it will make it easier for other developers to contribute if they want to ie, metadata.

- The command `set -x` is given here in order to debug the script, this will print the command which we are running and then prints the output.

- The commands `aws s3 ls`, `aws lambda list-functions` and `aws iam list-users` print the information of the given statement.

The command

`aws ec2 describe instances | jq '.Reservations[].Instances[].InstanceId'` will give you all the Instance IDs present in your `aws` in `JSON` format.

### Test the script

**Step 6**. Run the below command to see the output.

`./aws_resource_tracker.sh`

The output will look like this.

![Image o/p](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sk0ke7hl253msylpx9m1.png)

### Apply CronJob

To use cron jobs with your script, you can schedule it to run at specific intervals using the cron syntax. Here's an example of how you can modify your script to use cron jobs:

- Open a terminal and run the following command to edit the 
  crontab file:
  `crontab -e`

- If prompted, choose your preferred text editor.
   
![Image o/p](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f6328ifon40392btyjxb.png)

- Add a new line to the crontab file to schedule your 
  script. For example, to run the script every day at 9 AM, 
  you can add the following line:
`0 9 * * * /path/to/your/script.sh`

  Modify the path `/path/to/your/script.sh` to the actual 
  path where your script is located.
  
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4vc06fc6kjsfxxo6tv52.png)

- Save the crontab file and exit the text editor.

- The above cron expression (0 9 * * *) represents the schedule: minute (0), hour (9), any day of the month, any month, and any day of the week. You can customize the schedule based on your requirements using the cron syntax.

- By adding this line to the crontab file, your script will be executed automatically according to the defined schedule. The output of the script will be sent to your email address by default, or you can redirect the output to a file if needed.

- Make sure that the script is executable `(chmod +x /path/to/your/script.sh)` and that the necessary environment variables and AWS CLI configurations are set up correctly for the script to run successfully within the cron environment.

## Use Cases

- **Monitoring and auditing**: By running this script periodically using cron jobs, you can monitor and audit your AWS resources. It provides insights into the status and details of different resources, such as S3 buckets, EC2 instances, Lambda functions, and IAM users.

- **Resource inventory**: The script helps in maintaining an up-to-date inventory of your AWS resources. It lists the S3 buckets, EC2 instances, Lambda functions, and IAM users, allowing you to have a clear understanding of what resources exist in your environment.

- **Troubleshooting**: In case of any issues or incidents, this script can be used to quickly gather information about the relevant AWS resources. For example, if there is an issue with an EC2 instance, you can run the script to get the instance ID and other details for further investigation.

- **Automation and reporting**: The script can be integrated into an automated pipeline or workflow to generate regular reports about AWS resource usage. This information can be valuable for tracking costs, resource utilization, and compliance requirements.

- **Scalability and efficiency**: In larger environments with numerous AWS resources, manually retrieving information about each resource can be time-consuming and error-prone. By using this script, you can automate the process and retrieve resource details in a consistent and efficient manner.

Overall, this script simplifies the process of gathering information about AWS resources, enhances visibility into your infrastructure, and supports effective management and monitoring of your DevOps environment.

_Github link: https://github.com/Poonam1607/shell-scripting-projects_

> Resource: For better understanding and visual learning you can check out this tutorial - https://youtu.be/gx5E47R9fGk

---

This project is purely based on my learnings. It may occur error while performing in your setup. If you find any issue with it, feel free to reach out to me.

Thank you🖤!