Monitoring Amazon EC2 with CloudWatch and SNS Notifications.
Published: 2023-08-27 03:39:27


Monitoring Amazon Elastic Compute Cloud (Amazon EC2) instances is crucial for maintaining the health, performance, and availability of your applications and infrastructure. Amazon CloudWatch, a monitoring and management service, provides a comprehensive suite of tools for monitoring various AWS resources, including EC2 instances. Additionally, you can use Amazon Simple Notification Service (SNS) to receive notifications based on CloudWatch alarms, enabling you to promptly respond to any issues or anomalies. This integration helps you ensure that your EC2 instances are operating efficiently and that any potential problems are addressed in a timely manner.

**Pre-Requisites:**

- AWS Account: You must have an active AWS account. If you don't have one, you can sign up for an AWS account on the AWS website by providing the necessary information.

**Launch an EC2 Instance and install necessary packages**

We will initiate the launch of an EC2 instance employing the Amazon Linux 2023 AMI. This instance will utilize the t3a.small instance type, and the default settings will remain unchanged. However, feel free to modify these settings according to your needs. To demonstrate the process, I've configured SSH Access to be permitted from MyIP. Nevertheless, I strongly advise you to adopt a more precise access approach, such as using MyIP, to enhance security in practical scenarios.

In the userdata section, you can include the following commands to install the AWS CloudWatch Agent and set up the configuration file:

```
#!/bin/bash -xe

echo --- install packages ---
dnf update && dnf install -y amazon-cloudwatch-agent-1.247358.0-1.amzn2023.x86_64 \
    gcc \
    ec2-instance-connect \
    aws-cfn-bootstrap.noarch \
    openssh-8.7p1-8.amzn2023.0.4.x86_64 \
    rsyslog-8.2204.0-3.amzn2023.0.2.x86_64

echo --- create cw agent config file ---
cat << EOF > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
{
  "agent": {
    "run_as_user": "root"
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/secure",
            "log_group_name": "SSHunsuccessfulattempt",
            "log_stream_name": "{instance_id}",
            "retention_in_days": 3,
            "timestamp_format": "%b %d %H:%M:%S"
          }
        ]
      }
    }
  }
}
EOF

echo --- starting the cloudwatch agent ---
systemctl start amazon-cloudwatch-agent.service

echo --- modify sshd to log to file ---
systemctl stop sshd
sed -i 's|RestartSec=42s|RestartSec=42s\nStandardOutput=syslog\nStandardError=syslog\n|g' /lib/systemd/system/sshd.service
systemctl daemon-reload
systemctl start sshd

echo --- start syslog ---
systemctl start rsyslog

/opt/aws/bin/cfn-signal -e 0 --stack "ec2monitoring" --region "us-west-2" --resource MonitorCloudWatchLabInstance


```
**Instructions**

_To confirm that the Amazon CloudWatch agent is running, you can use the following command:_

```
sudo systemctl status amazon-cloudwatch-agent.service
```
Running this command in your terminal will provide information about the current status of the CloudWatch agent service, including whether it's active and running, any recent logs, and more. This will help you verify that the CloudWatch agent has been successfully started and is operational on your EC2 instance. 

_To view the configuration file of the Amazon CloudWatch agent, you can use the more command or any text viewer of your choice. The configuration file is typically located at /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json. Here's the command to view the configuration file using more:_

```
more /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
```
Running this command in your terminal will display the contents of the CloudWatch agent's configuration file, allowing you to inspect the settings and configurations that you've defined for the agent's behavior.

_To view log files, in the terminal session, enter the following._

```
cd /var/log
ls
```
_Open up a real-time display of the secure log file:_

```
sudo tail -f secure
```

_In the AWS Management Console search bar, enter cloudwatch, and click the CloudWatch result under Services:_

_Create SNS TOPIC_

_Create Subscription_

_Email Confirmation_


**Certainly, here's a concise recap of the progress you've made:**

_EC2 Instance:_

Configured and launched an EC2 instance with CloudWatch log agent running as a service.

_CloudWatch Logging:_

Created a CloudWatch Log Group and Log Stream to capture logs from the EC2 instance's "secure" log file.

_SNS Notifications:_

Established an SNS topic and subscription, ready to receive push notifications triggered by CloudWatch alarms.

**Cont'**

Navigate back to CloudWatch, and click Alarms > All alarms:

_Create Alarm._

In the search bar in the Metrics section, enter Incoming log events and press enter:
 Select Account Metrics -> IncomingLogEvents

_Now click on Graphed metrics, and select the following:_

	1 Minute as Period
	Sum as Statistic.

_In the Conditions section, enter and select:_

	Whenever IncomingLogEvents is...: Select Greater/Equal
	Than: Enter 2

_In the Notification section, click in the Send a notification to... box and select your ssh-fails topic:_
	The alarm is created and is now listed on the Alarms page. The valid States for an alarm are:

	In alarm - The alarm was triggered
	OK - The Alarm was not triggered, status is normal
	Insufficient data - Not enough data exists to either set 
        an Alarm or set the status to OK.

_From the Log groups page of CloudWatch, notice the Metric Filters column has no value. _

Although the Alarm is created, you have not set up a Metric Filter yet. You need to do that next so you can match against a specific pattern within the logs sent from your EC2 instance to CloudWatch.

_Select the checkbox for the SSHfail Log Group, then click Actions > Create Metric Filter:_

 [Mon, day, timestamp, ip, id, status = Invalid, ...]

_Enter the following:_ 

	Create filter name:
	Filter Name: InvalidSSHUsers
	Metric details:
	Metric name: Enter ssh-fails
	Metric namespace: Enter ssh-fails
	Metric value: Enter 2
_At the bottom of the page, click Next, and then click Create metric filter:_

_Attempt to SSH into your running EC2 instance again as the invalid user "Daniel", and then attempt several failures in a row._

**Tip:** Once you see the failed notification on the EC2 Instance Connect page, you can refresh the browser page several times to rack up several failed logins.

Aim for five or six attempts. Anything over two will raise an alarm.

After a minute or two, your alarm should be raised. Once raised, if not violated again, it will settle and reset on its own. That is, the alarm should transition to an OK state within a few minutes of no violations:

In conclusion, you've successfully orchestrated a monitoring and notification framework for your Amazon EC2 instance using AWS CloudWatch and Simple Notification Service (SNS). By configuring and launching an EC2 instance with the CloudWatch log agent, you've ensured the seamless collection and transmission of logs and metrics. The setup of a CloudWatch Log Group and Log Stream guarantees the efficient management and analysis of collected logs. Furthermore, the creation of an SNS topic, along with a subscription, positions you to receive prompt notifications via push notifications whenever CloudWatch alarms are triggered.

This integrated solution empowers you to proactively monitor the health and performance of your EC2 instance, centralize log management, and stay informed about critical events. Through these steps, you've taken significant strides in fortifying your AWS environment's reliability, security, and operational efficiency. As you continue to refine and adapt this framework, you'll be better equipped to ensure the uninterrupted operation of your applications and services.