【AWS】EC2でCloudWatchのカスタムメトリクス備忘録


カスタムメトリクスを使用すると、標準メトリクスでは監視できないパラメーターをロギングすることができる。カスタムメトリクスの導入には、統合 CloudWatch エージェントをインストールする必要がある。今回は、その方法をEC2を用いて紹介する。

https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html

## EC2へカスタムメトリクス導入手順

1. IAMロールを作成して、アタッチする
インスタンスにアタッチされた IAM ロールに CloudWatchAgentServerPolicyが必要

1. IAM コンソールを開き、ロール -> ロールの作成を押す
1. 信頼されたエンティティタイプは、「AWS のサービス」を選択し
1. EC2を選択し、次へを押す
1. 許可ポリシー で、CloudWatchAgentServerPolicy, AmazonSSMManagedInstanceCore, CloudWatchAgentAdminPolicy, AmazonSSMManagedEC2InstanceDefaultPolicyにチェックを入れる
    ![カスタムメトリクス1.PNG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/483556/17ee588a-9939-af70-569c-3c26896839f6.png)
    ![カスタムメトリクス2.PNG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/483556/e0b896bf-93f5-01e8-2516-f786d1230bc5.png)
1. 次へを押す
1. ロール名をCloudWatchAgentServerRoleで作成する
1. EC2に入り、CloudWatch エージェントをインストールする
    ```
    sudo yum install amazon-cloudwatch-agent
    ```
1. CloudWatch エージェント設定ファイル(JSON)を作成する
    ```
    sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
    ```
    上記コマンドを実行すると、下記のように20個ほど質問をされるので、回答する。
    ```
    [ec2-user@ip-*********]$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
    ================================================================
    = Welcome to the Amazon CloudWatch Agent Configuration Manager =
    =                                                              =
    = CloudWatch Agent allows you to collect metrics and logs from =
    = your host and send them to CloudWatch. Additional CloudWatch =
    = charges may apply.                                           =
    ================================================================
    On which OS are you planning to use the agent?
    1. linux
    2. windows
    3. darwin
    default choice: [1]:
    
    Trying to fetch the default region based on ec2 metadata...
    2024/01/02 01:54:17 I! imds retry client will retry 1 times
    Are you using EC2 or On-Premises hosts?
    1. EC2
    2. On-Premises
    default choice: [1]:
    
    Which user are you planning to run the agent?
    1. root
    2. cwagent
    3. others
    default choice: [1]:
    
    Do you want to turn on StatsD daemon?
    1. yes
    2. no
    default choice: [1]:
    
    Which port do you want StatsD daemon to listen to?
    default choice: [8125]
    
    What is the collect interval for StatsD daemon?
    1. 10s
    2. 30s
    3. 60s
    default choice: [1]:
    
    What is the aggregation interval for metrics collected by StatsD daemon?
    1. Do not aggregate
    2. 10s
    3. 30s
    4. 60s
    default choice: [4]:
    
    Do you want to monitor metrics from CollectD? WARNING: CollectD must be installed or the Agent will fail to start
    1. yes
    2. no
    default choice: [1]:
    
    Do you want to monitor any host metrics? e.g. CPU, memory, etc.
    1. yes
    2. no
    default choice: [1]:
    
    Do you want to monitor cpu metrics per core?
    1. yes
    2. no
    default choice: [1]:
    
    Do you want to add ec2 dimensions (ImageId, InstanceId, InstanceType, AutoScalingGroupName) into all of your metrics if the info is available?
    1. yes
    2. no
    default choice: [1]:
    
    Do you want to aggregate ec2 dimensions (InstanceId)?
    1. yes
    2. no
    default choice: [1]:
    
    Would you like to collect your metrics at high resolution (sub-minute resolution)? This enables sub-minute resolution for all metrics, but you can customize for specific metrics in the output 
    json file.
    1. 1s
    2. 10s
    3. 30s
    4. 60s
    default choice: [4]:
    
    Which default metrics config do you want?
    1. Basic
    2. Standard
    3. Advanced
    4. None
    default choice: [1]:
    
    Current config as follows:
    {
            "agent": {
                    "metrics_collection_interval": 60,
                    "run_as_user": "root"
            },
            "metrics": {
                    "aggregation_dimensions": [
                            [
                                    "InstanceId"
                            ]
                    ],
                    "append_dimensions": {
                            "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                            "ImageId": "${aws:ImageId}",
                            "InstanceId": "${aws:InstanceId}",
                            "InstanceType": "${aws:InstanceType}"
                    },
                    "metrics_collected": {
                            "collectd": {
                                    "metrics_aggregation_interval": 60
                            },
                            "disk": {
                                    "measurement": [
                                            "used_percent"
                                    ],
                                    "metrics_collection_interval": 60,
                                    "resources": [
                                            "*"
                                    ]
                            },
                            "mem": {
                                    "measurement": [
                                            "mem_used_percent"
                                    ],
                                    "metrics_collection_interval": 60
                            },
                            "statsd": {
                                    "metrics_aggregation_interval": 60,
                                    "metrics_collection_interval": 10,
                                    "service_address": ":8125"
                            }
                    }
            }
    }
    Are you satisfied with the above config? Note: it can be manually customized after the wizard completes to add additional items.
    1. yes
    2. no
    default choice: [1]:
    
    Do you have any existing CloudWatch Log Agent (http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html) configuration file to import for migration?
    1. yes
    2. no
    default choice: [2]:
    
    Do you want to monitor any log files?
    1. yes
    2. no
    default choice: [1]:
    2
    Do you want the CloudWatch agent to also retrieve X-ray traces?
    1. yes
    2. no
    default choice: [1]:
    2
    Existing config JSON identified and copied to:  /opt/aws/amazon-cloudwatch-agent/etc/backup-configs
    Saved config file to /opt/aws/amazon-cloudwatch-agent/bin/config.json successfully.
    Current config as follows:
    {
            "agent": {
                    "metrics_collection_interval": 60,
                    "run_as_user": "root"
            },
            "metrics": {
                    "aggregation_dimensions": [
                            [
                                    "InstanceId"
                            ]
                    ],
                    "append_dimensions": {
                            "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                            "ImageId": "${aws:ImageId}",
                            "InstanceId": "${aws:InstanceId}",
                            "InstanceType": "${aws:InstanceType}"
                    },
                    "metrics_collected": {
                            "collectd": {
                                    "metrics_aggregation_interval": 60
                            },
                            "disk": {
                                    "measurement": [
                                            "used_percent"
                                    ],
                                    "metrics_collection_interval": 60,
                                    "resources": [
                                            "*"
                                    ]
                            },
                            "mem": {
                                    "measurement": [
                                            "mem_used_percent"
                                    ],
                                    "metrics_collection_interval": 60
                            },
                            "statsd": {
                                    "metrics_aggregation_interval": 60,
                                    "metrics_collection_interval": 10,
                                    "service_address": ":8125"
                            }
                    }
            }
    }
    Please check the above content of the config.
    The config file is also located at /opt/aws/amazon-cloudwatch-agent/bin/config.json.
    Edit it manually if needed.
    Do you want to store the config in the SSM parameter store?
    1. yes
    2. no
    default choice: [1]:
    
    What parameter store name do you want to use to store your config? (Use 'AmazonCloudWatch-' prefix if you use our managed AWS policy)
    default choice: [AmazonCloudWatch-linux]
    
    Trying to fetch the default region based on ec2 metadata...
    2024/01/02 01:55:48 I! imds retry client will retry 1 times
    Which region do you want to store the config in the parameter store?
    default choice: [ap-northeast-1]
    
    Which AWS credential should be used to send json config to parameter store?
    1. ASIA4NJHGOZAMJNGFPIS(From SDK)
    2. Other
    default choice: [1]:
    
    Successfully put config to parameter store AmazonCloudWatch-linux.
    Program exits now.
    ```
    途中でメトリクスの詳細レベルを問われるが、その違いは下記である。
    ![カスタムメトリクス3.PNG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/483556/e2fa7268-d7b1-d721-2576-88c7ff8e05a8.png)
1. collectdをインストールする
    ```
    sudo amazon-linux-extras install collectd
    ```
1. CloudWatch Agentを起動する
    ```
    sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name
    ```
ここでconfiguration-parameter-store-nameは、質問で設定した値に変換する
例）AmazonCloudWatch-linux

これで、カスタムメトリクスがCloudWatchメトリクスに表示される
1. CloudWatchコンソール -> すべてのメトリクスを開く
1. CWAgentを押すと設定したカスタムメトリクス一覧が表示される
1. 左のチェックボックスにチェックを入れると、上部にグラフが表示される
    ![カスタムメトリクス4.PNG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/483556/63382f47-f2f9-73d4-e603-02d6efa0c10b.png)

CloudWatchで収集可能なメトリクスは下記リンク参照

https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html

## 参考文献
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html#create-iam-roles-for-cloudwatch-agent-roles