[JAWS-UG CLI] CloudWatch:#3 DynamoDBのアラーム設定 (ReadCapacity)

https://jawsug-cli.doorkeeper.jp/events/20523 でのハンズオン資料です。

AWS CLIを利用して、DynamoDBテーブルの「プロビジョニングされたスループット」の使用量が閾値に達したときに通知する設定を行ってみます。

参考: http://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/DeveloperGuide/dynamo-metricscollected.html

マネジメントコンソールでのテーブル作成時、Thrughput Alarmsの設定で80%を指定した場合のサンプルになります。

>  一定期間の期間内(period * evaluation-periods = この設定では60分後)に
> データのputやgetが無いと、INSUFFICIENT_DATAになる(OKにならなくなる)ようです。


前提条件
========

## CloudWatchへの権限

CloudWatchに対して以下の権限があること。
(今回のハンズオンでは使わない権限もあります。)

```json:
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Action": [
              "cloudwatch:DescribeAlarmHistory",
              "cloudwatch:DescribeAlarms",
              "cloudwatch:DescribeAlarmsForMetric",
              "cloudwatch:GetMetricStatistics",
              "cloudwatch:ListMetrics",
              "cloudwatch:PutMetricAlarm",
              "cloudwatch:SetAlarmState",
            ],
            "Effect": "Allow",
            "Resource": "*"
          }
        ]
      }
```

## AWS CLIのバージョン

以下のバージョンで動作確認済

* AWS CLI 1.7.18
* AWS CLI 1.7.8

```bash:コマンド:
aws --version
```

```text:結果(例):
aws-cli/1.7.18 Python/2.7.5 Darwin/13.4.0
```
0. 準備
=======


0.1. リージョンの決定
---------------------

作成するユーザのデフォルトリージョンを決めます。
(カレントユーザが利用するカレントリージョンも切り変わります。)

```bash:コマンド(東京リージョンの場合):
export AWS_DEFAULT_REGION='ap-northeast-1'
```


0.2. 変数の確認
---------------

プロファイルとリージョンが想定のものになっていることを確認します。

```bash:変数の確認:
aws configure list
```

```text:結果(例):
            Name                    Value             Type    Location
            ----                    -----             ----    --------
         profile      iam_full-prjZ-mbp13              env    AWS_DEFAULT_PROFILE
      access_key     ****************LOAQ shared-credentials-file
      secret_key     ****************I1O1 shared-credentials-file
          region           ap-northeast-1              env    AWS_DEFAULT_REGION
```


1. 事前作業
===========


1.1. 変数の確認
------------------------

```bash:変数の確認:
cat << ETX

DYNAMO_TABLE_NAME:  ${DYNAMO_TABLE_NAME}
SNS_TOPIC_ARN:      ${SNS_TOPIC_ARN}

ETX
```

1.2. DynamoDBテーブル名の指定
------------------------

DYNAMO_TABLE_NAMEが空か、想定と異なる場合は指定しなおします。

```bash:コマンド(例):
DYNAMO_TABLE_NAME='Thread'
```



1.3. 通知先SNSの決定
--------------------

SNS_TOPIC_NAMEが空か、想定と異なる場合は指定しなおします。

* 新規にSNSトピックを作成、購読する場合は、別途作成作業をしておくこと 。
  * http://qiita.com/tcsh/items/e05e21e7338668cbf3b8
  * http://qiita.com/tcsh/items/ca81576b0018bfa32cbe

```bash:コマンド:
SNS_TOPIC_NAME='dynamodb'
```

* SNSトピックのARN取得

```bash:コマンド:
SNS_TOPIC_ARN=`aws sns list-topics | jq -r .Topics[].TopicArn | grep ${SNS_TOPIC_NAME}` \
  && echo ${SNS_TOPIC_ARN}
```

1.4. 変数の確認 (再)
------------------------

```bash:変数の確認:
cat << ETX

DYNAMO_TABLE_NAME:  ${DYNAMO_TABLE_NAME}
SNS_TOPIC_ARN:      ${SNS_TOPIC_ARN}

ETX
```


2. メトリックの確認 (テーブルに入出力があった場合)
===================

> Note: テーブル作成後、一覧表示されるまで15分ほど必要になるようです。


2.1. 名前空間、ディメンションの指定
-----------------------------------

```bash:コマンド:
CWATCH_NAMESPACE='AWS/DynamoDB'
CWATCH_DIMENSIONS="Name=TableName,Value=${DYNAMO_TABLE_NAME}"
```

2.2. 一覧の取得
---------------

入力値の確認をします。

```bash:変数確認:
cat << ETX

      CWATCH_NAMESPACE:  ${CWATCH_NAMESPACE}
      CWATCH_DIMENSIONS: ${CWATCH_DIMENSIONS}

ETX
```

```bash:コマンド:
aws cloudwatch list-metrics \
        --namespace ${CWATCH_NAMESPACE} \
        --dimensions ${CWATCH_DIMENSIONS}
```

```json:結果(例):
      {
        "Metrics": [
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "ProvisionedReadCapacityUnits"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "ProvisionedWriteCapacityUnits"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "ConsumedReadCapacityUnits"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "ConsumedWriteCapacityUnits"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "Operation",
                      "Value": "PutItem"
                  },
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "SuccessfulRequestLatency"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "Operation",
                      "Value": "Scan"
                  },
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "SuccessfulRequestLatency"
          },
          {
              "Namespace": "AWS/DynamoDB",
              "Dimensions": [
                  {
                      "Name": "Operation",
                      "Value": "Scan"
                  },
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "MetricName": "ReturnedItemCount"
          }
        ]
      }
```



3. CloudWatchアラームの設定 (ReadCapacity)
===========


3.1. スループットの指定
-----------------------

```bash:コマンド(例):
DYNAMO_READ_UNIT='10'
```



3.2. アラーム名(その概要)の指定
-------------------------------

```bash:コマンド:
CWATCH_ALARM_NAME="${DYNAMO_TABLE_NAME}-ReadCapacityUnitsLimit-BasicAlarm"
CWATCH_ALARM_DESC="Read Capacity Units Limit Basic Alarm for ${DYNAMO_TABLE_NAME}"
```


3.3. メトリックの指定
---------------------

名前空間:
   AWS/DynamoDB

ディメンション:
   テーブル名

メトリック名:
   ConsumedWriteCapacityUnits

メトリック単位:
   Count

統計方法:
   Sum

```bash:コマンド:
CWATCH_NAMESPACE='AWS/DynamoDB'
CWATCH_DIMENSIONS="Name=TableName,Value=${DYNAMO_TABLE_NAME}"
CWATCH_METRIC_NAME='ConsumedReadCapacityUnits'
CWATCH_METRIC_UNIT='Count'
CWATCH_STATISTICS='Sum'
```


3.4. アラーム設定値の指定
-------------------------

閾値:
   80%

監視周期:
   300秒

評価期間:
   12周期 (300秒x12回=1時間)

比較方法:
   閾値以上

```bash:コマンド:
DYNAMO_READ_RATE='80'
CWATCH_ALARM_PERIOD='300'
CWATCH_EVALUATION_PERIOD='12'
CWATCH_COMPARISON='GreaterThanOrEqualToThreshold'
```


3.5. アラーム閾値の算出
-----------------------

```bash:コマンド:
CWATCH_THRESHOLD=`echo "scale=1; ${CWATCH_ALARM_PERIOD} * ${DYNAMO_READ_UNIT} * ${DYNAMO_READ_RATE}/100;" | bc`;
      echo ${CWATCH_THRESHOLD}
```

3.6. アラーム通知先
-------------------

```bash:コマンド:
CWATCH_ALARM_TOPIC=${SNS_TOPIC_ARN}
```


3.7. アラームの作成
-------------------

```bash:変数の確認:

cat << ETX

        CWATCH_NAMESPACE:         ${CWATCH_NAMESPACE}
        CWATCH_METRIC_NAME:       ${CWATCH_METRIC_NAME}
        CWATCH_METRIC_UNIT:       ${CWATCH_METRIC_UNIT}
        CWATCH_STATISTICS:        ${CWATCH_STATISTICS}
        CWATCH_DIMENSIONS:        ${CWATCH_DIMENSIONS}
        CWATCH_ALARM_PERIOD:      ${CWATCH_ALARM_PERIOD}
        CWATCH_EVALUATION_PERIOD: ${CWATCH_EVALUATION_PERIOD}
        CWATCH_THRESHOLD:         ${CWATCH_THRESHOLD}
        CWATCH_COMPARISON:        ${CWATCH_COMPARISON}
        CWATCH_ALARM_NAME:        ${CWATCH_ALARM_NAME}
        CWATCH_ALARM_DESC:       "${CWATCH_ALARM_DESC}"
        CWATCH_ALARM_TOPIC:       ${CWATCH_ALARM_TOPIC}

ETX
```

```bash:command:
aws cloudwatch put-metric-alarm \
        --namespace           ${CWATCH_NAMESPACE} \
        --metric-name         ${CWATCH_METRIC_NAME} \
        --unit                ${CWATCH_METRIC_UNIT} \
        --statistic           ${CWATCH_STATISTICS} \
        --dimensions          ${CWATCH_DIMENSIONS} \
        --period              ${CWATCH_ALARM_PERIOD} \
        --evaluation-periods  ${CWATCH_EVALUATION_PERIOD} \
        --threshold           ${CWATCH_THRESHOLD} \
        --comparison-operator ${CWATCH_COMPARISON} \
        --alarm-name          ${CWATCH_ALARM_NAME} \
        --alarm-description  "${CWATCH_ALARM_DESC}" \
        --alarm-actions       ${CWATCH_ALARM_TOPIC}
```

```text:result:
      (戻り値なし)
```


3.8. アラームの確認
-------------------

```bash:コマンド:
aws cloudwatch describe-alarms --alarm-names ${CWATCH_ALARM_NAME}
```

```json:結果(例):
      {
        "MetricAlarms": [
          {
              "EvaluationPeriods": 12,
              "AlarmArn": "arn:aws:cloudwatch:us-west-2:XXXXXXXXXXXX:alarm:ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "StateUpdatedTimestamp": "2015-02-15T07:24:06.461Z",
              "AlarmConfigurationUpdatedTimestamp": "2015-02-15T07:23:28.223Z",
              "ComparisonOperator": "GreaterThanOrEqualToThreshold",
              "AlarmActions": [
                  "arn:aws:sns:us-west-2:XXXXXXXXXXXX:dynamodb"
              ],
              "Namespace": "AWS/DynamoDB",
              "AlarmDescription": "Read Capacity Units Limit Basic Alarm for ProductCatalog",
              "StateReasonData": "{\"version\":\"1.0\",\"queryDate\":\"2015-02-15T07:24:06.455+0000\",\"unit\":\"Count\",\"statistic\":\"Sum\",\"period\":300,\"recentDatapoints\":[],\"threshold\":2400.0}",
              "Period": 300,
              "StateValue": "INSUFFICIENT_DATA",
              "Threshold": 2400.0,
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "Dimensions": [
                  {
                      "Name": "TableName",
                      "Value": "ProductCatalog"
                  }
              ],
              "Unit": "Count",
              "Statistic": "Sum",
              "StateReason": "Unchecked: Initial alarm creation",
              "InsufficientDataActions": [],
              "OKActions": [],
              "ActionsEnabled": true,
              "MetricName": "ConsumedReadCapacityUnits"
          }
        ]
      }
```


3.9. historyの確認
------------------

```bash:コマンド:
CWATCH_MAX_ITEMS='2'
```

```bash:コマンド:
aws cloudwatch describe-alarm-history \
        --alarm-name ${CWATCH_ALARM_NAME} \
        --max-items ${CWATCH_MAX_ITEMS}
```

```json:結果(例):
      {
        "AlarmHistoryItems": [
          {
              "Timestamp": "2015-02-14T06:37:59.453Z",
              "HistoryItemType": "StateUpdate",
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "HistoryData": "{\"version\":\"1.0\",\"oldState\":{\"stateValue\":\"INSUFFICIENT_DATA\",\"stateReason\":\"Unchecked: Initial alarm creation\"},\"newState\":{\"stateValue\":\"OK\",\"stateReason\":\"Threshold Crossed: 1 datapoint (8.0) was not greater than or equal to the threshold (240.0).\",\"stateReasonData\":{\"version\":\"1.0\",\"queryDate\":\"2015-02-14T06:37:59.447+0000\",\"startDate\":\"2015-02-14T06:22:00.000+0000\",\"unit\":\"Count\",\"statistic\":\"Sum\",\"period\":300,\"recentDatapoints\":[8.0],\"threshold\":240.0}}}",
              "HistorySummary": "Alarm updated from INSUFFICIENT_DATA to OK"

          },
          {
              "Timestamp": "2015-02-14T06:37:58.808Z",
              "HistoryItemType": "ConfigurationUpdate",
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "HistoryData": "{\"version\":\"1.0\",\"type\":\"Create\",\"createdAlarm\":{\"threshold\":240.0,\"stateValue\":\"INSUFFICIENT_DATA\",\"metricName\":\"ConsumedReadCapacityUnits\",\"period\":300,\"dimensions\":[{\"name\":\"TableName\",\"value\":\"ProductCatalog\"}],\"alarmName\":\"ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\",\"unit\":\"Count\",\"actionsEnabled\":true,\"okactions\":[],\"alarmActions\":[\"arn:aws:sns:us-west-2:XXXXXXXXXXXX:dynamodb\"],\"namespace\":\"AWS/DynamoDB\",\"evaluationPeriods\":12,\"comparisonOperator\":\"GreaterThanOrEqualToThreshold\",\"statistic\":\"Sum\",\"alarmArn\":\"arn:aws:cloudwatch:us-west-2:XXXXXXXXXXXX:alarm:ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\",\"insufficientDataActions\":[],\"alarmDescription\":\"Read Capacity Units Limit Basic Alarm for ProductCatalog",\"alarmConfigurationUpdatedTimestamp\":\"2015-02-14T06:37:58.808+0000\",\"stateUpdatedTimestamp\":\"2015-02-14T06:37:58.808+0000\"}}",
              "HistorySummary": "Alarm \"ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\" created"
          }
        ]
      }
```

3.10. アラームの送信テスト
-------------------------

```bash:コマンド:
CWATCH_STATE_VALUE='ALARM'
CWATCH_STATE_REASON='test alarm.'
```

```bash:command:
aws cloudwatch set-alarm-state \
        --alarm-name ${CWATCH_ALARM_NAME} \
        --state-value ${CWATCH_STATE_VALUE} \
        --state-reason "${CWATCH_STATE_REASON}"
```

```text:result:
      (戻り値なし)
```


3.11. historyの確認
------------------

```bash:コマンド:
CWATCH_MAX_ITEMS='3'
```

```bash:コマンド:
aws cloudwatch describe-alarm-history \
        --alarm-name ${CWATCH_ALARM_NAME} \
        --max-items ${CWATCH_MAX_ITEMS}
```

```json:結果(例):
      {
        "AlarmHistoryItems": [
          {
              "Timestamp": "2015-02-14T06:45:27.209Z",
              "HistoryItemType": "StateUpdate",
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "HistoryData": "{\"version\":\"1.0\",\"oldState\":{\"stateValue\":\"ALARM\",\"stateReason\":\"test alarm.\"},\"newState\":{\"stateValue\":\"OK\",\"stateReason\":\"Threshold Crossed: 1 datapoint (8.0) was not greater than or equal to the threshold (240.0).\",\"stateReasonData\":{\"version\":\"1.0\",\"queryDate\":\"2015-02-14T06:45:27.200+0000\",\"startDate\":\"2015-02-14T06:25:00.000+0000\",\"unit\":\"Count\",\"statistic\":\"Sum\",\"period\":300,\"recentDatapoints\":[8.0],\"threshold\":240.0}}}",
              "HistorySummary": "Alarm updated from ALARM to OK"
          },
          {
              "Timestamp": "2015-02-14T06:45:26.465Z",
              "HistoryItemType": "Action",
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "HistoryData": "{\"actionState\":\"Succeeded\",\"notificationResource\":\"arn:aws:sns:us-west-2:XXXXXXXXXXXX:dynamodb\",\"stateUpdateTimestamp\":\"2015-02-14T06:45:26.450+0000\",\"publishedMessage\":\"{\\\"default\\\":\\\"{\\\\\\\"AlarmName\\\\\\\":\\\\\\\"ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\\\\\\\",\\\\\\\"AlarmDescription\\\\\\\":\\\\\\\"Read Capacity Units Limit Basic Alarm for ProductCatalog\\\\\\\",\\\\\\\"AWSAccountId\\\\\\\":\\\\\\\"XXXXXXXXXXXX\\\\\\\",\\\\\\\"NewStateValue\\\\\\\":\\\\\\\"ALARM\\\\\\\",\\\\\\\"NewStateReason\\\\\\\":\\\\\\\"test alarm.\\\\\\\",\\\\\\\"StateChangeTime\\\\\\\":\\\\\\\"2015-02-14T06:45:26.450+0000\\\\\\\",\\\\\\\"Region\\\\\\\":\\\\\\\"US-West-2\\\\\\\",\\\\\\\"OldStateValue\\\\\\\":\\\\\\\"OK\\\\\\\",\\\\\\\"Trigger\\\\\\\":{\\\\\\\"MetricName\\\\\\\":\\\\\\\"ConsumedReadCapacityUnits\\\\\\\",\\\\\\\"Namespace\\\\\\\":\\\\\\\"AWS/DynamoDB\\\\\\\",\\\\\\\"Statistic\\\\\\\":\\\\\\\"SUM\\\\\\\",\\\\\\\"Unit\\\\\\\":\\\\\\\"Count\\\\\\\",\\\\\\\"Dimensions\\\\\\\":[{\\\\\\\"name\\\\\\\":\\\\\\\"TableName\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"ProductCatalog\\\\\\\"}],\\\\\\\"Period\\\\\\\":300,\\\\\\\"EvaluationPeriods\\\\\\\":12,\\\\\\\"ComparisonOperator\\\\\\\":\\\\\\\"GreaterThanOrEqualToThreshold\\\\\\\",\\\\\\\"Threshold\\\\\\\":240.0}}\\\",\\\"sms\\\":\\\"ALARM: \\\\\\\"ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\\\\\\\" in US-West-2\\\",\\\"email\\\":\\\"You are receiving this email because your Amazon CloudWatch Alarm \\\\\\\"ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\\\\\\\" in the US-West-2 region has entered the ALARM state, because \\\\\\\"test alarm.\\\\\\\" at \\\\\\\"Saturday 14 February, 2015 06:45:26 UTC\\\\\\\".\\\\n\\\\nView this alarm in the AWS Management Console:\\\\nhttps://console.aws.amazon.com/cloudwatch/home?region=us-west-2#s=Alarms&alarm=ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\\\\n\\\\nAlarm Details:\\\\n- Name:                       ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm\\\\n- Description:                Read Capacity Units Limit Basic Alarm for ProductCatalog\\\\n- State Change:               OK -> ALARM\\\\n- Reason for State Change:    test alarm.\\\\n- Timestamp:                  Saturday 14 February, 2015 06:45:26 UTC\\\\n- AWS Account:                XXXXXXXXXXXX\\\\n\\\\nThreshold:\\\\n- The alarm is in the ALARM state when the metric is GreaterThanOrEqualToThreshold 240.0 for 300 seconds. \\\\n\\\\nMonitored Metric:\\\\n- MetricNamespace:            AWS/DynamoDB\\\\n- MetricName:                 ConsumedReadCapacityUnits\\\\n- Dimensions:                 [TableName = ProductCatalog]\\\\n- Period:                     300 seconds\\\\n- Statistic:                  Sum\\\\n- Unit:                       Count\\\\n\\\\nState Change Actions:\\\\n- OK: \\\\n- ALARM: [arn:aws:sns:us-west-2:XXXXXXXXXXXX:dynamodb]\\\\n- INSUFFICIENT_DATA: \\\\n\\\"}\"}",
              "HistorySummary": "Successfully executed action arn:aws:sns:us-west-2:XXXXXXXXXXXX:dynamodb"
          },
          {
              "Timestamp": "2015-02-14T06:45:26.450Z",
              "HistoryItemType": "StateUpdate",
              "AlarmName": "ProductCatalog-ReadCapacityUnitsLimit-BasicAlarm",
              "HistoryData": "{\"version\":\"1.0\",\"oldState\":{\"stateValue\":\"OK\",\"stateReason\":\"Threshold Crossed: 1 datapoint (8.0) was not greater than or equal to the threshold (240.0).\",\"stateReasonData\":{\"version\":\"1.0\",\"queryDate\":\"2015-02-14T06:37:59.447+0000\",\"startDate\":\"2015-02-14T06:22:00.000+0000\",\"unit\":\"Count\",\"statistic\":\"Sum\",\"period\":300,\"recentDatapoints\":[8.0],\"threshold\":240.0}},\"newState\":{\"stateValue\":\"ALARM\",\"stateReason\":\"test alarm.\"}}",
              "HistorySummary": "Alarm updated from OK to ALARM"
          }
        ],
        "NextToken": "None___3"
      }
```


 
完了
====