enebular INFOMOTIONでfluentdモニタリング

## 概要

今回はtd-agent（fluentd）のMonitoring Agentの情報をenebular INFOMOTIONで可視化します。

想定の環境としては同一ネットワーク内に2台のサーバがあり、両方ともfluentdが稼働している状況です。今回は1方のサーバにNode-REDをインストールしてMonitoring Agentの情報を定期的に取得してMilkcocoaに投げます。

## Monitoring Agentの設定

対象サーバ両方のtd-agent.confに以下を追記してtd-agentを再起動するだけです。

```bash
<source>
  type monitor_agent
  bind 0.0.0.0
  port 24220
</source>
```

以下のようにcurlで動作確認します。

```bash
curl -s http://＜対象ホスト＞:24220/api/plugins.json | jq .
```

以下のようなJSONが帰ってくればOKです。

```json
{
  "plugins": [
    {
      "config": {
        "port": "24220",
        "bind": "0.0.0.0",
        "type": "monitor_agent"
      },
      "retry_count": null,
      "output_plugin": false,
      "type": "monitor_agent",
      "plugin_id": "object:3fdda85f7894"
    },
    {
      "config": {
        "format": "/((?<head>[^ ]*) *(?<app_id>[^ ]*) *(?<timestamp>[^ ]*) *(?<stream>[^ ]*) *(?<action>[^ ]*) *(?<message>.*))/",
        "tag": "container",
        "pos_file": "＜パス＞",
        "path": "＜パス＞*",
        "type": "tail"
      },
      "retry_count": null,
      "output_plugin": false,
      "type": "tail",
      "plugin_id": "object:3fdda8b16d48"
    },
    {
      "config": {
        "rewriterule0": "head ^(.*)$ $1",
        "type": "rewrite_tag_filter"
      },
      "retry_count": null,
      "output_plugin": true,
      "type": "rewrite_tag_filter",
      "plugin_id": "object:3fdda8ebf0d0"
    },
    {
      "config": {
        "type": "forward"
      },
      "retry_count": 0,
      "buffer_total_queued_size": 0,
      "buffer_queue_length": 0,
      "output_plugin": true,
      "type": "forward",
      "plugin_id": "object:3fdda8ba7f00"
    }
  ]
}
```

## Node-REDでFlowを作る

対象サーバのどちらか1方に[Node-REDをインストール](http://nodered.jp/docs/getting-started/installation.html)後、以下のFlowを作ります。

```json
[{"id":"4b69c020.d73198","type":"http request","z":"5b2676c5.07cfb","name":"","method":"GET","ret":"txt","url":"http://＜対象ホスト2＞:24220/api/plugins.json","tls":"","x":382,"y":79.5,"wires":[["fa4f4574.7d551"]]},{"id":"72c141c.aa5434","type":"inject","z":"5b2676c5.07cfb","name":"","topic":"＜対象ホスト2＞","payload":"","payloadType":"date","repeat":"60","crontab":"","once":false,"x":138,"y":84,"wires":[["4b69c020.d73198"]]},{"id":"33db5750.8d4218","type":"debug","z":"5b2676c5.07cfb","name":"","active":true,"console":"false","complete":"false","x":515,"y":245.25,"wires":[]},{"id":"fa4f4574.7d551","type":"json","z":"5b2676c5.07cfb","name":"","x":555,"y":56.5,"wires":[["cb1eec69.15c06"]]},{"id":"cb1eec69.15c06","type":"splitter","z":"5b2676c5.07cfb","name":"","property":"payload.plugins","x":113.5,"y":137.5,"wires":[["9ed351c9.144168"]]},{"id":"9ed351c9.144168","type":"change","z":"5b2676c5.07cfb","name":"","rules":[{"t":"delete","p":"payload.config","pt":"msg"},{"t":"set","p":"payload.host","pt":"msg","to":"topic","tot":"msg"},{"t":"set","p":"topic","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":294.5,"y":136.5,"wires":[["3e90309d.e7ed2"]]},{"id":"df7a4bc4.a1c748","type":"inject","z":"5b2676c5.07cfb","name":"","topic":"＜対象ホスト1＞","payload":"","payloadType":"date","repeat":"60","crontab":"","once":false,"x":140,"y":40,"wires":[["e53db351.d3c45"]]},{"id":"e53db351.d3c45","type":"http request","z":"5b2676c5.07cfb","name":"","method":"GET","ret":"txt","url":"http://＜対象ホスト1＞:24220/api/plugins.json","tls":"","x":384,"y":40.5,"wires":[["fa4f4574.7d551"]]},{"id":"3e90309d.e7ed2","type":"milkcocoa out","z":"5b2676c5.07cfb","milkcocoa":"7b028185.45982","dataStore":"fluentd","operation":"send","targetId":"","name":"","x":471.5,"y":137,"wires":[["33db5750.8d4218"]]},{"id":"7b028185.45982","type":"milkcocoa","z":"5b2676c5.07cfb","appId":"＜あなたのApp ID＞"}]
```

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/3351653b-1b0b-51f6-24b5-3c1be7ff8449.png)

本来は対象ホストのマスタなどを何らかのストレージに保持して対象ホストが増加した場合にスケールするFlowにする必要があります。

これで1分間隔でtd-agent（fluentd）のMonitoring Agentの情報がMilkcocoaに送信されます。

## enebular INFOMOTIONで可視化

あとはenebular INFOMOTIONの設定をします。[データソースの設定](http://enebular.com/developers/infomotion-datasource)や[INFOMOTIONの作成](http://enebular.com/developers/infomotion-start)の詳細は各リンク先のドキュメントを参照します。

### データソース設定

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/261c0477-f433-f147-f007-2d953849c0ee.png)

### INFOMOTION設定

| 項目 | 設定する値 |
|:-:|:-:|
| TYPE | metricsgraphics |
| DATASOURCE | fluentd |
| rows y_accessor | buffer_total_queued_size |
| legend name | Buffer Size |
| limit | 20 |

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/8194abe3-607c-9938-1dd9-3b1e37b67c9e.png)

ここで一旦Liveモードにします。以下のように表示されればOKです。

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/37f86a25-5a91-4234-9673-679d1dbbc5da.png)

### フィルタ設定

グラフ左上のアイコンをクリックしてフィルタ設定を開きます。

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/b0ea92a7-0b4b-dd99-1fcb-d62e1e837ad2.png)

以下のように対象ホストと対象プラグインを絞り込むフィルタを設定します。

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/3a7a2712-cd81-fa2e-a394-776c59eb05d9.png)

### 完成

ずっと0なんでつまんないですが以下のように可視化できました。

![image](https://qiita-image-store.s3.amazonaws.com/0/21109/79b0f2f9-3bd8-3cd7-1b40-032052b92d5e.png)