Kibana Graphファーストインプレッション

# はじめに
最近、グラフDBあたりをちょっと学習していて、タイムリーにKibana Graphの情報をキャッチしたのでその流れで触ったみた

* http://qiita.com/nakamura-tsuyoshi/items/d284ad1eb7ca848d5d79
* http://qiita.com/nakamura-tsuyoshi/items/4a0cbf20ce9ba4c934b9
* http://qiita.com/nakamura-tsuyoshi/items/5ae38b04373a224d9a34


# 環境準備
## elasticsearchのインストール
いつものように[公式サイト](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html)から楽チンにインストールします

### バーションの確認
```
[root@elastic_test ec2-user]# curl localhost:9200
{
  "name" : "Ord",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.3.2",
    "build_hash" : "b9e4a6acad4008027e4038f6abed7f7dba346f94",
    "build_timestamp" : "2016-04-21T16:03:47Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}
```

## 続いてkibanaのインストール
これも[公式サイト](https://www.elastic.co/guide/en/kibana/4.5/setup.html#setup-repositories)から楽チンにインストールします
`http://ip:5601/`にアクセスすれば画面が見れます

## graphを入れてみる
公式のこの[あたり](https://www.elastic.co/downloads/graph)を参考にした
elasticsearch側とkibana側両方にpluginを入れる必要がある

### elasticsearchのpluginをインストール
#### licenseを入れる
```
[root@elastic_test ~]# /usr/share/elasticsearch/bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.2/license-2.3.2.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.3.2/license-2.3.2.zip checksums if available ...
Downloading .DONE
Installed license into /usr/share/elasticsearch/plugins/license
```

#### graph pluginを入れる
```
[root@elastic_test ~]# /usr/share/elasticsearch/bin/plugin install graph
-> Installing graph...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/graph/2.3.2/graph-2.3.2.zip ...
Downloading ....DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/graph/2.3.2/graph-2.3.2.zip checksums if available ...
Downloading .DONE
Installed graph into /usr/share/elasticsearch/plugins/graph
```

### kibanaのpluginをインストール
#### graphを入れる
```
[root@elastic_test ~]# /opt/kibana/bin/kibana plugin --install elasticsearch/graph/latest
Installing graph
Attempting to transfer from https://download.elastic.co/elasticsearch/graph/graph-latest.tar.gz
Transferring 69265 bytes....................
Transfer complete
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete
```

#### kibana再起動
```
[root@elastic_test ~]# /etc/init.d/kibana restart
kibana stopped.
kibana started
```

## ブラウザでインストールされたか確認
* `http://ip:5601/`にアクセス
![Settings   Kibana.png](https://qiita-image-store.s3.amazonaws.com/0/106937/451d85c2-b28d-09da-dbfd-92b43a644a28.png)

* graphが表示された
![Kibana.png](https://qiita-image-store.s3.amazonaws.com/0/106937/9588e8b4-7e95-4bbd-145a-77c1a94f7ebd.png)


# サンプルデータを作る
## データ構造
![20f31aeb-d47f-1d5c-d485-99b2b2c47f9a.png](https://qiita-image-store.s3.amazonaws.com/0/106937/c74a40d1-2189-8daa-cec0-5274bca10320.png)

* 店子が複数あるショップのユーザの購入データをインデックスさせる

## elasticsearchにテンプレートを作成する
```
[root@elastic_test ~]# curl -XPUT localhost:9200/_template/moall -d '
> {
>   "order": 0,
>   "template": "moall-*",
>   "settings": {
>     "index": {
>       "number_of_shards": "1",
>       "number_of_replicas": "0"
>     }
>   },
>   "mappings": {
>     "moall": {
>       "_source": {
>         "enabled": true
>       },
>       "properties": {
>         "order_time": {
>           "format": "YYYY-MM-dd HH:mm:ss",
>           "type": "date"
>         },
>         "price": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "user_id": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "user_name": {
>           "index": "not_analyzed",
>           "type": "string"
>         },
>         "user_gender": {
>           "index": "not_analyzed",
>           "type": "string"
>         },
>         "user_age": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "item_id": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "item_name": {
>           "index": "not_analyzed",
>           "type": "string"
>         },
>         "shop_id": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "shop_name": {
>           "index": "not_analyzed",
>           "type": "string"
>         },
>         "category_id": {
>           "index": "not_analyzed",
>           "type": "integer"
>         },
>         "category_name": {
>           "index": "not_analyzed",
>           "type": "string"
>         }
>       }
>     }
>   },
>   "aliases": {
>
>   }
> }'
{"acknowledged":true}
```

## データを入れる
```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/1  -d '
> {
>     "order_time" : "2016-04-02 22:22:22",
>     "price": 9500,
>     "user_id": 1,
>     "user_name": "user_name1",
>     "user_gender": "male",
>     "user_age": 37,
>     "item_id": 1,
>     "item_name": "商品A",
>     "shop_id": 1,
>     "shop_name": "ショップA",
>     "category_id": 1,
>     "category_name": "カテゴリ1"
> }'
{"_index":"moall-201604","_type":"moall","_id":"1","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/2  -d '
> {
>     "order_time" : "2016-04-03 11:21:11",
>     "price": 1000,
>     "user_id": 2,
>     "user_name": "user_name2",
>     "user_gender": "male",
>     "user_age": 17,
>     "item_id": 1,
>     "item_name": "商品A",
>     "shop_id": 1,
>     "shop_name": "ショップA",
>     "category_id": 1,
>     "category_name": "カテゴリ1"
> }'
{"_index":"moall-201604","_type":"moall","_id":"2","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/3  -d '
> {
>     "order_time" : "2016-04-04 09:45:20",
>     "price": 1200,
>     "user_id": 3,
>     "user_name": "user_name3",
>     "user_gender": "female",
>     "user_age": 27,
>     "item_id": 2,
>     "item_name": "商品B",
>     "shop_id": 1,
>     "shop_name": "ショップA",
>     "category_id": 1,
>     "category_name": "カテゴリ1"
> }'
{"_index":"moall-201604","_type":"moall","_id":"3","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/4  -d '
> {
>     "order_time" : "2016-04-05 10:18:22",
>     "price": 15000,
>     "user_id": 4,
>     "user_name": "user_name4",
>     "user_gender": "male",
>     "user_age": 44,
>     "item_id": 3,
>     "item_name": "商品C",
>     "shop_id": 2,
>     "shop_name": "ショップB",
>     "category_id": 1,
>     "category_name": "カテゴリ1"
> }'
{"_index":"moall-201604","_type":"moall","_id":"4","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/5  -d '
> {
>     "order_time" : "2016-03-22 10:18:22",
>     "price": 2000,
>     "user_id": 1,
>     "user_name": "user_name1",
>     "user_gender": "male",
>     "user_age": 37,
>     "item_id": 4,
>     "item_name": "商品D",
>     "shop_id": 3,
>     "shop_name": "ショップC",
>     "category_id": 2,
>     "category_name": "カテゴリ2"
> }'
{"_index":"moall-201604","_type":"moall","_id":"5","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/6  -d '
> {
>     "order_time" : "2016-02-11 10:18:22",
>     "price": 4000,
>     "user_id": 2,
>     "user_name": "user_name2",
>     "user_gender": "male",
>     "user_age": 17,
>     "item_id": 4,
>     "item_name": "商品D",
>     "shop_id": 3,
>     "shop_name": "ショップC",
>     "category_id": 2,
>     "category_name": "カテゴリ2"
> }'
{"_index":"moall-201604","_type":"moall","_id":"6","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

```
[root@elastic_test ~]# curl -X POST http://localhost:9200/moall-201604/moall/7  -d '
> {
>     "order_time" : "2016-03-25 10:18:22",
>     "price": 5000,
>     "user_id": 1,
>     "user_name": "user_name1",
>     "user_gender": "male",
>     "user_age": 37,
>     "item_id": 5,
>     "item_name": "商品E",
>     "shop_id": 3,
>     "shop_name": "ショップC",
>     "category_id": 2,
>     "category_name": "カテゴリ2"
> }'
{"_index":"moall-201604","_type":"moall","_id":"7","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}
```

# kibana graphで確認
![Kibana-1.png](https://qiita-image-store.s3.amazonaws.com/0/106937/45b15981-1ece-c848-7758-774888da0d47.png)

* 購入ユーザと商品を表示させた

![Kibana-2.png](https://qiita-image-store.s3.amazonaws.com/0/106937/6f157f69-44bd-7521-46b2-5d7078e03591.png)

* 購入ユーザと商品とショップを表示させた
* user_name1はショップCで２回購入しているためラインが太く表現されている

## 発行されてるリクエストは
下記のように確認ができる
![Kibana-4.png](https://qiita-image-store.s3.amazonaws.com/0/106937/6eb2fff2-7c38-8abc-d05f-d84d9ff1bd89.png)
* なのでcurl経由で叩けば数値としてデータが取得できる



# ちょっとハマった点やメモなど
## サンプルデータが少なかったので設定値を変更しないとグラフが出てこなかった
![Kibana-3.png](https://qiita-image-store.s3.amazonaws.com/0/106937/d50d1ddc-38a2-8d79-58ae-a2c5f3414dcd.png)

* significant設定を外した。多く使われてる重要語を抽出してくれる的な機能。詳しい説明は下記あたり。
 * https://www.elastic.co/guide/en/graph/current/graph-api-rest.html#api-rest-graph-explore
 * https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-significantterms-aggregation.html
* Certaintyを1に設定した

## 異なるインデックスの複数指定ができない
* 購入データに関してのグラフとuserどうしのつながりのグラフを一緒に観てみる的なことが無理？みたいです

## 出てきたダイアグラムをいい感じに動かせない
* クリックしてドラッグしてこのアイテムはこのスペースに持ってきて、・・・的な操作はできなかった
* 拡大、縮小、移動はできたけど

## アイテムとアイテム間の繋がりに種類を持たすことはできない
* 例えばグラフDBだとrelationsを作る際にfrendsを表すrelationと身内を表すrelationを分けて貼れたりするけど、それ系のものができなかった

## 導入がちょー楽
* すでに何かのデータをelasticsearchに入れて運用していれば、プラグインでとりあえずインストールすれば使える。
* 新たな視点でのデータの見える化ができて良さそう
 * 社内ツール向きかな


## Kibanaの再起動だけではダメでelasticsearchの再起動も必要
再起動しないと下記のようなエラーを吐いた

```
[2016-05-05 09:26:21,130][DEBUG][action.admin.indices.mapping.put] [Ord] failed to put mappings on indices [[moall-201604]], type [_graph]
InvalidTypeNameException[Document mapping type name can't start with '_']
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:288)
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)
	at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:468)
	at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
```


# 最後
* 何と言っても導入が楽なので既存のインデックスに試してみて、可視化させてみたらあたらしいデータ的な特徴が見つかるかもなぁと思った。
* 30日間は無料みたいだが、それ以降はライセンス契約が必要。どの程度費用かかるか少し探してみたけど見つからなかった
* まぁものが全然違うのでGraphDBと比較みたいな視点はあまり意味がないと思った