dockerでElasticSearch立ち上げとお試し操作

ElasticSearchの学習をしていくメモ
Kibanaはおまけ程度にたちあげてます

# dockerfile

```dockerfile

FROM docker.elastic.co/elasticsearch/elasticsearch:7.4.1
RUN elasticsearch-plugin install analysis-kuromoji
```


# docker-compose.yml

```docker-compose.yml
version: '3.7'
services:
  elastic1:
    build:
      context: ./docker
      dockerfile: elasticsearch
    container_name: elastic1
    environment:
      - node.name=elastic1
      - discovery.seed_hosts=elastic2
      - cluster.initial_master_nodes=elastic1,elastic2
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - esnet

  elastic2:
    build:
      context: ./docker
      dockerfile: elasticsearch
    container_name: elastic2
    environment:
      - node.name=elastic2
      - discovery.seed_hosts=elastic1
      - cluster.initial_master_nodes=elastic1,elastic2
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data2:/usr/share/elasticsearch/data
    networks:
      - esnet

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.4.1
    links:
      - elastic1:elasticsearch
    ports:
      - 5601:5601
    networks:
      - esnet

volumes:
  data1:
    driver: local
  data2:
    driver: local

networks:
  esnet:
```

```bash
-> % docker build && docker-compose up -d
```
でdocker-composeの起動を行う

※めちゃくちゃ重たいので他にコンテナを立ち上げてると立ち上がらない可能性あり

# ヘルスチェック
```bash
-> % curl -X GET "localhost:9200/_cat/nodes?v&pretty"
```
# create database

```bash
-> % curl -X PUT "localhost:9200/user?pretty&pretty"
```

```bash
-> % curl -X PUT "localhost:9200/user?pretty&pretty"
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "user"
}
```

# delete database

```bash
curl -X DELETE "localhost:9200/user?pretty&pretty"
```
```bash
-> % curl -X DELETE "localhost:9200/user?pretty&pretty"
{
  "acknowledged" : true
}
```

# show databases
```bash
curl -X GET "localhost:9200/_cat/indices?v&pretty"
```
```bash
-> % curl -X GET "localhost:9200/_cat/indices?v&pretty"
health status index                    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   test                     4aTJ7KwmQcmsNF-8eGIzgA   1   1          1            0     10.7kb          5.3kb
green  open   .kibana_task_manager_1   bZ0T1LvwTfOG8PxTN3ouZg   1   1          2            4       91kb         45.5kb
green  open   .apm-agent-configuration eEDd2yaASzunjKh8hTZ7DA   1   1          0            0       460b           230b
green  open   .kibana_1                rFy1NZniRHCBCAKNFTZRHQ   1   1          5            0     42.5kb         21.2kb
```

# データを入れる

```testdata.json
{
    "Email": "yamada@mail.com",
    "Full Name": "山田太郎",
    "Country": "日本",
    "User Id": 9427,
    "Created At": "2010-05-01T11:13:55.643Z"
}
```
```bash
curl -X POST "localhost:9200/test/user?pretty&pretty" -H 'Content-Type: application/json' -d @testdata.json
```
```bash
-> % curl -X POST "localhost:9200/test/user?pretty&pretty" -H 'Content-Type: application/json' -d @testdata.json
{
  "_index" : "test",
  "_type" : "user",
  "_id" : "u_majW4BpQEVFre2znXK",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}
```

# データを取得
``` bash
curl -X GET "localhost:9200/test/user/u_majW4BpQEVFre2znXK"
```
```bash
-> % curl -X GET "localhost:9200/test/user/u_majW4BpQEVFre2znXK" | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   281  100   281    0     0   1587      0 --:--:-- --:--:-- --:--:--  1578
{
  "_index": "test",
  "_type": "user",
  "_id": "u_majW4BpQEVFre2znXK",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "Email": "yamada@mail.com",
    "Full Name": "山田太郎",
    "Country": "日本",
    "User Id": 9427,
    "Created At": "2010-05-01T11:13:55.643Z"
  }
}
```

# テストデータを複数インポートするのは面倒
bulkAPIを使う

```bash
curl -X POST 'localhost:9200/test/user/_bulk?pretty' -H 'Content-Type: application/x-ndjson' --data-binary @testdata.json
```
```testdata.json
{ "index" : {}       }
{"FullName":"HollieHegmann","Country":"Anguilla","CreatedAt":"2015-10-08T04:50:27.146Z","Id":0,"Email":"Camren.Daugherty@claire.ca"}
{ "index" : {}       }
{"FullName":"AlexCassinV","Country":"AntiguaandBarbuda","CreatedAt":"2006-04-17T16:22:10.784Z","Id":1,"Email":"Jadyn.Lehner@mallie.co.uk"}
```

```json
{インデックス情報}
{データ情報}
{インデックス情報}
{データ情報}
```

の形式でjsonファイルを作成する。
最後に改行を含めることを忘れない

```bash
-> % curl -X POST 'localhost:9200/test/user/_bulk?pretty' -H 'Content-Type: application/x-ndjson' --data-binary @testdata.json
{
  "took" : 1035,
  "errors" : true,
  "items" : [
    {
      "index" : {
        "_index" : "test",
        "_type" : "user",
        "_id" : "Av_TjW4B_Mnm7UYRXv2w",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 3,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "test",
        "_type" : "user",
        "_id" : "A__TjW4B_Mnm7UYRXv2x",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 3,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "test",
        "_type" : "user",
        "_id" : "BP_TjW4B_Mnm7UYRXv2x",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,

...
```

# 検索してみる
```bash
curl 'localhost:9200/test/user/_search?size=3'
```
sizeは指定しないこともできるが念のため指定しておいたほうがいい

```bash
-> % curl 'localhost:9200/test/user/_search?size=3' |jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   843  100   843    0     0  10948      0 --:--:-- --:--:-- --:--:-- 10948
{
  "took": 70,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1000,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "test",
        "_type": "user",
        "_id": "u_majW4BpQEVFre2znXK",
        "_score": 1,
        "_source": {
          "Email": "yamada@mail.com",
          "Full Name": "山田太郎",
          "Country": "日本",
          "User Id": 9427,
          "Created At": "2010-05-01T11:13:55.643Z"
        }
      },
      {
        "_index": "test",
        "_type": "user",
        "_id": "Av_TjW4B_Mnm7UYRXv2w",
        "_score": 1,
        "_source": {
          "FullName": "HollieHegmann",
          "Country": "Anguilla",
          "CreatedAt": "2015-10-08T04:50:27.146Z",
          "Id": 0,
          "Email": "Camren.Daugherty@claire.ca"
        }
      },
      {
        "_index": "test",
        "_type": "user",
        "_id": "A__TjW4B_Mnm7UYRXv2x",
        "_score": 1,
        "_source": {
          "FullName": "AlexCassinV",
          "Country": "AntiguaandBarbuda",
          "CreatedAt": "2006-04-17T16:22:10.784Z",
          "Id": 1,
          "Email": "Jadyn.Lehner@mallie.co.uk"
        }
      }
    ]
  }
}
```

 # 検索(オプション付き)

```bash
curl 'localhost:9200/test/user/_search?size=3' -H 'Content-Type: application/json' -d @query.json
```
検索オプション用のjsonを用意することで絞り込み検索を行うことができる

```query.json
{
  "query": {
    "match": {
      "Full Name": "山田"
    }
  }
}
```

```bash
-> % curl 'localhost:9200/test/user/_search?size=3' -H 'Content-Type: application/json' -d @query.json |jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   472  100   409  100    63  51125   7875 --:--:-- --:--:-- --:--:-- 59000
{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 0.5753642,
    "hits": [
      {
        "_index": "test",
        "_type": "user",
        "_id": "u_majW4BpQEVFre2znXK",
        "_score": 0.5753642,
        "_source": {
          "Email": "yamada@mail.com",
          "Full Name": "山田太郎",
          "Country": "日本",
          "User Id": 9427,
          "Created At": "2010-05-01T11:13:55.643Z"
        }
      }
    ]
  }
}
```