Amplify + ElasticSearchService でさっくりと2点間距離検索・・できるのか？

amplify+AppSyncで拠点から拠点への距離順でソートしたい
https://qiita.com/ikegam1/items/5d6ad33b13811f7ca87f

と

Amplify, AppSync, ElasticSearch, DynamoDBで@searchableを使ったキーワード検索を実装したい
https://qiita.com/ikegam1/items/edb7fe40073d3e9cdd63

の合わせ技をやりたい。
具体的には、Amplify側のCLI(つまりコマンド)にてgeo_point Typeを用いるgraphqlのエンドポイントをdeployしたい。

## やりたいこと

- amplify使う
- AppSyncはamplify cliから透過的に
- DynamoDBはamplify cliから透過的に
- ElasticSearchはamplify cliから透過的に
- 緯度経度を用いたクエリで対象との距離を取得

## 環境

- win10 Home(surface go) + WSL2 + ubuntu18.04
- node v14.1.0
- npm 6.14.5

## Setup Amplify Backend

さて、何度も書いてる初期設定。
使ったことないんだけど、今度はnext.jsにしてみる。まあ本題はUI側じゃないのでなんでもいい。

next-amp ってプロジェクトフォルダで進めます。
なお、下記のあたりのamplify-cliはインストール済としていきます。

https://docs.amplify.aws/start/getting-started/installation/q/integration/next#option-2-follow-the-instructions

```bash
$ npx create-next-app next-amp
$ cd next-amp
$ amplify init

Scanning for plugins...
Plugin scan successful
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project nextamp
? Enter a name for the environment dev
? Choose your default editor: Vim (via Terminal, Mac OS only)
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path:  src
? Distribution Directory Path: build
? Build Command:  npm run-script build
? Start Command: npm run-script start
Using default provider  awscloudformation

...

$ npm install aws-amplify @aws-amplify/ui-react
$ amplify add api

? Please select from one of the below mentioned services: GraphQL
? Provide API name: nextamp
? Choose the default authorization type for the API API key
? Enter a description for the API key: xxxxxxxx
? After how many days from now the API key should expire (1-365): 7
? Do you want to configure advanced settings for the GraphQL API No, I am done.
? Do you have an annotated GraphQL schema? No
? Do you want a guided schema creation? Yes
? What best describes your project: Single object with fields (e.g., “Todo” with ID, name, description)
? Do you want to edit the schema now? Yes

The following types do not have '@auth' enabled. Consider using @auth with @model
         - Tree

...

```

schemaはこんな感じです。
modelに`@searchable`を入れてるのでElasticSearchが構築されます。

```graphql:schema.graphql
type Tree @model
@searchable {
 id: ID!
 species: String!
 location: Location!
 height: Float
}
type Location {
 lat: Float!
 lon: Float!
}
input LocationInput {
  lat: Float!
  lon: Float!
}
input NearbyTreesInput {
  location: LocationInput!
  km: Int!
  limit: Int!
}
type Query {
  nearbyTrees(input: NearbyTreesInput!): NearbyTreesConnection!
}
type NearbyTreesConnection {
 items: [NearbyTreeConnectionItem!]!
}
type NearbyTreeConnectionItem {
 tree: Tree!
 distance: Float!
}
```

そしてpush

```
$ amplify push

✔ Successfully pulled backend environment dev from the cloud.

Current Environment: dev

| Category | Resource name | Operation | Provider plugin   |
| -------- | ------------- | --------- | ----------------- |
| Api      | nextamp       | Create    | awscloudformation |
? Are you sure you want to continue? Yes

The following types do not have '@auth' enabled. Consider using @auth with @model
         - Tree
Learn more about @auth here: https://docs.amplify.aws/cli/graphql-transformer/directives#auth


GraphQL schema compiled successfully.

Edit your schema at /mnt/c/Users/xxxxx/next-amp/amplify/backend/api/nextamp/schema.graphql or place .graphql files in a directory at /mnt/c/Users/xxxxx/next-amp/amplify/backend/api/nextamp/schema
? Do you want to generate code for your newly created GraphQL API Yes
? Choose the code generation language target javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested] 2
```

はい、ここまでは初期設定。
今回は試さないけど、AppSyncのコンソールでごく普通のクエリは実行できるはず。

## AppSyncのResolverを設定

で、ElasticSearchのgeo_pointを使いたい。
schema.graphqlでそんなの書けないすよね、たぶん。
ということで、Resolverってやつをいじってみる。
`amplify/backend/api/nextamp/resolvers` の中に下記の二つのファイルを作成します。

- Query.nearbyTrees.req.vtl
- Query.nearbyTrees.res.vtl


```js:Query.nearbyTrees.req.vtl
#set( $indexPath = "/tree/doc/_search" )
#set( $input = $ctx.args.input)
#set( $distance = $util.defaultIfNull($input.km, 1000) )
{
  "version": "2018-05-29",
  "operation" : "GET",
  "path": "$indexPath",
  "params": {
    "body": {
      "size": #if( $input.limit ) $input.limit #else 100 #end,
      "from": #if( $input.from ) $input.from #else 0 #end,
      "sort": [{
        "_geo_distance": {
          "location": $util.toJson($input.location),
          "order": "asc",
          "unit": "km", 
          "distance_type": "plane" 
        }
      }],
      "query": {
        "bool" : {
          "must" : {
              "match_all" : {}
          },
          "filter" : {
            "geo_distance" : {
                "distance" : "${distance}km",
                "location" : $util.toJson($input.location)
            }
          }
        }
      }
    }
  }
}
```

```js:Query.nearbyTrees.res.vtl
#**
    $context.result contains the full response of the Elasticsearch query.
    Select a subset of information or iterate through hits to return the
    same shape as is expected by this field.
*#
#set( $es_items = [] )
#foreach( $entry in $context.result.hits.hits)
  #set ( $item = {
    "tree": $entry.get("_source"),
    "distance": $entry.get("sort").get(0)
  })
  $util.qr($es_items.add($item))
#end

#set( $es_response = {
  "items": $es_items,
  "total": $context.result.hits.total
} )

#if( $velocityCount > 1 ) , #end
$util.toJson($es_response)
```

## ElasticSearchのindex作成

### ElasticSearch アクセス許可

コマンドを通したいので一時的にIPを通すことにします。

ElasticSearchServiceコンソールのアクセスポリシーを変更します。

![es02.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/74c878f1-092a-e5e5-59f3-66db2115b45f.png)

![es03.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/7be70edf-8695-882d-4194-168aa98784f0.png)




### mapping設定

`curl -XPUT {host}/tree -d @mapping.json -H 'Content-Type:application/json'`

`host`についてはawsのコンソールでelasticSearchServiceを開き、endpointを確認します。

```json:mapping.json
{
  "mappings": {
    "_doc": {
      "properties": {
        "createdAt": {
          "type": "date"
        },
        "height": {
          "type": "float"
        },
        "id": {
          "type": "text"
        },
        "species": {
          "type": "text"
        },
        "updatedAt": {
          "type": "date"
        },
        "location": {
          "type": "geo_point"
        }
      }
    } 
  }
}
```

![es01.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/6e2f3cd5-88b9-4294-b526-13189c7fa815.png)

コマンド実行後、こうなりました。

![es04.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/734bdb79-f561-a6a5-e6da-bbfc3616dfb1.png)

### AppSyncコンソールからデータを登録

一先ずデータを入れておいてみますね。

![es05.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/b1f7c976-904d-02ed-4bfb-510a127bb250.png)

登録はあっさり。入力値は適当です。

## クエリ　１度目

この状態で一回クエリ実行してみます。

・・が、

`Cannot return null for non-nullable type: 'NearbyTreesConnection' within parent 'Query' (/nearbyTrees)`
（キャプチャは撮り忘れ）

と返ってきてしまいました。。
・・まあ正直ここは予想できてました。
さて、どうすれば解決するのか。

## schema.graphqlを変更

少し変更してみます。
NearbyTreesConnectionをnon-nullable typeじゃなくしてみよう。id以外はoptionにしてみようかな。
あと、total件数を返り値でもらうようにしました。

```graphql:schema.graphql
type Tree @model
@searchable {
 id: ID!
 species: String
 location: Location
 height: Float
}
type Location {
 lat: Float
 lon: Float
}
input LocationInput {
  lat: Float
  lon: Float
}
input NearbyTreesInput {
  location: LocationInput
  km: Int
  limit: Int
  from: Int
}
type Query {
  nearbyTrees(input: NearbyTreesInput!): NearbyTreesConnection
}
type NearbyTreesConnection {
 items: [NearbyTreeConnectionItem]
 total: Int
}
type NearbyTreeConnectionItem {
 tree: Tree
 distance: Float
}
```

そして、`amplify push`

・・からのQuery

・・

お・・？

・・

![es06.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/47c164f9-2477-a6eb-91d5-b3ed4a681478.png)

なんかうまくいけたっぽい！
(id3はlat:47,lon:122で登録されてる)

つぎはitemsの内容をいじりたいな。

## 複合条件とソートに対応する

実際には緯度経度だけでクエリやソートを行うわけではなくて、
例えば都道府県のIDで絞ったり、新着順でソートしたりといった事もありえる。
とりあえず、下記の条件を組み込めるようにしてみたい。

- 高度(height)の一致
- createdAtでの逆順ソート

### まずはAppSync通さず

ElasticSearchが受け入れてくれるクエリをあらかじめ作ってみたり実行してみたりしておきます。

**全件取得:**

````
$ curl -XPOST "{host}/tree/doc/_search" -d '{"query": {"match_all": {}}}' -H 'Content-Type:application/json'
{"took":212,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":4,"max_score":1.0,"hits":[{"_index":"tree","_type":"doc","_id":"id3","_score":1.0,"_source":{"createdAt": "2020-10-02T15:45:11.743Z", "species": "tes", "__typename": "Tree", "location": {"lon": 122.0, "lat": 47.0}, "id": "id3", "height": 1.5, "updatedAt": "2020-10-02T15:45:11.743Z"}},{"_index":"tree","_type":"doc","_id":"id4","_score":1.0,"_source":{"createdAt": "2020-10-02T19:31:32.604Z", "species": "aaa", "__typename": "Tree", "location": {"lon": 122.0, "lat": 1.5}, "id": "id4", "height": 1.5, "updatedAt": "2020-10-02T19:31:32.604Z"}},{"_index":"tree","_type":"doc","_id":"id5","_score":1.0,"_source":{"createdAt": "2020-10-02T19:31:48.949Z", "species": "aaa", "__typename": "Tree", "location": {"lon": 122.0, "lat": 33.0}, "id": "id5", "height": 3.0, "updatedAt": "2020-10-02T19:31:48.949Z"}},{"_index":"tree","_type":"doc","_id":"id6","_score":1.0,"_source":{"createdAt": "2020-10-02T19:32:06.449Z", "species": "abb", "__typename": "Tree", "location": {"lon": 111.0, "lat": 11.0}, "id": "id6", "height": 3.0, "updatedAt": "2020-10-02T19:32:06.449Z"}}]}}i
```

これはまあ単純

**次は絞り込み:**

ざっくりいってこういうクエリが実行できればひとまずはOK.

```json:q.json
{
    "from": 0,
    "size": 100,
    "query": {
      "bool": {
        "must": {"match_all": {}},
        "filter":{
          "bool":{
            "must":[
              {
                "geo_distance" : {
                    "distance" : "10000km",
                    "location" : {
                        "lat": 47.1,
                        "lon": 122.1
                    }
                },
                "term": {
                   "height": 1.5
                }
              }
            ]
          }
        }
      }
    },
    "sort"  : {
      "_geo_distance": {
        "location" : {
            "lat": 47.1,
            "lon": 122.1
        },
        "order": "asc",
        "unit": "km",
        "distance_type": "plane"
      },
      "createdAt": "desc"
    }
}
```

```bash
$ curl -XPOST "{host}/tree/doc/_search" -d @q.json -H 'Content-Type:application/json'
{"took":6,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":2,"max_score":null,"hits":[{"_index":"tree","_type":"doc","_id":"id3","_score":null,"_source":{"createdAt": "2020-10-02T15:45:11.743Z", "species": "tes", "__typename": "Tree", "location": {"lon": 122.0, "lat": 47.0}, "id": "id3", "height": 1.5, "updatedAt": "2020-10-02T15:45:11.743Z"},"sort":[13.455305682258599,1601653511743]},{"_index":"tree","_type":"doc","_id":"id4","_score":null,"_source":{"createdAt": "2020-10-02T19:31:32.604Z", "species": "aaa", "__typename": "Tree", "location": {"lon": 122.0, "lat": 1.5}, "id": "id4", "height": 1.5, "updatedAt": "2020-10-02T19:31:32.604Z"},"sort":[5070.505764236352,1601667092604]}]}}i
```

おっけー、いけてる。


schema.graphqlのinputのところをこうします。
paginationを意識して`from`とかも入れてます。

```graphql:schema.graphql
...

input NearbyTreesFilter {
  height: Float
  species: String
}
input NearbyTreesInput {
  location: LocationInput
  km: Int
  limit: Int
  from: Int
  filter: NearbyTreesFilter
  sort: String
}

...
```

requestの方のmapping templateはこうしてみます

```js:Query.nearbyTrees.req.vtl
#set( $indexPath = "/tree/doc/_search" )
#set( $input = $ctx.args.input)
#set( $distance = $util.defaultIfNull($input.km, 50000) )
#set( $nonKeywordFields = ["height"] )
#set( $sortDirection = "desc")
#set( $sortField = $util.defaultIfNull($input.sort, "createdAt") )
{
  "version": "2018-05-29",
  "operation" : "GET",
  "path": "$indexPath",
  "params": {
    "body": {
      "size": #if( $input.limit ) $input.limit #else 100 #end,
      "from": #if( $input.from ) $input.from #else 1 #end,
      "sort": {
        #if( $input.km )
        "_geo_distance": {
          "location": $util.toJson($input.location),
          "order": "asc",
          "unit": "km", 
          "distance_type": "plane" 
        }
        #else
        "$sortField": "$sortDirection",
        "_geo_distance": {
          "location": $util.toJson($input.location),
          "order": "asc",
          "unit": "km", 
          "distance_type": "plane" 
        }       
        #end
      },
      "query": {
        "bool": {
          "must": {"match_all": {}},
          "filter":{
            "bool":{
              "must":[
                {
                  "geo_distance" : {
                      "distance" : "${distance}km",
                      "location" : $util.toJson($input.location)
                  }
                }
                #if( $input.filter.height )
                ,
                {
                    "term": {
                       "height": $input.filter.height
                    }
                }
                #end
              ]
            }
          }
        }
      },
    }
  }
}
```

```js:Query.nearbyTrees.res.vtl
#**
    $context.result contains the full response of the Elasticsearch query.
    Select a subset of information or iterate through hits to return the
    same shape as is expected by this field.
*#
#set( $es_items = [] )
#foreach( $entry in $context.result.hits.hits)
  #if ( $entry.get("sort").size() > 1 )
  #set ( $item = {
    "tree": $entry.get("_source"),
    "distance": $entry.get("sort").get(1)
  })
  #else
  #set ( $item = {
    "tree": $entry.get("_source"),
    "distance": $entry.get("sort").get(0)
  })
  #end
  $util.qr($es_items.add($item))
#end

#set( $es_response = {
  "items": $es_items,
  "total": $context.result.hits.total
} )

#if( $velocityCount > 1 ) , #end
$util.toJson($es_response)
```

・・vtlの書き方詳しくなくて、ひとまず決め打ちな書き方でダサいけど、やりたい事ができるかの検証なので・・と言い訳しておきます。すくなくともfilterでは`eq`だとか使いたいし、`$util.transform.toElasticsearchQueryDSL`を活用しなきゃ、とは思ってる。


細かいことは置いといて、ひとまず `amplify push` .

・

・

・・そしてquery..

・

・



でけた！
height=3で絞りつつ、登録日降順。
パラメータ変えたり、登録日昇順にしたり、距離だけの指定にしてみたりもしたけど動作OK.

![es07.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/308709/43f5a771-d419-6257-bc08-66947133dfd9.png)

# 結論

amplifyから比較的簡単にESを使い倒すことができそう



・・しかし、
next.jsにしてみたものの、フロント側一切触りませんでしたね。。