EC2インスタンスにconsulを導入してクラスタを作成する手順

AWS上にEC２インスタンスを２台立てて、それぞれをconsulのServer, Clientとしてクラスタを構築するまでの手順です。
EC2インスタンスに決まったタグをつけて自動でConsulクラスタにjoinできるような設定にしました。

## インストール

```bash
$ wget https://releases.hashicorp.com/consul/1.0.3/consul_1.0.3_linux_amd64.zip
$ unzip consul_1.0.3_linux_amd64.zip
$ ./consul -v
Consul v1.0.3
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)
```

バージョン確認できればOK

[Installing Consul - Consul by HashiCorp](https://www.consul.io/intro/getting-started/install.html)

## 必要なディレクトリの作成とパーミッションの設定

```bash
$ sudo mv ./consul /usr/local/bin/ # anywhere you can detect by the PATH variable
$ sudo mkdir /etc/consul.d
$ sudo chown ec2-user:ec2-user /etc/consul.d
$ sudo mkdir /opt/consul
$ sudo chown ec2-user:ec2-user /opt/consul
```

* /etc/consul.d  
設定ファイルを置くディレクトリ
* /opt/consul  
consulが状態を管理したりのためにデータを置くディレクトリ

↑どちらもオプションなどで指定できるので他のディレクトリでも可

## 設定ファイルを書く

### サーバー側

```json:config-server.json
{
  "datacenter": "aws",
  "data_dir": "/opt/consul",
  "server": true,
  "retry_join": [
    "provider=aws tag_key=Name tag_value=consul"
  ]
}
```

### クライアント側

```json:config-client
{
  "datacenter": "aws",
  "data_dir": "/opt/consul",
  "retry_join": [
    "provider=aws tag_key=Name tag_value=consul"
  ]
}
```

[Configuration - Consul by HashiCorp#amazon-ec2](https://www.consul.io/docs/agent/options.html#amazon-ec2)

## EC2インスタンスにタグを設定する

設定ファイルで`tag_key=Name tag_value=consul`と指定したので、Nameタグにconsulという値を指定しました。

![consul_tag.png](https://qiita-image-store.s3.amazonaws.com/0/20045/15979611-f36c-0892-7067-b9a7654a7584.png)


## IAMロールで`ec2:DescribeInstances`を許可

どちらのインスタンスにも付与する

```json:iam-role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ec2:DescribeInstances",
            "Resource": "*"
        }
    ]
}
```

IAMロールじゃなくても

* 直接consulの起動オプションに`access_key_id`と`secret_access_key`を渡す
* 環境変数`AWS_ACCESS_KEY_ID`と`AWS_SECRET_ACCESS_KEY`を設定する
* shared credential file(~/.aws/credentials等)を準備する

でも可能

## Security GroupでConsulが使うポート番号を許可

8300と8301を開けました。
下記のリンク等を参考にして追加で必要なポートがあれば許可する必要があります。

https://www.consul.io/docs/agent/options.html#ports-used
https://qiita.com/ryo0301/items/a65709c4d99408f1df0a

## consul agentを起動

### サーバー側

```
$ consul agent --config-dir=/etc/consul.d --bootstrap-expect 1
BootstrapExpect is set to 1; this is the same as Bootstrap mode.
bootstrap = true: do not enable unless necessary
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.0.3'
           Node ID: '973f32e6-875b-542d-bf18-83e37da1ae62'
         Node name: 'ip-10-0-3-27'
        Datacenter: 'aws' (Segment: '<all>')
            Server: true (Bootstrap: true)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.0.3.27 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2018/02/11 07:13:42 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:973f32e6-875b-542d-bf18-83e37da1ae62 Address:10.0.3.27:8300}]
    2018/02/11 07:13:42 [INFO] serf: EventMemberJoin: ip-10-0-3-27.aws 10.0.3.27
    2018/02/11 07:13:42 [INFO] serf: EventMemberJoin: ip-10-0-3-27 10.0.3.27
    2018/02/11 07:13:42 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2018/02/11 07:13:42 [INFO] raft: Node at 10.0.3.27:8300 [Follower] entering Follower state (Leader: "")
    2018/02/11 07:13:42 [WARN] serf: Failed to re-join any previously known node
    2018/02/11 07:13:42 [INFO] serf: Attempting re-join to previously known node: ip-10-0-3-19: 10.0.3.19:8301
    2018/02/11 07:13:42 [INFO] consul: Adding LAN server ip-10-0-3-27 (Addr: tcp/10.0.3.27:8300) (DC: aws)
    2018/02/11 07:13:42 [INFO] consul: Handled member-join event for server "ip-10-0-3-27.aws" in area "wan"
    2018/02/11 07:13:42 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2018/02/11 07:13:42 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2018/02/11 07:13:42 [INFO] agent: started state syncer
    2018/02/11 07:13:42 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
    2018/02/11 07:13:42 [INFO] agent: Joining LAN cluster...
    2018/02/11 07:13:42 [INFO] discover-aws: Address type  is not supported. Valid values are {private_v4,public_v4,public_v6}. Falling back to 'private_v4'
    2018/02/11 07:13:42 [INFO] discover-aws: Region not provided. Looking up region in metadata...
    2018/02/11 07:13:42 [WARN] serf: Failed to re-join any previously known node
    2018/02/11 07:13:42 [INFO] discover-aws: Region is ap-northeast-1
    2018/02/11 07:13:42 [INFO] discover-aws: Filter instances with Name=consul
    2018/02/11 07:13:42 [INFO] discover-aws: Instance i-05cbabfc1ec12b005 has private ip 10.0.3.27
    2018/02/11 07:13:42 [INFO] discover-aws: Instance i-01503c36e371ea41a has private ip 10.0.3.19
    2018/02/11 07:13:42 [INFO] agent: Discovered LAN servers: 10.0.3.27 10.0.3.19
    2018/02/11 07:13:42 [INFO] agent: (LAN) joining: [10.0.3.27 10.0.3.19]
    2018/02/11 07:13:42 [INFO] agent: (LAN) joined: 1 Err: <nil>
    2018/02/11 07:13:42 [INFO] agent: Join LAN completed. Synced with 1 initial agents
    2018/02/11 07:13:49 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2018/02/11 07:13:49 [INFO] raft: Node at 10.0.3.27:8300 [Candidate] entering Candidate state in term 3
    2018/02/11 07:13:49 [INFO] raft: Election won. Tally: 1
    2018/02/11 07:13:49 [INFO] raft: Node at 10.0.3.27:8300 [Leader] entering Leader state
    2018/02/11 07:13:49 [INFO] consul: cluster leadership acquired
    2018/02/11 07:13:49 [INFO] consul: New leader elected: ip-10-0-3-27
    2018/02/11 07:13:49 [INFO] consul: member 'ip-10-0-3-19' reaped, deregistering
    2018/02/11 07:13:49 [INFO] agent: Synced node info
```

### クライアント側

```
$ consul agent --config-dir=/etc/consul.d
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.0.3'
           Node ID: '473abde2-1a30-f946-a1b0-e73cdb67a1ba'
         Node name: 'ip-10-0-3-19'
        Datacenter: 'aws' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.0.3.19 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2018/02/11 07:14:15 [INFO] serf: EventMemberJoin: ip-10-0-3-19 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2018/02/11 07:14:15 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2018/02/11 07:14:15 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2018/02/11 07:14:15 [INFO] agent: started state syncer
    2018/02/11 07:14:15 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
    2018/02/11 07:14:15 [INFO] agent: Joining LAN cluster...
    2018/02/11 07:14:15 [INFO] discover-aws: Address type  is not supported. Valid values are {private_v4,public_v4,public_v6}. Falling back to 'private_v4'
    2018/02/11 07:14:15 [INFO] discover-aws: Region not provided. Looking up region in metadata...
    2018/02/11 07:14:15 [WARN] manager: No servers available
    2018/02/11 07:14:15 [ERR] agent: failed to sync remote state: No known Consul servers
    2018/02/11 07:14:15 [INFO] discover-aws: Region is ap-northeast-1
    2018/02/11 07:14:15 [INFO] discover-aws: Filter instances with Name=consul
    2018/02/11 07:14:15 [INFO] discover-aws: Instance i-05cbabfc1ec12b005 has private ip 10.0.3.27
    2018/02/11 07:14:15 [INFO] discover-aws: Instance i-01503c36e371ea41a has private ip 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: Discovered LAN servers: 10.0.3.27 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: (LAN) joining: [10.0.3.27 10.0.3.19]
    2018/02/11 07:14:15 [INFO] serf: EventMemberJoin: ip-10-0-3-27 10.0.3.27
    2018/02/11 07:14:15 [INFO] consul: adding server ip-10-0-3-27 (Addr: tcp/10.0.3.27:8300) (DC: aws)
    2018/02/11 07:14:15 [INFO] agent: (LAN) joined: 2 Err: <nil>
    2018/02/11 07:14:15 [INFO] agent: Join LAN completed. Synced with 2 initial agents
    2018/02/11 07:14:17 [INFO] agent: Synced node info
```

## 確認

どちらかのインスタンスで

```
$ consul members
Node          Address         Status  Type    Build  Protocol  DC   Segment
ip-10-0-3-27  10.0.3.27:8301  alive   server  1.0.3  2         aws  <all>
ip-10-0-3-19  10.0.3.19:8301  alive   client  1.0.3  2         aws  <default>
```

以上