Ansibleを使用して、CentOS 7にRedis-Clusterを構築する

# Ansibleを使用して、CentOS 7にRedis-Clusterを構築する

## 作成するに至った経緯
- Redisを手動でインストールするのが面倒なので、自動化したかったため
- Redis-Clusterを構築するために必要な各ディレクトリ/ファイルの作成、コンフィグの編集を手動で行うのが面倒だったため。

## 対象者
- CentOS 7 に、Redisを簡単にインストールしたい方
- CentOS 7 にて、Redis-Cluster構築を簡単に行いたい方

## 前提条件
- Ansibleはインストール済みであるとする
- config、inventoryの設定は完了済みであるとする
- 鍵生成、鍵交換、疎通確認は完了済みであるとする
- 今回は、proxy環境下ではないものとする
- 今回は、一つのサーバーにRedisのMasterとSlaveを配置する

## 実行環境
```
# cat /etc/redhat-release 
CentOS Linux release 7.7.1908 (Core)
# ansible --version
ansible 2.8.5
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Aug  7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
```

## 本playbookで行っていることの概要
- yum updateの実施
- Redisに必要なパッケージのインストール
- Redisのインストール
- ディレクトリの作成
- redis.confファイルのコピー
- configの設定

|項目|設定値|
|:--|:--|
|bind|0.0.0.0|
|port番号|700x|
|cluster-enabled|yes|
|cluster-config-file|/opt/redis/(7000～7005)/nodes.conf
|cluster-node-timeout|5000|
|appendonly|yes|
|logfile| /var/log/redis/redis_700x.log|

- 各Redisサーバーの起動(7000～7005)

## 本playbook実行後に得られるもの
- redisがインストールされる
- Redis-Cluster構築に必要なファイルが配置・設定された状態になる
- Redis-Clusterに使用する各サーバーが起動された状態になる

## 本playbookの実行方法
```
# ansible-playbook SetupRedisCluster.yml
```

## Redis-Clusterを構築するplaybook
```SetupRedisCluster.yml
- name: Setup Redis-Cluster
  hosts: localhost
  vars:
    redis_port_number:
    - 7000
    - 7001
    - 7002
    - 7003
    - 7004
    - 7005
  tasks:
  - name: Upgrade all packages
    yum:
      name: '*'
      state: latest

  - name: Install a list of packages
    yum:
      name:
      - epel-release
      - http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
      state: latest

  - name: Install redis
    yum:
      name: redis
      enablerepo: remi
      state: present

  - name: mkdir redis config directory
    file:
      path: /opt/redis/{{ item }}
      state: directory
      owner: redis
      group: redis
      mode: '0755'
    with_items: "{{ redis_port_number }}"

  - name: Copy redis.conf
    copy:
      src: /etc/redis.conf
      dest: /opt/redis/{{ item }}
      owner: redis
      group: redis
      mode: '0755'
    with_items: "{{ redis_port_number }}"

  - name: Setting bind IPv4Address
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^bind'
      insertafter: '^#bind'
      line: bind 0.0.0.0
    with_items: "{{ redis_port_number }}"

  - name: Setting port number
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^port'
      insertafter: '^#port'
      line: port {{ item }}
    with_items: "{{ redis_port_number }}"

  - name: Setting cluster-enabled
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^cluster-enabled'
      insertafter: '^#cluster-enabled'
      line: cluster-enabled yes
    with_items: "{{ redis_port_number }}"

  - name: Setting cluster-config-file
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^cluster-config-file'
      insertafter: '^#cluster-config-file'
      line: cluster-config-file /opt/redis/{{ item }}/nodes.conf
    with_items: "{{ redis_port_number }}"

  - name: Setting cluster-node-timeout
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^cluster-node-timeout'
      insertafter: '^#cluster-node-timeout'
      line: cluster-node-timeout 5000
    with_items: "{{ redis_port_number }}"

  - name: Setting appendonly
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^appendonly'
      insertafter: '^#appendonly'
      line: appendonly yes
    with_items: "{{ redis_port_number }}"

  - name: Setting logfile
    lineinfile:
      path: /opt/redis/{{ item }}/redis.conf
      mode: '0755'
      regexp: '^logfile /var/log/'
      insertafter: '^#logfile /var/log/'
      line: logfile /var/log/redis/redis_{{ item }}.log
    with_items: "{{ redis_port_number }}"

  - name: Start Redis-Server
    shell: redis-server /opt/redis/{{ item }}/redis.conf &
    with_items: "{{ redis_port_number }}"

```

## playbook実行時のログ
```
# ansible-playbook SetupRedisCluster.yml

PLAY [Setup Redis-Cluster] ****************************************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Upgrade all packages] ***************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Install a list of packages] *********************************************************************************************************************************************************************************
ok: [localhost]

TASK [Install redis] **********************************************************************************************************************************************************************************************
changed: [localhost]

TASK [mkdir redis config directory] *******************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Copy redis.conf] ********************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting bind IPv4Address] ***********************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting port number] ****************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting cluster-enabled] ************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting cluster-config-file] ********************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting cluster-node-timeout] *******************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting appendonly] *****************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Setting logfile] ********************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

TASK [Start Redis-Server] *****************************************************************************************************************************************************************************************
changed: [localhost] => (item=7000)
changed: [localhost] => (item=7001)
changed: [localhost] => (item=7002)
changed: [localhost] => (item=7003)
changed: [localhost] => (item=7004)
changed: [localhost] => (item=7005)

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=14   changed=11   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0       
```
playbook実行により、Redis-Clusterを構築する各サーバーの起動まで完了いたしました。

## Redisのバージョン確認
まずは、インストールされたRedisのバージョンを確認してみましょう。

```
# redis-cli --version
redis-cli 5.0.6
# redis-server --version
Redis server v=5.0.6 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=c3d7ebb6b1a2844b
```

## Redis-Clusterの構築
それでは、下記のコマンドを叩いて、Redis-Clusterを構築しましょう。

```
# redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 68736e1b3bc1824aad3e27306fcecf2b50c70cd2 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
M: 0d8522a13d99dd0442b6231ee08403f4553ed497 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
M: dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
S: 5c5a2ad352471ca0e4920b17621c69830ea52893 127.0.0.1:7003
   replicates 68736e1b3bc1824aad3e27306fcecf2b50c70cd2
S: 8b0cfa1aa3918c57e88450b1aafc9183125cda53 127.0.0.1:7004
   replicates 0d8522a13d99dd0442b6231ee08403f4553ed497
S: cab9ff577d34cabf773ce185c214a3557d8ad884 127.0.0.1:7005
   replicates dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b
```

下記の表示が出てきたら、"yes"と入力し、Enterを押下します。

```
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 68736e1b3bc1824aad3e27306fcecf2b50c70cd2 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 0d8522a13d99dd0442b6231ee08403f4553ed497 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 8b0cfa1aa3918c57e88450b1aafc9183125cda53 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 0d8522a13d99dd0442b6231ee08403f4553ed497
M: dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: cab9ff577d34cabf773ce185c214a3557d8ad884 127.0.0.1:7005
   slots: (0 slots) slave
   replicates dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b
S: 5c5a2ad352471ca0e4920b17621c69830ea52893 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 68736e1b3bc1824aad3e27306fcecf2b50c70cd2
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
```

## Redis-Clusterの状態確認
構築したRedis-Clusterの状態を確認してみましょう。

```
# redis-cli -p 7000 cluster nodes
redis-cli -p 7000 cluster nodes
0d8522a13d99dd0442b6231ee08403f4553ed497 127.0.0.1:7001@17001 master - 0 1570302275000 2 connected 5461-10922
8b0cfa1aa3918c57e88450b1aafc9183125cda53 127.0.0.1:7004@17004 slave 0d8522a13d99dd0442b6231ee08403f4553ed497 0 1570302274505 5 connected
dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b 127.0.0.1:7002@17002 master - 0 1570302275815 3 connected 10923-16383
cab9ff577d34cabf773ce185c214a3557d8ad884 127.0.0.1:7005@17005 slave dc7e9921a8e0e8b36f2a80ac819dc9a07f9c492b 0 1570302275000 6 connected
68736e1b3bc1824aad3e27306fcecf2b50c70cd2 127.0.0.1:7000@17000 myself,master - 0 1570302275000 1 connected 0-5460
5c5a2ad352471ca0e4920b17621c69830ea52893 127.0.0.1:7003@17003 slave 68736e1b3bc1824aad3e27306fcecf2b50c70cd2 0 1570302275000 4 connected
```

##まとめ
Ansibleを使用して、CentOS 7にRedis-Clusterの構築することができました。
今回は自宅環境だったため、1つのサーバーにMasterとSlaveを組んでしまいました。実際の運用を想定すると、別々のサーバーでRedis-Clusterを構築できるかを試す必要があるかなと思います。そのときは下記のようにplaybookと、Redis-Clusterを構築するときのコマンドの内容を変更する必要あるかと思います。

```
- name: Setup Redis-Cluster
  hosts: redis_node    ## inventoryに2つのサーバーのIPを記述しておく
  vars:
    redis_port_number: ## port番号を3にする
    - 7000
    - 7001
    - 7002
```

```
# redis-cli --cluster create <redis-master.ipaddr>:7000 <redis-master.ipaddr>:7001 <redis-master.ipaddr>:7002 <redis-slave.ipaddr>:7000 <redis-slave.ipaddr>:7001 <redis-slave.ipaddr>:7002 --cluster-replicas 1
```

あとはsystemdで自動起動の設定もplaybookに反映できればよいかと思います。ただ、今回のplaybookを作成したことで、hostsとredis_port_numberの値を変えるだけで容易にRedis-Clusterが組めるようになったのは大きいかなと思います。

## 参考URL
#### 今回使用したモジュール
- [【docs.ansible.com】yum – Manages packages with the yum package manager](https://docs.ansible.com/ansible/latest/modules/yum_module.html?highlight=yum)
- [【docs.ansible.com】file – Manage files and file properties](https://docs.ansible.com/ansible/latest/modules/file_module.html?highlight=file)
- [【docs.ansible.com】copy – Copy files to remote locations](https://docs.ansible.com/ansible/latest/modules/copy_module.html?highlight=copy)
- [【docs.ansible.com】lineinfile – Manage lines in text files](https://docs.ansible.com/ansible/latest/modules/lineinfile_module.html?highlight=lineinfile)
- [【docs.ansible.com】shell – Execute shell commands on targets](https://docs.ansible.com/ansible/latest/modules/shell_module.html?highlight=shell)

#### Redis-Cluster関連
- [Redis cluster tutorial](https://redis.io/topics/cluster-tutorial)
- [CentOS 7でRedisを複数立ち上げる](https://qiita.com/tomgoodsun/items/670602251d47f1f71a3d)
- [Redis 5でのCluster構築](https://qiita.com/teekee/items/d4c630fff95349573403)
- [Redis 5.0でクラスタ（Redis Cluster）を構成してみる](https://kazuhira-r.hatenablog.com/entry/2018/11/25/234444)
- [Redis 5.0 を用いたクラスタ構築と検証](https://shirobrak.hatenablog.com/entry/2018/10/28/205733)