DRBD9をインストール (drbdmanage add-nodeまで。)

## ビルド済みパッケージからインストール

* [第3章 コンパイル済みDRBDバイナリパッケージのインストール](https://blog.3ware.co.jp/drbd-users-guide-9.0/ch-install-packages.html)
    * ubuntu16.04以降、debian8用

```bash:/etc/apt/sources.list.d/drbd9.list
deb http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu xenial main
#deb http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu trusty main
#deb http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu devel main
```

```bash:
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 34893610CEAA9512
apt update
apt-get install -y drbd-utils python-drbdmanage drbd-dkms
```

* /lib/modules/4.8.0-39-generic/updates/dkms/
    * drbd.ko
    * drbd_transport_tcp.ko

### Debian8ではtrustyのリポジトリを使う。

xenialのリポジトリだと以下エラーになる。

> drbd-utils : Depends: libstdc++6 (>= 5.2) but 4.9.2-10 is to be installed

```
echo "deb http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu trusty main" > /etc/apt/sources.list.d/drbd9.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 34893610CEAA9512
apt update
apt-get install -y drbd-utils python-drbdmanage drbd-dkms
```

## ソースからインストール

* [DRBD9環境をdrbdmanageを使って簡単構築 #1(インストール編) - ブログ名はまだ無い(仮)](http://ksawada.hatenablog.com/entry/2016/06/10/111030)
* ubuntu 16.10

```bash:
apt install -y "build-essential git checkinstall"
#hostname build
```

```bash:drbd-9.0
git clone "git://git.drbd.org/drbd-9.0.git"
cd drbd-9.0
#export KDIR="/lib/modules/4.8.0-38-generic/build"
make -s
make install
checkinstall -y -D --pkgname="drbd-9.0"
```

* `drbd-9.0_9.0-1_amd64.deb`が出来上がる

```bash:drbd-utils
apt install -y "autoconf pkg-config flex xsltproc"
git clone "git://git.drbd.org/drbd-utils.git"
cd drbd-utils
./autogen.sh
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc

#時間が20分はかかる
make -s
make install
checkinstall -y -D --pkgname="drbd-utils"
```

```bash:drbdmanage
git clone "git://git.drbd.org/drbdmanage.git"
cd drbdmanage
make -s
make install
checkinstall -y -D --pkgname="drbdmanage"
```

----

## 準備 : CloudAtCostの場合

CloudAtCost の場合、 Debian8で15GBのDISKを選ぶと以下構成になる。

```diff
  Device     Boot    Start      End  Sectors  Size Id Type
  /dev/sda1  *        2048   499711   497664  243M 83 Linux
  /dev/sda2         501758 20969471 20467714  9.8G  5 Extended
+ /dev/sda3       20969472 30719999  9750528  4.7G 83 Linux
  /dev/sda5         501760 20969471 20467712  9.8G 8e Linux LVM
```

* /dev/sda5 が 10GBで作成、
* /dev/sda3 に 残りのDISK容量割当てられる。
    * このsda3を領域として使う。

----

Debian8(jessie) にて確認。trustyリポジトリでインストールしておく。

| ホスト名 | ip |役割|
|:--|:--|:--|
| drbd1 | xxx.xxx.xxx.101 |master|
| drbd2 | xxx.xxx.xxx.102 |slave|
| drbd3 | xxx.xxx.xxx.103 |slave|

### drbdmanage init

drbdpool という Volume group が必要。

```diff:確認
  $ drbdmanage init
  
  You are going to initialize a new drbdmanage cluster.
  CAUTION! Note that:
    * Any previous drbdmanage cluster information may be removed
    * Any remaining resources managed by a previous drbdmanage installation
      that still exist on this system will no longer be managed by drbdmanage
  
  Confirm:
  
    yes/no: yes
  Error: External command failed:
  lvcreate -n .drbdctrl_0 -L 4m drbdpool
  Command output:
+   (stderr)   Volume group "drbdpool" not found
  Command exited with exit_code 5
  
  Initialization failed
```

```bash:drbdpool作成
vgcreate drbdpool /dev/sda3
```

```bash:drbdpool作成やり直したい場合
# dd if=/dev/zero of=/dev/sda3 bs=1M count=100
# reboot
```

```rb:init
$ drbdmanage init  # 1台目でのみ実施

You are going to initialize a new drbdmanage cluster.
CAUTION! Note that:
  * Any previous drbdmanage cluster information may be removed
  * Any remaining resources managed by a previous drbdmanage installation
    that still exist on this system will no longer be managed by drbdmanage

Confirm:

  yes/no: yes
Empty drbdmanage control volume initialized on '/dev/drbd0'.
Empty drbdmanage control volume initialized on '/dev/drbd1'.
Waiting for server: .
Operation completed successfully
```

```
# 間違って2台目、3台目でも実行してしまった場合は、以下コマンドで戻せる。
# drbdmanage uninit
```

### 確認

```rb:確認
$ drbdadm status
.drbdctrl role:Primary
  volume:0 disk:UpToDate
  volume:1 disk:UpToDate
```

* /etc/drbd.d/drbdctrl.res が自動で作成された

```nginx:/etc/drbd.d/drbdctrl.res
resource .drbdctrl {
    net {
        cram-hmac-alg       sha256;
        shared-secret       "xxxxxxxxxxxxxxxxxxxx";
        allow-two-primaries no;
    }
    volume 0 {
        device      minor 0;
        disk        /dev/drbdpool/.drbdctrl_0;
        meta-disk   internal;
    }
    volume 1 {
        device      minor 1;
        disk        /dev/drbdpool/.drbdctrl_1;
        meta-disk   internal;
    }
    on drbd1 {
        node-id     0;
        address     ipv4 xxx.xxx.xxx.101:6999;
    }
    connection-mesh {
        hosts drbd1;
        net {
            protocol C;
        }
    }
}
```

```rb:poolサイズ(4752MBの未使用領域)
$ drbdmanage list-nodes
+--------------------------------------------------------------+
| Name      | Pool Size | Pool Free |                  | State |
|--------------------------------------------------------------|
| localhost |      4760 |      4752 |                  |    ok |
+--------------------------------------------------------------+
```


## add nodes

```
drbdmanage add-node drbd2 xxx.xxx.xxx.102
drbdmanage add-node drbd3 xxx.xxx.xxx.103
```


```bash:確認
$ drbdmanage list-nodes
+----------------------------------------------------------------+
| Name  | Pool Size | Pool Free |                        | State |
|----------------------------------------------------------------|
| drbd1 |      4760 |      4752 |                        |    ok |
| drbd2 |      4760 |      4752 |                        |    ok |
| drbd3 |      4760 |      4752 |                        |    ok |
+----------------------------------------------------------------+
```

```bash:status
$ drbdadm status
.drbdctrl role:Primary
  volume:0 disk:UpToDate
  volume:1 disk:UpToDate
  drbd2 role:Secondary
    volume:0 peer-disk:UpToDate
    volume:1 peer-disk:UpToDate
  drbd3 role:Secondary
    volume:0 peer-disk:UpToDate
    volume:1 peer-disk:UpToDate
```


```diff:/etc/drbd.d/drbdctrl.res
  resource .drbdctrl {
      net {
          cram-hmac-alg       sha256;
          shared-secret       "xxxxxxxxxxxxxxxxxxxx";
          allow-two-primaries no;
      }
      volume 0 {
          device      minor 0;
          disk        /dev/drbdpool/.drbdctrl_0;
          meta-disk   internal;
      }
      volume 1 {
          device      minor 1;
          disk        /dev/drbdpool/.drbdctrl_1;
          meta-disk   internal;
      }
      on drbd1 {
          node-id     0;
          address     ipv4 xxx.xxx.xxx.101:6999;
      }
+     on drbd3 {
+         node-id     2;
+         address     ipv4 xxx.xxx.xxx.102:6999;
+     }
+     on drbd2 {
+         node-id     1;
+         address     ipv4 xxx.xxx.xxx.103:6999;
+     }
      connection-mesh {
+         hosts drbd1 drbd3 drbd2;
          net {
              protocol C;
          }
      }
  }
```

## ノードの削除

```rb:エラーの場合
$ drbdmanage list-nodes
+------------------------------------------------------------------------------------------------------------+
| Name  | Pool Size | Pool Free |         |                                                            State |
|------------------------------------------------------------------------------------------------------------|
| drbd1 |      4760 |      4752 |         |                                                               ok |
| drbd2 |   unknown |   unknown |         | offline/quorum vote ignored, pending actions: adjust connections |
| drbd3 |   unknown |   unknown |         | offline/quorum vote ignored, pending actions: adjust connections |
+------------------------------------------------------------------------------------------------------------+
```

```bash:複数同時に書ける
drbdmanage delete-node drbd2 drbd3
```

## リソース登録

```bash:master機でのみ実施
drbdmanage add-resource r0
# 削除は drbdmanage delete-resource r0
```

```bash:
$ drbdmanage list-volumes --groupby Size --show Port
+------------------------------------------------------------------+
| Name | Vol ID | Size | Minor | Port |                    | State |
|------------------------------------------------------------------|
| r0   |      * |    * |     * | 7000 |                    |     * |
+------------------------------------------------------------------+
```

```bash:r0に4Gのvolume作成
drbdmanage add-volume r0 4
```

### 削除

```rb:指定を間違えた場合
# drbdmanage list-volumes --groupby Size --show Port
+--------------------------------------------------------------+
| Name | Vol ID |     Size | Minor | Port |            | State |
|--------------------------------------------------------------|
| r0   |      1 |    4 GiB |   101 | 7000 |            |    ok |
| r0   |      0 | 4.65 TiB |   100 | 7000 |            |    ok |
+--------------------------------------------------------------+

root@drbd1:~# drbdmanage delete-volume r0 
usage: drbdmanage remove-volume [-h] [-q] [-f] name vol_id
drbdmanage remove-volume: error: too few arguments

root@drbd1:~# drbdmanage delete-volume r0 0
You are going to remove a volume from all nodes of the cluster.
Please confirm:
  yes/no: yes
Operation completed successfully

root@drbd1:~# drbdmanage delete-volume r0 1
You are going to remove a volume from all nodes of the cluster.
Please confirm:
  yes/no: yes
Operation completed successfully
```

## デプロイ

```bash:3台に配布
drbdmanage deploy-resource r0 3
```

## 失敗事例

```bash:
mkfs.ext4 /dev/drbd0
```

> root@drbd1:~# drbdmanage add-resource r0
> Error: I/O error while accessing persistent configuration storage

```
root@drbd1:~# drbdmanage list-volumes --groupby Size --show Port
+-----------------------------------------------------------------+
| Name | Vol ID | Size | Minor | Port |                   | State |
|-----------------------------------------------------------------|
| r0   |      * |    * |     * | 7000 |                   |     * |
+-----------------------------------------------------------------+
```

やり直し。