【2024年1月版】Ubuntu22.04、Ubuntu24.04 でVIPふってクラスタリング [pacemaker, corosync, pcs]

# はじめに

- Ubuntu で VIP ふってクラスタリングしたいな、そうだ pacemaker + corosync しよう
- HAクラスタはpacemaker標準となった pcs でコントロール
- CRM関連コマンド(crm_xxx)、cibadminコマンド、crmコマンドは使わない

:::note info
2024年1月 Ubuntu22.04の記事でしたが、24.04でも同様に実行して動作することを確認しました。
:::

# 本記事の流れ

- とりあえず構築
- pacemaker + corosync で知っておくべきことを説明

# 環境

- ノードは２つ
  - ノード1: ha01(192.168.100.240)
  - ノード2: ha02(192.168.100.241)
- VIPは 192.168.100.245 を設定
- クラスター名は testcluster を設定
- STONITHは利用しない
  - 不安定ノードの強制停止／再起動はしない
- 2017年頃に利用していた pcs から、バージョンが新しいため、いくつかのコマンドが変わっていたので注意

```terminal
$ uname -a
Linux ha01 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:56:13 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
```

```terminal
$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
```


# インストール

- ノード ha01, ha02 の双方で行う
- パッケージのインストールとユーザ作成、サービス化

```terminal
$ sudo apt install -y corosync pacemaker pcs
$ sudo passwd hacluster
New password: 
Retype new password: 
$ sudo systemctl enable --now pcsd
```
```terminal
ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5
```
```terminal
ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5
```

- hosts を設定
```terminal
sudo vim /etc/hosts
```

- hosts の確認
  - hosts編集の際、自ホスト名が127.0.0.1に存在しないように注意
```terminal
$ sudo cat /etc/hosts
127.0.0.1 localhost

192.168.100.240 ha01
192.168.100.241 ha02
```

# クラスターの設定

- ha01 で操作

```terminal
ha01:~$ sudo pcs host auth ha01 ha02 -u hacluster
Password: 
ha01: Authorized
ha02: Authorized

ha01:~$ sudo pcs cluster setup testcluster ha01 ha02
No addresses specified for host 'ha01', using 'ha01'
No addresses specified for host 'ha02', using 'ha02'
Destroying cluster on hosts: 'ha01', 'ha02'...
ha01: Successfully destroyed cluster
ha02: Successfully destroyed cluster
Requesting remove 'pcsd settings' from 'ha01', 'ha02'
ha01: successful removal of the file 'pcsd settings'
ha02: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to 'ha01', 'ha02'
ha01: successful distribution of the file 'corosync authkey'
ha01: successful distribution of the file 'pacemaker authkey'
ha02: successful distribution of the file 'corosync authkey'
ha02: successful distribution of the file 'pacemaker authkey'
Sending 'corosync.conf' to 'ha01', 'ha02'
ha01: successful distribution of the file 'corosync.conf'
ha02: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.

ha01:~$ sudo pcs cluster start --all
ha02: Starting Cluster...
ha01: Starting Cluster...

ha01:~$ sudo pcs property set stonith-enabled=false
ha01:~$ sudo pcs property set no-quorum-policy=ignore


ha01:~$ sudo pcs resource create VIP ocf:heartbeat:IPaddr2 ip=192.168.100.245 cidr_netmask=24 op monitor interval=10s

ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha01 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:09:31 2023
  * Last change:  Thu Jan  5 01:00:30 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5
    inet 192.168.100.245/24 brd 192.168.100.255 scope global secondary enp0s5
```

:::note info
以下のエラーが発生して、testclusterが作成できない場合は、`--force` をつけるとうまくいきます。この症状は、Ubuntu24.04で発生しました。強制的に現在存在しているクラスターを破棄するので注意が必要です。

↓ うまくいかない時
```
$ sudo pcs cluster setup testcluster ha01 ha02
No addresses specified for host 'ha01', using 'ha01'
No addresses specified for host 'ha02', using 'ha02'
Error: ha02: The host seems to be in a cluster already as the following services are found to be running: 'corosync', 'pacemaker'. If the host is not part of a cluster, stop the services and retry, use --force to override
Error: ha02: The host seems to be in a cluster already as cluster configuration files have been found on the host. If the host is not part of a cluster, run 'pcs cluster destroy' on host 'ha02' to remove those configuration files, use --force to override
Error: ha01: The host seems to be in a cluster already as the following services are found to be running: 'corosync', 'pacemaker'. If the host is not part of a cluster, stop the services and retry, use --force to override
Error: ha01: The host seems to be in a cluster already as cluster configuration files have been found on the host. If the host is not part of a cluster, run 'pcs cluster destroy' on host 'ha01' to remove those configuration files, use --force to override
Error: Some nodes are already in a cluster. Enforcing this will destroy existing cluster on those nodes. You should remove the nodes from their clusters instead to keep the clusters working properly, use --force to override
Error: Errors have occurred, therefore pcs is unable to continue
```

↓ `--force` でうまくいく
```
ha01:~$ sudo pcs cluster setup --force testcluster ha01 ha02
No addresses specified for host 'ha01', using 'ha01'
No addresses specified for host 'ha02', using 'ha02'
Warning: ha02: The host seems to be in a cluster already as the following services are found to be running: 'corosync', 'pacemaker'. If the host is not part of a cluster, stop the services and retry
Warning: ha02: The host seems to be in a cluster already as cluster configuration files have been found on the host. If the host is not part of a cluster, run 'pcs cluster destroy' on host 'ha02' to remove those configuration files
Destroying cluster on hosts: 'ha01', 'ha02'...
ha02: Successfully destroyed cluster
ha01: Successfully destroyed cluster
Requesting remove 'pcsd settings' from 'ha01', 'ha02'
ha02: successful removal of the file 'pcsd settings'
ha01: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to 'ha01', 'ha02'
ha02: successful distribution of the file 'corosync authkey'
ha02: successful distribution of the file 'pacemaker authkey'
ha01: successful distribution of the file 'corosync authkey'
ha01: successful distribution of the file 'pacemaker authkey'
Sending 'corosync.conf' to 'ha01', 'ha02'
ha02: successful distribution of the file 'corosync.conf'
ha01: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.
```
:::

- ha02 で操作

```terminal
ha02:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha01 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:01:54 2023
  * Last change:  Thu Jan  5 01:00:30 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5
```

# クラスターのステータス確認
```terminal
$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha01 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:09:31 2023
  * Last change:  Thu Jan  5 01:00:30 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
```

# 指定ノードをオンラインからスタンバイに設定

- ha01 をスタンバイに
- ha01 がオンラインからスタンバイに変更される
- ha02 に VIP がふられる

- ha01 で操作（VIP 192.168.100.245 がふられている）
```terminal
ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:37:27 2023
  * Last change:  Thu Jan  5 01:37:23 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
```

- スタンバイに設定
```terminal
ha01:~$ sudo pcs node standby ha01
```

- VIP が変わっていることを確認
```terminal
ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:38:05 2023
  * Last change:  Thu Jan  5 01:38:01 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Node ha01: standby
  * Online: [ ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5
```

- ha02 で操作（VIP 192.168.100.245 がふられている）
```terminal
ha02:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:40:48 2023
  * Last change:  Thu Jan  5 01:38:01 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Node ha01: standby
  * Online: [ ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5
    inet 192.168.100.245/24 brd 192.168.100.255 scope global secondary enp0s5
```

# 指定ノードをスタンバイからオンラインに設定

- VIP は変更されないことを確認

```terminal
ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:20:20 2023
  * Last change:  Thu Jan  5 02:20:15 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Node ha01: standby
  * Online: [ ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha01:~$ sudo pcs node unstandby ha01

ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:21:14 2023
  * Last change:  Thu Jan  5 02:21:06 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
```

# VIP のつけかえ

- ha02 に VIP がふられている状態から ha01 に VIP をふりかえ

- VIP を確認
```terminal
ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5
    inet 192.168.100.245/24 brd 192.168.100.255 scope global secondary enp0s5

ha02:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:42:01 2023
  * Last change:  Thu Jan  5 02:32:05 2023 by root via crm_resource on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5

ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:44:24 2023
  * Last change:  Thu Jan  5 02:32:05 2023 by root via crm_resource on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
```

- ha01 に VIP をふりかえ
```terminal
$ sudo pcs resource move VIP ha01
```

- VIP を確認
```terminal
ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5
    inet 192.168.100.245/24 brd 192.168.100.255 scope global secondary enp0s5

ha01:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:47:44 2023
  * Last change:  Thu Jan  5 02:44:48 2023 by root via crm_resource on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5

ha02:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 02:48:08 2023
  * Last change:  Thu Jan  5 02:44:48 2023 by root via crm_resource on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha01 ha02 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha01

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled
```

# 指定クラスターの停止

- ha01 のクラスターを停止
- ha01 がオンラインからオフラインに
- ha02 に VIP がふられる
- ha01 のクラスターはステータスも確認できない状態になる

```terminal
ha01:~$ sudo pcs cluster stop ha01
ha01: Stopping Cluster (pacemaker)...
ha01: Stopping Cluster (corosync)...```
```

- VIP が変わっていることを確認

- ha01 で操作（VIP 192.168.100.245 がはずれている）
```terminal
ha01:~$ sudo pcs status
Error: error running crm_mon, is pacemaker running?
  crm_mon: Error: cluster is not available on this node

ha01:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.240/24 brd 192.168.100.255 scope global enp0s5
```

- ha02 で操作（VIP 192.168.100.245 がふられている）
```terminal
ha02:~$ sudo pcs status
Cluster name: testcluster
Cluster Summary:
  * Stack: corosync
  * Current DC: ha02 (version 2.1.2-ada5c3b36e2) - partition with quorum
  * Last updated: Thu Jan  5 01:17:39 2023
  * Last change:  Thu Jan  5 01:00:30 2023 by root via cibadmin on ha01
  * 2 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ ha02 ]
  * OFFLINE: [ ha01 ]

Full List of Resources:
  * VIP	(ocf:heartbeat:IPaddr2):	 Started ha02

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

ha02:~$ ip a | grep inet
    inet 127.0.0.1/8 scope host lo
    inet 192.168.100.241/24 brd 192.168.100.255 scope global enp0s5
    inet 192.168.100.245/24 brd 192.168.100.255 scope global secondary enp0s5
```

# クラスター全体の停止
```terminal
$ sudo pcs cluster stop --all
ha02: Stopping Cluster (pacemaker)...
ha01: Stopping Cluster (pacemaker)...
ha01: Stopping Cluster (corosync)...
ha02: Stopping Cluster (corosync)...
```

# 指定クラスターの起動
```terminal
$ sudo pcs cluster start ha01
ha01: Starting Cluster...
```

# クラスターの削除
```terminal
$ sudo pcs cluster destroy --all
Warning: Unable to load CIB to get guest and remote nodes from it, those nodes will not be deconfigured.
ha02: Stopping Cluster (pacemaker)...
ha01: Stopping Cluster (pacemaker)...
ha01: Successfully destroyed cluster
ha02: Successfully destroyed cluster
```

# firewall で開放が必要なポート

- ufw でポート指定する場合
  - 2224/tcp
  - 3121/tcp
  - 5403/tcp
  - 5404/udp
  - 5405/udp
  - 21064/tcp
  - 9929/tcp
  - 9929/udp

- firewall-cmd で指定する場合
  - firewall-cmd --add-service=high-availability
  - firewall-cmd --runtime-to-permanent

# pcs コマンド解説

編集中...

# 参考資料

※ 全体的に pacemaker のバージョンが古いのでコマンドが違う

- [試して覚える Pacemaker-2.0入門 『シェアードナッシングな高可用クラスタの実現』](http://linux-ha.osdn.jp/wp/wp-content/uploads/LinuxHAJapan-OSC2021OnlineSpring-20210305.pdf)
- [試して覚える Pacemaker-2.0入門 『構築・リソース設定』](http://linux-ha.osdn.jp/wp/wp-content/uploads/LinuxHAJapan-OSC2020OnlineFall-20201023.pdf)
- [Pacemakerで学ぶ HAクラスタ](http://linux-ha.osdn.jp/wp/wp-content/uploads/OSC2019_Tokyo-Spring.pdf)
- [Pacemakerと HAProxyではじめる 高可用ロードバランサ入門](http://linux-ha.osdn.jp/wp/wp-content/uploads/LinuxHAJapan-OSC2021OnlineFall-20211022.pdf)
- [試して覚えるPacemaker入門 排他制御編](http://linux-ha.osdn.jp/wp/wp-content/uploads/076783ca53a363270d253bbb98b59e83.pdf)
- [試して覚えるPacemaker入門 PG-REX(Pacemaker + PostgreSQLによるシェアードナッシングHA構成)構築(PDF)](http://linux-ha.osdn.jp/wp/wp-content/uploads/20170805-OSC-Kyoto-2017.pdf)
- [試して覚えるPacemaker入門　PG-REX(Pacemaker + PostgreSQLによるシェアードナッシングHA構成)運用(PDF)](http://linux-ha.osdn.jp/wp/wp-content/uploads/OSC2018_Osaka.pdf)
- [高信頼システム構築標準教科書 － 仮想化と高可用性 －](https://linuc.org/textbooks/system/)
- [徹底攻略LPIC Level3 304教科書＋問題集［Version 2.0］対応 徹底攻略シリーズ](https://www.amazon.co.jp/dp/B01EKXKZ4S)
- [LPI 304 Virtualization & High Availability Practice Exam](https://www.amazon.co.jp/dp/B00V0TLQCG)
- [HAクラスタをDRBDとPacemakerで作ってみよう [Pacemaker編]](https://knowledge.sakura.ad.jp/19690/)
- [最強WEB問題集LinuC Lv3-304(Ver2.0)](https://ping-t.com/modules/bulletin/index.php?page=article&storyid=702)
- [オライリー サーバ負荷分散技術](https://www.amazon.co.jp/dp/4873110653)
- [［24時間365日］サーバ/インフラを支える技術 ……スケーラビリティ、ハイパフォーマンス、省力運用 WEB+DB PRESS plus](https://www.amazon.co.jp/dp/B07JHRVXGT)
- [DRBD Tech Info - 使っておぼえるシリーズ](https://blog.drbd.jp/category/%e4%bd%bf%e3%81%a3%e3%81%a6%e3%81%8a%e3%81%bc%e3%81%88%e3%82%8b%e3%82%b7%e3%83%aa%e3%83%bc%e3%82%ba/)
- [DRBD Tech Info - ドキュメント](https://blog.drbd.jp/%e3%83%89%e3%82%ad%e3%83%a5%e3%83%a1%e3%83%b3%e3%83%88/)
- [LINUX-HA JAPAN](http://linux-ha.osdn.jp/wp/)
- [LINUX-HA JAPAN - Pacemakerの概要](http://linux-ha.osdn.jp/wp/manual/pacemaker_outline)
- [LINUX-HA JAPAN - マニュアル](http://linux-ha.osdn.jp/wp/manual)
- [Heartbeat＋Pacemaker＋DRBDで高可用Linux 体験！ 新しくなったLinux-HA](https://atmarkit.itmedia.co.jp/ait/articles/1009/08/news112.html)
- [VirtualBox と Rocky Linux 8 で始める Pacemaker ～ VirtualBox でも STONITH 機能が試せる! VirtualBMCの活用](http://linux-ha.osdn.jp/wp/wp-content/uploads/LinuxHAJapan-OSC2022OnlineFall-20221028.pdf)
- [Red Hat Enterprise Linux8 > 高可用性クラスターの設定および管理 > 第2章 Pacemaker の使用の開始](https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_getting-started-with-pacemaker-configuring-and-managing-high-availability-clusters)
- [Red Hat Enterprise Linux 8 高可用性クラスターの設定および管理](https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/8/pdf/configuring_and_managing_high_availability_clusters/red_hat_enterprise_linux-8-configuring_and_managing_high_availability_clusters-ja-jp.pdf)
- [【入門】 PostgreSQL＋Pacemaker＋DRBDで 高可用性構成を構築してみよう](https://www.sraoss.co.jp/wp-content/uploads/files/event_seminar/material/2014/20140704_osc2014_nagoya.pdf)
- [Oracle Linux 8 - 高可用性クラスタリングの設定](https://docs.oracle.com/cd/F22978_01/availability/index.html)
- [MIRACLE Linux - DRBD、Heartbeat、Pacemaker による Zabbix サーバの HA クラスタ構築](file:///Users/dev/Downloads/zabbix_and_linux-ha.pdf)
- [インフラ構築手順書 - PaceMaker+DRBD構築](http://infra.blog.shinobi.jp/Entry/45/)
- [PacemakerのMaster/Slave構成の基本と事例紹介(DRBD、PostgreSQLレプリケーション)](https://www.slideshare.net/tatsuyaw/pacemaker-osc2014tokyo-31882542)

# さいごに

- かんたんでしたね