Db2 for OpenShift and K8s 起動・停止

Kubernetes版Db2の起動・停止手順は2パターンあります。

 * １：Db2プロセスの起動・停止
 * ２：Db2 Pod の起動・停止（レプリカ数増減）

上記「1」はマニュアルに独立した解説ページがありますが、「2」はバックアップリストアの一手順として記述されるのみで、独立して記載されないため、実環境で試した上でメモとして残したいと思います。

## 参考情報(マニュアルリンク)

#### １：Db2プロセスの起動・停止
 * [Db2マニュアル「Stopping and starting a Db2 instance」](https://www.ibm.com/docs/en/db2/11.5?topic=db2-stopping-starting-instance)
     * プロセス停止・起動前後で、build-in-HAソフト操作が必要

#### ２：Db2 Pod の起動・停止（レプリカ数増減）

ストレージSnapshotのリストア手順中に記述されます。
 * [Db2マニュアル「Performing a snapshot restore with Db2 container commands - Db2uInstance」](https://www.ibm.com/docs/en/db2/11.5?topic=restores-using-container-commands-db2uinstance)


## 環境
 * Red Hat OpenShift on IBM Cloud (ROKS)
 * Db2 for OpenShift and k8s(Db2uInstance) 11.5.9
     * 構成/Deploy手順 -> [Link]()


## Db2 Pod 起動・停止手順
Db2 Podはデプロイすると起動状態で作成されるため、停止→起動の順に記述します

#### Db2 Pod停止

##### 0. 停止前の状態確認

Db2uInstanceの名称を変数にセットしておく

```text:マニュアル記載コマンド
DB2U_INSTANCE=$(oc get db2uinstance --no-headers | head -n1 | awk '{print $1}'); echo $DB2U_INSTANCE
```

```text:操作対象CRインスタンス名がわかっている場合の例
DB2U_INSTANCE=db2uinstance-3
```

インスタンスの状態確認
```
oc get db2uinstance $DB2U_INSTANCE
```

oc get all出力を確認
 - 操作対象のDb2u/etcd/ldapのPodの状態を確認
     - Status:Running, Ready: 1/1 であること

```
oc get all
```


##### 1. Db2uInstanceをメンテナンスモードにする

```
oc annotate db2uinstance $DB2U_INSTANCE db2u.databases.ibm.com/maintenance-pause-reconcile=true --overwrite
```


##### 2. スケールダウン(Pod停止)
レプリカ数を事前に確認しておく
```
NUM_REPLICAS=$(oc get db2uengine ${DB2U_ENGINE} -ojsonpath={.spec.replicas}); echo $NUM_REPLICAS
```

Db2エンジン名をセット
```
DB2U_ENGINE=$(oc get db2uengine --selector="app=${DB2U_INSTANCE},type=engine" --no-headers | awk '{print $1}'); echo $DB2U_ENGINE
```

ETCDのStatefulSet名をセット
```
ETCD_STS=$(oc get sts --selector="app=${DB2U_INSTANCE},component=etcd" --no-headers | awk '{print $1}'); echo $ETCD_STS
```

LDAPのStatefulSet名をセット
```
LDAP_DEPLOY=$(oc get deployment --selector="app=${DB2U_INSTANCE},role=ldap" --no-headers | awk '{print $1}'); echo $LDAP_DEPLOY
```

Db2エンジンのレプリカ数を0に変更(→Db2停止)
```
oc patch db2uengine ${DB2U_ENGINE} --type merge --patch '{"spec": {"replicas": 0} }'
```

ETCDのレプリカ数を0に変更(→Etcd停止)
```
oc scale sts ${ETCD_STS} --replicas=0
```

LDAPのレプリカ数を0に変更(→LDAP停止)
```
oc scale deploy ${LDAP_DEPLOY} --replicas=0
```

##### 3. 停止確認
Db2u/ETCD/LDAPのPodが無くなっている
```
oc get all
```


#### Db2 Pod起動

##### 1. スケールアップ(Pod起動)

ETCD起動
```
oc scale sts ${ETCD_STS} --replicas=1
```

Db2起動
```
oc patch db2uengine ${DB2U_ENGINE} --type merge --patch '{"spec": { "replicas": '${NUM_REPLICAS}' } }'
```

LDAP起動
```
oc scale deploy ${LDAP_DEPLOY} --replicas=1
```

##### 2. メンテナンスモード終了

```
oc annotate db2uinstance $DB2U_INSTANCE db2u.databases.ibm.com/maintenance-pause-reconcile- --overwrite
```

##### 3. 稼働確認
 - 操作対象のDb2u/etcd/ldapのPodの状態を確認
     - Status:Running, Ready: 1/1 であること

```
oc get all
```

## Db2停止中の oc get pods出力の変遷
停止中は Status: Terminating の状態となり、停止が完了すると oc get pod コマンドの出力が無くなります

```text:停止中
# oc get all
NAME                                                        READY   STATUS        RESTARTS      AGE
pod/c-db2uinstance-3-db2u-0                                 1/1     Terminating   0             27h
pod/c-db2uinstance-3-etcd-0                                 1/1     Running       0             27h
pod/c-db2uinstance-3-ldap-df8b64c59-7vf6v                   1/1     Running       0             27h
pod/c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed     0             27h
```

```text:停止完了後
# oc get all
NAME                                                        READY   STATUS      RESTARTS      AGE
pod/c-db2uinstance-3-etcd-0                                 1/1     Running     0             27h
pod/c-db2uinstance-3-ldap-df8b64c59-7vf6v                   1/1     Running     0             27h
pod/c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed   0             27h
```

## Db2 停止 → 起動ログ
上に書き出した手順で実際に停止・起動を行ったときのログです

#### 停止ログ
```
# DB2U_INSTANCE=db2uinstance-3

# oc get db2uinstance $DB2U_INSTANCE
NAME             STATE   MAINTENANCESTATE   AGE
db2uinstance-3   Ready   None               26h

# oc get all | grep $DB2U_INSTANCE
pod/c-db2uinstance-3-db2u-0                                 1/1     Running     0             26h
pod/c-db2uinstance-3-etcd-0                                 1/1     Running     0             26h
pod/c-db2uinstance-3-ldap-df8b64c59-7vf6v                   1/1     Running     0             26h
pod/c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed   0             26h
service/c-db2uinstance-3-db2u                              ClusterIP   172.21.102.17    <none>        50000/TCP,50001/TCP,25000/TCP,25001/TCP,25002/TCP,25003/TCP,25004/TCP,25005/TCP   26h
service/c-db2uinstance-3-db2u-engn-svc                     NodePort    172.21.141.74    <none>        50001:31853/TCP,50000:30582/TCP                                                   26h
service/c-db2uinstance-3-db2u-head-engn-svc                NodePort    172.21.212.246   <none>        50001:32708/TCP,50000:30335/TCP                                                   26h
service/c-db2uinstance-3-db2u-internal                     ClusterIP   None             <none>        50000/TCP,9443/TCP,50052/TCP                                                      26h
service/c-db2uinstance-3-etcd                              ClusterIP   None             <none>        2379/TCP,2380/TCP                                                                 26h
service/c-db2uinstance-3-ldap                              ClusterIP   172.21.142.167   <none>        50389/TCP26h
deployment.apps/c-db2uinstance-3-ldap              1/1     1            1           26h
replicaset.apps/c-db2uinstance-3-ldap-df8b64c59               1         1         1       26h
statefulset.apps/c-db2uinstance-3-etcd   1/1     26h
job.batch/c-db2uinstance-3-restore-morph   1/1           7s         26h

# oc annotate db2uinstance $DB2U_INSTANCE db2u.databases.ibm.com/maintenance-pause-reconcile=true --overwrite
db2uinstance.db2u.databases.ibm.com/db2uinstance-3 annotated

# NUM_REPLICAS=$(oc get db2uengine ${DB2U_ENGINE} -ojsonpath={.spec.replicas}); echo $NUM_REPLICAS
1　　← (停止前)レプリカ数は1

# DB2U_ENGINE=$(oc get db2uengine --selector="app=${DB2U_INSTANCE},type=engine" --no-headers | awk '{print $1}'); echo $DB2U_ENGINE
c-db2uinstance-3-db2u

# ETCD_STS=$(oc get sts --selector="app=${DB2U_INSTANCE},component=etcd" --no-headers | awk '{print $1}'); echo $ETCD_STS
c-db2uinstance-3-etcd

# LDAP_DEPLOY=$(oc get deployment --selector="app=${DB2U_INSTANCE},role=ldap" --no-headers | awk '{print $1}'); echo $LDAP_DEPLOY
c-db2uinstance-3-ldap

# oc patch db2uengine ${DB2U_ENGINE} --type merge --patch '{"spec": {"replicas": 0} }'
db2uengine.db2u.databases.ibm.com/c-db2uinstance-3-db2u patched

# oc scale sts ${ETCD_STS} --replicas=0
statefulset.apps/c-db2uinstance-3-etcd scaled

# oc scale deploy ${LDAP_DEPLOY} --replicas=0
deployment.apps/c-db2uinstance-3-ldap scaled

# oc get pods
NAME                                                    READY   STATUS      RESTARTS      AGE
c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed   0             27h

# date ; oc get all | grep -i $DB2U_INSTANCE
2023年 12月 26日 火曜日 04:38:19 JST
pod/c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed   0             27h
service/c-db2uinstance-3-db2u                              ClusterIP   172.21.102.17    <none>        50000/TCP,50001/TCP,25000/TCP,25001/TCP,25002/TCP,25003/TCP,25004/TCP,25005/TCP   27h
service/c-db2uinstance-3-db2u-engn-svc                     NodePort    172.21.141.74    <none>        50001:31853/TCP,50000:30582/TCP                                                   27h
service/c-db2uinstance-3-db2u-head-engn-svc                NodePort    172.21.212.246   <none>        50001:32708/TCP,50000:30335/TCP                                                   27h
service/c-db2uinstance-3-db2u-internal                     ClusterIP   None             <none>        50000/TCP,9443/TCP,50052/TCP                                                      27h
service/c-db2uinstance-3-etcd                              ClusterIP   None             <none>        2379/TCP,2380/TCP                                                                 27h
service/c-db2uinstance-3-ldap                              ClusterIP   172.21.142.167   <none>        50389/TCP     27h
deployment.apps/c-db2uinstance-3-ldap              0/0     0            0           27h
replicaset.apps/c-db2uinstance-3-ldap-df8b64c59               0         0         0       27h
statefulset.apps/c-db2uinstance-3-etcd   0/0     27h
job.batch/c-db2uinstance-3-restore-morph   1/1           7s         27h

```


#### 起動ログ
```
# oc scale sts ${ETCD_STS} --replicas=1
statefulset.apps/c-db2uinstance-3-etcd scaled

# oc patch db2uengine ${DB2U_ENGINE} --type merge --patch '{"spec": { "replicas": '${NUM_REPLICAS}' } }'
db2uengine.db2u.databases.ibm.com/c-db2uinstance-3-db2u patched

# oc scale deploy ${LDAP_DEPLOY} --replicas=1
deployment.apps/c-db2uinstance-3-ldap scaled

# oc annotate db2uinstance $DB2U_INSTANCE db2u.databases.ibm.com/maintenance-pause-reconcile- --overwrite
db2uinstance.db2u.databases.ibm.com/db2uinstance-3 annotated

# oc get all | grep -i $DB2U_INSTANCE
pod/c-db2uinstance-3-db2u-0                                 1/1     Running     0             115s
pod/c-db2uinstance-3-etcd-0                                 1/1     Running     0             2m12s
pod/c-db2uinstance-3-ldap-df8b64c59-pxkr6                   1/1     Running     0             99s
pod/c-db2uinstance-3-restore-morph-98sh7                    0/1     Completed   0             27h
service/c-db2uinstance-3-db2u                              ClusterIP   172.21.102.17    <none>        50000/TCP,50001/TCP,25000/TCP,25001/TCP,25002/TCP,25003/TCP,25004/TCP,25005/TCP   27h
service/c-db2uinstance-3-db2u-engn-svc                     NodePort    172.21.141.74    <none>        50001:31853/TCP,50000:30582/TCP                                                   27h
service/c-db2uinstance-3-db2u-head-engn-svc                NodePort    172.21.212.246   <none>        50001:32708/TCP,50000:30335/TCP                                                   27h
service/c-db2uinstance-3-db2u-internal                     ClusterIP   None             <none>        50000/TCP,9443/TCP,50052/TCP                                                      27h
service/c-db2uinstance-3-etcd                              ClusterIP   None             <none>        2379/TCP,2380/TCP                                                                 27h
service/c-db2uinstance-3-ldap                              ClusterIP   172.21.142.167   <none>        50389/TCP    27h
deployment.apps/c-db2uinstance-3-ldap              1/1     1            1           27h
replicaset.apps/c-db2uinstance-3-ldap-df8b64c59               1         1         1       27h
statefulset.apps/c-db2uinstance-3-etcd   1/1     27h
job.batch/c-db2uinstance-3-restore-morph   1/1           7s         27h
```

#### 起動完了後のDb2稼働確認
* Podへのログイン可否　→　OK
* Db2プロセス起動状態　→　OK(db2sysc)
* DBディレクトリ(カタログ)出力　→　OK
* データベース接続　→　OK
* SQL実行(読取/書き込み)　→ ともにOK
```
# oc rsh c-db2uinstance-3-db2u-0 bash
Defaulted container "db2u" out of: db2u, instdb (init), init-labels (init), init-kernel (init)
[db2uadm@c-db2uinstance-3-db2u-0 /]$ su - db2inst1
Last login: Tue Dec 26 06:24:30 UTC 2023
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2gcf -s

DB2 State : Available
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
db2uadm       1      0  0 06:24 ?        00:00:00 /bin/sh -x /etc/runit/entrypoint.sh
db2uadm       7      1  0 06:24 ?        00:00:00 runsvdir -P /etc/service log: ........................................
db2uadm       8      7  0 06:24 ?        00:00:00 runsv wolverine
db2uadm       9      7  0 06:24 ?        00:00:00 runsv db2u
db2uadm      10      7  0 06:24 ?        00:00:00 runsv sshd
db2uadm      11      7  0 06:24 ?        00:00:00 runsv db2uapi
db2uadm      12      9  0 06:24 ?        00:00:00 /usr/bin/coreutils --coreutils-prog-shebang=sleep /usr/bin/sleep infin
db2uadm      13     11  0 06:24 ?        00:00:00 db2u-apiserver --type control
db2uadm      14      8  0 06:24 ?        00:00:00 svlogd -tt ./main
db2uadm      15     10  0 06:24 ?        00:00:00 svlogd -tt ./main
db2uadm      16     10  0 06:24 ?        00:00:00 /bin/sh -e ./run
root        222     16  0 06:24 ?        00:00:00 sudo -E /usr/sbin/sshd -D -p 50022 -e
root        227    222  0 06:24 ?        00:00:00 /usr/sbin/sshd -D -p 50022 -e
db2uadm     509      7  0 06:24 ?        00:00:00 runsv sssd
db2uadm     511    509  0 06:24 ?        00:00:00 /bin/sh -e ./run
root        512    511  0 06:24 ?        00:00:00 sudo -E /usr/sbin/sssd -i --logger=files
root        513    512  0 06:24 ?        00:00:00 /usr/sbin/sssd -i --logger=files
root        514    513  0 06:24 ?        00:00:00 /usr/libexec/sssd/sssd_be --domain local --uid 0 --gid 0 --logger=file
root        515    513  0 06:24 ?        00:00:00 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files
root        516    513  0 06:24 ?        00:00:00 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --logger=files
root       3261      1  0 06:24 ?        00:00:00 db2wdog 0 [db2inst1]
db2inst1   3263   3261 11 06:24 ?        00:00:30 db2sysc 0
root       3277   3261  0 06:24 ?        00:00:00 db2ckpwd 0
root       3278   3261  0 06:24 ?        00:00:00 db2ckpwd 0
root       3279   3261  0 06:24 ?        00:00:00 db2ckpwd 0
db2inst1   3281   3261  0 06:24 ?        00:00:00 db2vend (PD Vendor Process - 1) 0
db2inst1   3291   3261  0 06:24 ?        00:00:00 db2acd 0 ,0,0,0,1,0,0,00000000,0,0,0000000000000000,0000000000000000,0
root       3371      8  0 06:24 ?        00:00:00 su - db2uhausr -l --session-command=/db2uhausr/service/wolverine/run
db2uhau+   3499   3371  0 06:25 ?        00:00:00 /bin/bash -x /db2uhausr/service/wolverine/run
db2uhau+   3624   3499  1 06:25 ?        00:00:02 /usr/bin/python3 -m wolverine.ha.service --pid-file /db2u/tmp/ha.pid
db2uadm    3724      0  0 06:27 pts/0    00:00:00 bash
root       3750   3724  0 06:27 pts/0    00:00:00 su - db2inst1
db2inst1   3751   3750  0 06:27 pts/0    00:00:00 -ksh
db2inst1   4383   3751  0 06:29 pts/0    00:00:00 ps -ef
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ ps -ef | grep -i db2
db2uadm       1      0  0 06:24 ?        00:00:00 /bin/sh -x /etc/runit/entrypoint.sh
db2uadm       7      1  0 06:24 ?        00:00:00 runsvdir -P /etc/service log: ....................................................................................................................................................................................................................................................................................................................................................................................................+ exec
db2uadm       8      7  0 06:24 ?        00:00:00 runsv wolverine
db2uadm       9      7  0 06:24 ?        00:00:00 runsv db2u
db2uadm      10      7  0 06:24 ?        00:00:00 runsv sshd
db2uadm      11      7  0 06:24 ?        00:00:00 runsv db2uapi
db2uadm      12      9  0 06:24 ?        00:00:00 /usr/bin/coreutils --coreutils-prog-shebang=sleep /usr/bin/sleep infinity
db2uadm      13     11  0 06:24 ?        00:00:00 db2u-apiserver --type control
db2uadm      14      8  0 06:24 ?        00:00:00 svlogd -tt ./main
db2uadm      15     10  0 06:24 ?        00:00:00 svlogd -tt ./main
db2uadm      16     10  0 06:24 ?        00:00:00 /bin/sh -e ./run
db2uadm     509      7  0 06:24 ?        00:00:00 runsv sssd
db2uadm     511    509  0 06:24 ?        00:00:00 /bin/sh -e ./run
root       3261      1  0 06:24 ?        00:00:00 db2wdog 0 [db2inst1]
db2inst1   3263   3261 11 06:24 ?        00:00:30 db2sysc 0
root       3277   3261  0 06:24 ?        00:00:00 db2ckpwd 0
root       3278   3261  0 06:24 ?        00:00:00 db2ckpwd 0
root       3279   3261  0 06:24 ?        00:00:00 db2ckpwd 0
db2inst1   3281   3261  0 06:24 ?        00:00:00 db2vend (PD Vendor Process - 1) 0
db2inst1   3291   3261  0 06:24 ?        00:00:00 db2acd 0 ,0,0,0,1,0,0,00000000,0,0,0000000000000000,0000000000000000,00000000,00000000,00000000,00000000,00000000,00000000,0000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000039dc07000,0000000000000000,0000000000000000,1,0,0,,,,,a89f94,14,1e014,2,0,1,0000000000041fc0,0x240000000,0x240000000,1600000,2,2,14
root       3371      8  0 06:24 ?        00:00:00 su - db2uhausr -l --session-command=/db2uhausr/service/wolverine/run
db2uhau+   3499   3371  0 06:25 ?        00:00:00 /bin/bash -x /db2uhausr/service/wolverine/run
db2uhau+   3624   3499  1 06:25 ?        00:00:02 /usr/bin/python3 -m wolverine.ha.service --pid-file /db2u/tmp/ha.pid
db2uadm    3724      0  0 06:27 pts/0    00:00:00 bash
root       3750   3724  0 06:27 pts/0    00:00:00 su - db2inst1
db2inst1   3751   3750  0 06:27 pts/0    00:00:00 -ksh
db2inst1   4384   3751  0 06:29 pts/0    00:00:00 ps -ef
db2inst1   4385   3751  0 06:29 pts/0    00:00:00 grep --color=auto -i db2
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 list db directory

 System Database Directory

 Number of entries in the directory = 1

Database 1 entry:

 Database alias                       = SAMPLEDB
 Database name                        = SAMPLEDB
 Local database directory             = /mnt/blumeta0/db2/databases
 Database release level               = 15.00
 Comment                              =
 Directory entry type                 = Indirect
 Catalog database partition number    = 0
 Alternate server hostname            =
 Alternate server port number         =

[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 connect to sampledb

   Database Connection Information

 Database server        = DB2/LINUXX8664 11.5.9.0
 SQL authorization ID   = DB2INST1
 Local database alias   = SAMPLEDB

[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 list tables

Table/View                      Schema          Type  Creation time
------------------------------- --------------- ----- --------------------------
T1                              DB2INST1        T     2023-12-25-03.17.08.074539

  1 record(s) selected.

[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 "select * from t1"

C1          C2                               C3
----------- -------------------------------- --------------------------
          1 db created                       2023-12-25-03.17.20.923992

  1 record(s) selected.

[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ fter pod restart' , current timestamp)"                          <
DB20000I  The SQL command completed successfully.
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 "select * from t1"

C1          C2                               C3
----------- -------------------------------- --------------------------
          1 db created                       2023-12-25-03.17.20.923992
          2 after pod restart                2023-12-26-06.30.21.960806

  2 record(s) selected.

[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ db2 terminate
DB20000I  The TERMINATE command completed successfully.
[db2inst1@c-db2uinstance-3-db2u-0 - Db2U db2inst1]$ exit
[db2uadm@c-db2uinstance-3-db2u-0 /]$ exit
exit
```