[IBMCloud] IKS構成情報のVeleroによるバックアップ

## 目的
Kubernetesクラスタの構成ファイルを手動で取得するのではなく、Veleroによる一括バックアップを検証し、バックアップデータが正しく取得・保存されていることを確認
IBMCloudのドキュメントに流れの記載はあるがイマイチわかりにくいため、検証する

[Veleroを使用したデータのエクスポート](https://cloud.ibm.com/docs/containers?topic=containers-data-portability&locale=ja#export-velero)

## 1. Velero CLIのインストール
以下のドキュメントを参考にMacOSにインストールします
https://velero.io/docs/v1.14/basic-install/#install-the-cli

```
#brew install velero

==> Downloading https://ghcr.io/v2/homebrew/core/velero/manifests/1.15.0
####################################################################################################################################################################################################################### 100.0%
==> Fetching velero
==> Downloading https://ghcr.io/v2/homebrew/core/velero/blobs/sha256:f6e745d1d8d7918875b381b1b02b6413950eb2e20cdd803e047c30c1ec456e30
####################################################################################################################################################################################################################### 100.0%
==> Pouring velero--1.15.0.arm64_sonoma.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions
==> Summary
🍺  /opt/homebrew/Cellar/velero/1.15.0: 10 files, 66.5MB
==> Running `brew cleanup velero`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> `brew cleanup` has not been run in the last 30 days, running now...
```

## 2. IBM Cloud Kubernetes Service CLIのインストール
以下のドキュメントを参考にMacOSにインストールします
https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/#install-kubectl-on-macos

```
% brew install kubectl
==> Downloading https://formulae.brew.sh/api/formula.jws.json
####################################################################################################################################################################################################################### 100.0%
==> Downloading https://formulae.brew.sh/api/cask.jws.json

kubernetes-cli 1.31.1 is already installed but outdated (so it will be upgraded).
==> Downloading https://ghcr.io/v2/homebrew/core/kubernetes-cli/manifests/1.32.0
####################################################################################################################################################################################################################### 100.0%
==> Fetching kubernetes-cli
==> Downloading https://ghcr.io/v2/homebrew/core/kubernetes-cli/blobs/sha256:3c077fc014d4684fcb56068a0871be83e39258a184a5bbd2e950474af6dca2a7
####################################################################################################################################################################################################################### 100.0%
==> Upgrading kubectl
  1.31.1 -> 1.32.0
==> Pouring kubernetes-cli--1.32.0.arm64_sonoma.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions
==> Summary
🍺  /opt/homebrew/Cellar/kubernetes-cli/1.32.0: 255 files, 60.7MB
==> Running `brew cleanup kubernetes-cli`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Removing: /opt/homebrew/Cellar/kubernetes-cli/1.31.1... (237 files, 60.2MB)
Removing: /Users/kh/Library/Caches/Homebrew/kubernetes-cli_bottle_manifest--1.31.1... (9.3KB)
==> Upgrading 1 dependent of upgraded formula:
Disable this behaviour by setting HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
minikube 1.32.0 -> 1.34.0
==> Downloading https://ghcr.io/v2/homebrew/core/minikube/manifests/1.34.0
Already downloaded: /Users/kh/Library/Caches/Homebrew/downloads/8836daab937b9a94737b9577a592cf1b4b24420187c8a0a949e94da2f4018009--minikube-1.34.0.bottle_manifest.json
==> Checking for dependents of upgraded formulae...
==> No broken dependents found!
```

**インストール、バージョン確認**
```
% kubectl version --client
Client Version: v1.32.0
Kustomize Version: v5.5.0
```

## 3. 保存先用IBM Cloud Object Storageインスタンスの作成 
IBMCloud Portalの検索メニューより、「Object Storage」を選択
![貼り付けた画像_2024_12_18_9_23.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/1fe513b0-a17f-52fd-0048-73f0db40ef9d.png)

[IBMCloud] -> [Standard]を選択
任意のサービス名を入力し、環境に合わせたリソースグループを選択後、[Create]を押下
![貼り付けた画像_2024_12_18_9_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/9f05c443-90db-299e-10d4-876f5cd58be2.png)

## 4. バケットの作成
作成したCloud Object Storageのページよりバケットを作成する

IBMCloud Portal -> [Infrastructure] -> [Storage] -> [Object storage]を選択
![貼り付けた画像_2024_12_18_9_29.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/b6b98a25-99ff-dd81-a164-e6c6f3d34954.png)

[Instance] -> [作成したInstance名] -> [Create bucket]を選択
![貼り付けた画像_2024_12_18_9_32.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/65cace49-7b9c-c107-58e4-514067b1e73f.png)

[Create a Custom Bucket]の[Create]を選択
![貼り付けた画像_2024_12_18_9_34.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/ad285be5-aa49-2d46-3b04-a1443c682510.png)

[Unique bucket name]に任意名を入力
Resiliencyは[Cross Region]を選択
Locationは[United State Geo(us-geo)]を選択
その他は必要に応じて選択し、最後に[Create bucket]を押下
![貼り付けた画像_2024_12_18_9_40.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/3f8cc9de-2983-d88b-646f-9226f402b320.png)

## 5. サービスクレデンシャルの作成

Instanceのページから[Service credentials]を選択し、[New Credential]を押下
![貼り付けた画像_2024_12_18_9_47.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/4f464b80-308d-4324-586d-702d7d647502.png)

[Name]に任意名を入力
[Role]は[Manager]を選択
Service IDがあればそちらを選択し、新規作成であれば[Auto Generated]を選択
Include HMAC Credentialを[ON]にし、[Add]を押下
![貼り付けた画像_2024_12_18_9_49.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/08fa769c-e334-6a8b-e9bd-ac21e27825e4.png)

## 6. VeleroからObject Storageへのアクセス用資格情報ファイルの作成
Veleroをインストールする端末でファイルを作成する

作成したInstanceのService credentialsにある
<font color=blue>access_key_id</font>をaws_access_key_idにコピペ
<font color=green>secret_access_key</font>をaws_secret_access_keyにコピペ

![貼り付けた画像_2024_12_18_10_06.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/69c27c23-a5c0-d93d-6e5a-dc0668270058.png)



```
% cat velero-credential
[default]
aws_access_key_id=<blue-access_key_id>
aws_secret_access_key=<green-secret_access_key>
```

## 7. IAMグループの作成
IBMCloud Portalから[Manage] -> [Access(IAM)] -> [Access groups] -> [Create]を押下
![貼り付けた画像_2024_12_18_10_10.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/fa32b085-6e9c-4473-6ead-00ad303e20e5.png)

任意の[Name]を入力し、[Create]を押下
![貼り付けた画像_2024_12_18_10_12.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/16d89a69-8fca-8c60-953a-7faadac0e27f.png)

[Service IDs]を選択し、[Add]を押下
![貼り付けた画像_2024_12_18_10_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/26b11b6b-c11d-5ffb-1015-bf97cf2599f7.png)

サービスクレデンシャルの作成で作成した名前を選択し、[Add]を押下
![貼り付けた画像_2024_12_18_10_14.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/65946c0e-a1c2-e0df-d802-8690279de06b.png)

[Access]のタブを選択し、[Assign access]を押下
![貼り付けた画像_2024_12_18_10_16.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/703043/359486a8-9485-79de-5f60-aa1a75225a70.png)

[Service] -> Cloud Object Storage
[Resources] -> Service Instanceを選択し、作成したObject Storageのインスタンスを選択
[Roles and actions] 
-> Service accessで[Manager]にチェック
-> Platform accessで[Viwer]にチェック
最後に、[Assign]を押下

## 8. Veleroのインストール
インストール時のパラメータ修正
```
velero install --provider aws --bucket <bucket-name> --secret-file <hmac-credentials-file> --use-volume-snapshots=false --default-volumes-to-fs-backup --use-node-agent --plugins velero/velero-plugin-for-aws:v1.9.0 --image velero/velero:v1.13.0 --backup-location-config region=us-geo,s3ForcePathStyle="true",s3Url=https://s3.direct.us.cloud-object-storage.appdomain.cloud
```
--buckert = 作成したバケット名を入力
--secret-file = ローカルに作成した資格情報ファイルを指定
```
[default]
aws_access_key_id=<blue-access_key_id>
aws_secret_access_key=<green-secret_access_key>
```

インストールの実行
```
% velero install --provider aws --bucket acs-paas-velero-backup --secret-file velero-credential --use-volume-snapshots=false --default-volumes-to-fs-backup --use-node-agent --plugins velero/velero-plugin-for-aws:v1.9.0 --image velero/velero:v1.13.0 --backup-location-config region=us-geo,s3ForcePathStyle="true",s3Url=https://s3.direct.us.cloud-object-storage.appdomain.cloud
CustomResourceDefinition/backuprepositories.velero.io: attempting to create resource
CustomResourceDefinition/backuprepositories.velero.io: attempting to create resource client
CustomResourceDefinition/backuprepositories.velero.io: created
CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: attempting to create resource client
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource client
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource client
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource client
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: attempting to create resource client
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: attempting to create resource client
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource client
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource client
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
CustomResourceDefinition/datadownloads.velero.io: attempting to create resource
CustomResourceDefinition/datadownloads.velero.io: attempting to create resource client
CustomResourceDefinition/datadownloads.velero.io: created
CustomResourceDefinition/datauploads.velero.io: attempting to create resource
CustomResourceDefinition/datauploads.velero.io: attempting to create resource client
CustomResourceDefinition/datauploads.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero: attempting to create resource
Namespace/velero: attempting to create resource client
Namespace/velero: created
ClusterRoleBinding/velero: attempting to create resource
ClusterRoleBinding/velero: attempting to create resource client
ClusterRoleBinding/velero: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: attempting to create resource client
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: attempting to create resource client
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: attempting to create resource client
BackupStorageLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: attempting to create resource client
Deployment/velero: created
DaemonSet/node-agent: attempting to create resource
DaemonSet/node-agent: attempting to create resource client
DaemonSet/node-agent: created
Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.
```

PODの動作確認
```
% kubectl get pods -n velero
NAME                      READY   STATUS    RESTARTS   AGE
node-agent-6v7wk          1/1     Running   0          24m
node-agent-rwdsn          1/1     Running   0          24m
velero-7f4f557fdd-rrf82   1/1     Running   0          24m
```

## 9. バックアップの実施
defaultネームスペースのすべてのPVC、PV、ポッドをバックアップ
```
% velero backup create mybackup --include-resources pvc,pv,pod --default-volumes-to-fs-backup --snapshot-volumes=false --include-namespaces default --exclude-namespaces kube-system,test-namespace
Backup request "mybackup" submitted successfully.
Run `velero backup describe mybackup` or `velero backup logs mybackup` for more details.
```

バックアップの確認
```
% velero backup describe mybackup
Name:         mybackup
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.30.7+IKS
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=30

Phase:  Completed


Namespaces:
  Included:  default
  Excluded:  kube-system, test-namespace

Resources:
  Included:        pvc, pv, pod
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  default

Velero-Native Snapshot PVs:  false
Snapshot Move Data:          false
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    10m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2024-12-18 10:25:53 +0900 JST
Completed:  2024-12-18 10:26:22 +0900 JST

Expiration:  2025-01-17 10:25:53 +0900 JST

Total items to be backed up:  10
Items backed up:              10

Backup Volumes:
  <error getting backup volume info: Get "https://s3.direct.us.cloud-object-storage.appdomain.cloud/acs-paas-velero-backup/backups/mybackup/mybackup-volumeinfo.json.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=5ae1ed4a475744f8b95da1a78d99628a%2F20241218%2Fus-geo%2Fs3%2Faws4_request&X-Amz-Date=20241218T012741Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=b45171aa19b67479fee7465b20d801d4eae01ebb72e34344b5013add6a6e6ab0": context deadline exceeded>

HooksAttempted:  0
HooksFailed:     0
```