OpenShift - oc command Tips - Krew plugin

OpenShift の様々な操作で使用する `oc` コマンドの `Krew plugin` をご紹介します。

https://krew.sigs.k8s.io/

`Krew plugin` は `kubectl / oc` コマンドに実装されている plugin 機能 ( `kubectl plugin / oc plugin`) に配布・管理機能を追加します。`Krew plugin` も plugin の1つとして実装されています。

https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/

執筆時点では、`oc` コマンドにおける `Krew plugin` の使用は `Technology Preview` となっています。

https://docs.openshift.com/container-platform/4.16/cli_reference/openshift_cli/managing-cli-plugins-krew.html

### Install Krew plugin
上記ドキュメンから Link されている以下を参考に `Krew plugin` をインストールします。

https://krew.sigs.k8s.io/docs/user-guide/setup/install/

```shell
$ (
>   set -x; cd "$(mktemp -d)" &&
>   OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
>   ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
>   KREW="krew-${OS}_${ARCH}" &&
>   curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
>   tar zxvf "${KREW}.tar.gz" &&
>   ./"${KREW}" install krew
> )
++ mktemp -d
+ cd /tmp/tmp.RfH5SyD106
++ uname
++ tr '[:upper:]' '[:lower:]'
+ OS=linux
++ uname -m
++ sed -e s/x86_64/amd64/ -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/'
+ ARCH=amd64
+ KREW=krew-linux_amd64
+ curl -fsSLO https://github.com/kubernetes-sigs/krew/releases/latest/download/krew-linux_amd64.tar.gz

+ tar zxvf krew-linux_amd64.tar.gz
./LICENSE
./krew-linux_amd64
+ ./krew-linux_amd64 install krew
Adding "default" plugin index from https://github.com/kubernetes-sigs/krew-index.git.
Updated the local copy of plugin index.
Installing plugin: krew
Installed plugin: krew
\
 | Use this plugin:
 | 	kubectl krew
 | Documentation:
 | 	https://krew.sigs.k8s.io/
 | Caveats:
 | \
 |  | krew is now installed! To start using kubectl plugins, you need to add
 |  | krew's installation directory to your PATH:
 |  | 
 |  |   * macOS/Linux:
 |  |     - Add the following to your ~/.bashrc or ~/.zshrc:
 |  |         export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
 |  |     - Restart your shell.
 |  | 
 |  |   * Windows: Add %USERPROFILE%\.krew\bin to your PATH environment variable
 |  | 
 |  | To list krew commands and to get help, run:
 |  |   $ kubectl krew
 |  | For a full list of available plugins, run:
 |  |   $ kubectl krew search
 |  | 
 |  | You can find documentation at
 |  |   https://krew.sigs.k8s.io/docs/user-guide/quickstart/.
 | /
/
```
```shell
$ export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"

$ echo $PATH
/root/.krew/bin:....

$ ls -l /root/.krew/bin
total 0
lrwxrwxrwx. 1 root root 34 Nov 13 10:15 kubectl-krew -> /root/.krew/store/krew/v0.4.4/krew
```
```shell
$ oc krew version
OPTION            VALUE
GitTag            v0.4.4
GitCommit         343e657
IndexURI          https://github.com/kubernetes-sigs/krew-index.git
BasePath          /root/.krew
IndexPath         /root/.krew/index/default
InstallPath       /root/.krew/store
BinPath           /root/.krew/bin
DetectedPlatform  linux/amd64

$ oc krew list
PLUGIN            VERSION
krew              v0.4.4

$ oc krew --help
krew is the kubectl plugin manager.
You can invoke krew through kubectl: "kubectl krew [command]..."

Usage:
  kubectl krew [command]

Available Commands:
  help        Help about any command
  index       Manage custom plugin indexes
  info        Show information about an available plugin
  install     Install kubectl plugins
  list        List installed kubectl plugins
  search      Discover kubectl plugins
  uninstall   Uninstall plugins
  update      Update the local copy of the plugin index
  upgrade     Upgrade installed plugins to newer versions
  version     Show krew version and diagnostics

Flags:
  -h, --help      help for krew
  -v, --v Level   number for the log level verbosity

Use "kubectl krew [command] --help" for more information about a command.
```

### Search plugin
`oc krew search` で plugin を検索する事ができます。
```shell
$ oc krew search
NAME                            DESCRIPTION                                         INSTALLED
～中略～
validate                        Validation of resources for native Kubernetes t...  no
vela                            Easily interact with KubeVela                       no
view-allocations                List allocations per resources, nodes, pods.        no
view-cert                       View certificate information stored in secrets      no
view-quotas                     List resource quotas in colors                      no
view-secret                     Decode Kubernetes secrets                           no
view-serviceaccount-kubeconfig  Show a kubeconfig setting to access the apiserv...  no
view-utilization                Shows cluster cpu and memory utilization            no
view-webhook                    Visualize your webhook configurations               no
viewnode                        Displays nodes with their pods and containers a...  no
virt                            Control KubeVirt virtual machines using virtctl     no
volsync                         Manage replication with the VolSync operator        no
vpa-recommendation              Compare VPA recommendations to actual resources...  no
wait-job                        Waits for a Job to complete or fail                 no
warp                            Sync and execute local files in Pod                 no
whisper-secret                  Create secrets with improved privacy                no
who-can                         Shows who has RBAC permissions to access Kubern...  no
whoami                          Show the subject that's currently authenticated...  no
windows-debug                   Windows node access via kubectl                     no
```
同様に、以下の URL からも確認することが出来ます。

https://krew.sigs.k8s.io/plugins/

一例として `view-utilization` の内容を確認してみます。
```shell
$ oc krew info view-utilization
NAME: view-utilization
INDEX: default
URI: https://github.com/etopeter/kubectl-view-utilization/releases/download/v0.3.3/kubectl-view-utilization-v0.3.3.tar.gz
SHA256: bcdd9925d13ff3837f61336269f9d45e338ca14df2f8174120bd571217d99918
VERSION: v0.3.3
HOMEPAGE: https://github.com/etopeter/kubectl-view-utilization
DESCRIPTION:
This plugin shows cluster resource utilization based on cpu and memory. It collects pod requests and node available resources to calculate metrics.

CAVEATS:
\
 | This plugin needs the following programs:
 | * bash
 | * awk (gawk,mawk,awk)
/
```
### Install plugin
`oc krew install` で plugin をインストールしてみます。
```shell
$ oc krew install view-allocations
Updated the local copy of plugin index.
Installing plugin: view-allocations
Installed plugin: view-allocations
\
 | Use this plugin:
 |      kubectl view-allocations
 | Documentation:
 |      https://github.com/davidB/kubectl-view-allocations
/
WARNING: You installed plugin "view-allocations" from the krew-index plugin repository.
   These plugins are not audited for security by the Krew maintainers.
   Run them at your own risk.
```
インストール時に `WARNING` 表示される通り、セキュリティー上のリスクがある点には注意が必要です。
```shell
$ oc krew list
PLUGIN            VERSION
krew              v0.4.4
view-allocations  v0.20.1
```
`view-allocations` plugin はリソースの状況を分かりやすく確認することが出来ます。
```shell
$ oc view-allocations --help
kubectl plugin to list allocations (cpu, memory, gpu,... X utilization, requested, limit, allocatable,...)

Usage: kubectl-view_allocations [OPTIONS]

Options:
      --context <CONTEXT>              The name of the kubeconfig context to use
  -n, --namespace <NAMESPACE>          Show only pods from this namespace
  -l, --selector <SELECTOR>            Show only resource match this label selector
  -u, --utilization                    Force to retrieve utilization (for cpu and memory), require to have metrics-server https://github.com/kubernetes-sigs/metrics-server
  -z, --show-zero                      Show lines with zero requested and zero limit and zero allocatable
      --precheck                       pre-check access and refersh token on kubeconfig by running `kubectl cluster-info`
      --accept-invalid-certs           accept invalid certificats (dangerous)
  -r, --resource-name <RESOURCE_NAME>  Filter resources shown by name(s), by default all resources are listed
  -g, --group-by <GROUP_BY>            Group information hierarchically (default: -g resource -g node -g pod) [possible values: resource, node, pod, namespace]
  -o, --output <OUTPUT>                Output format [default: table] [possible values: table, csv]
  -h, --help                           Print help
  -V, --version                        Print version

https://github.com/davidB/kubectl-view-allocations
```

```shell
$ oc view-allocations -n openshift-image-registry
 Resource                                                   Requested     Limit  Allocatable     Free
  cpu                                                     (0%) 180.0m        __         87.2     87.0
     99.888.7.17                                           (0%) 10.0m        __         15.9     15.9
        node-ca-h6l5t                                           10.0m        __           __       __
     99.888.7.4                                            (0%) 10.0m        __          7.9      7.9
        node-ca-fp4mp                                           10.0m        __           __       __
     99.888.7.5                                            (0%) 10.0m        __          7.9      7.9
        node-ca-lbgbm                                           10.0m        __           __       __
     99.888.7.6                                            (0%) 20.0m        __          7.9      7.9
        cluster-image-registry-operator-588f48965-8mqfc         10.0m        __           __       __
        node-ca-cdrl6                                           10.0m        __           __       __
     33.444.666.4                                          (0%) 10.0m        __         15.9     15.9
        node-ca-njqcw                                           10.0m        __           __       __
     33.444.666.5                                          (0%) 10.0m        __         15.9     15.9
        node-ca-k478v                                           10.0m        __           __       __
     33.444.55.5                                          (1%) 110.0m        __         15.9     15.8
        image-registry-86c77dbfc4-9zz48                        100.0m        __           __       __
        node-ca-v2nfp                                           10.0m        __           __       __
  ephemeral-storage                                                __        __       878.4G       __
     99.888.7.17                                                   __        __        99.3G       __
     99.888.7.4                                                    __        __        99.3G       __
     99.888.7.5                                                    __        __        99.3G       __
     99.888.7.6                                                    __        __        99.3G       __
     33.444.666.4                                                  __        __       282.4G       __
     33.444.666.5                                                  __        __        99.3G       __
     33.444.55.5                                                   __        __        99.3G       __
  memory                                                 (0%) 376.0Mi        __      327.6Gi  327.2Gi
     99.888.7.17                                          (0%) 10.0Mi        __       27.8Gi   27.8Gi
        node-ca-h6l5t                                          10.0Mi        __           __       __
     99.888.7.4                                           (0%) 10.0Mi        __       57.4Gi   57.4Gi
        node-ca-fp4mp                                          10.0Mi        __           __       __
     99.888.7.5                                           (0%) 10.0Mi        __       57.4Gi   57.4Gi
        node-ca-lbgbm                                          10.0Mi        __           __       __
     99.888.7.6                                           (0%) 60.0Mi        __       57.4Gi   57.3Gi
        cluster-image-registry-operator-588f48965-8mqfc        50.0Mi        __           __       __
        node-ca-cdrl6                                          10.0Mi        __           __       __
     33.444.666.4                                         (0%) 10.0Mi        __       72.2Gi   72.2Gi
        node-ca-njqcw                                          10.0Mi        __           __       __
     33.444.666.5                                         (0%) 10.0Mi        __       27.8Gi   27.8Gi
        node-ca-k478v                                          10.0Mi        __           __       __
     33.444.55.5                                         (1%) 266.0Mi        __       27.8Gi   27.5Gi
        image-registry-86c77dbfc4-9zz48                       256.0Mi        __           __       __
        node-ca-v2nfp                                          10.0Mi        __           __       __
  nvidia.com/gpu                                                   __        __          1.0       __
     33.444.666.4                                                  __        __          1.0       __
  pods                                                       (1%) 9.0  (1%) 9.0        970.0    961.0
     99.888.7.17                                             (1%) 1.0  (1%) 1.0        160.0    159.0
     99.888.7.4                                              (1%) 1.0  (1%) 1.0        110.0    109.0
     99.888.7.5                                              (1%) 1.0  (1%) 1.0        110.0    109.0
     99.888.7.6                                              (2%) 2.0  (2%) 2.0        110.0    108.0
     33.444.666.4                                            (1%) 1.0  (1%) 1.0        160.0    159.0
     33.444.666.5                                            (1%) 1.0  (1%) 1.0        160.0    159.0
     33.444.55.5                                             (1%) 2.0  (1%) 2.0        160.0    158.0
```

### Create plugin
独自の plugin を作成することもできます。

https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/#writing-kubectl-plugins

ここでは、以下のような簡単な Bash Script を plugin として使用してみます。
```shell
$ cat sample.sh
#!/bin/bash -x

echo "$(basename $0)"
oc get pod -A | egrep -v "Running|Completed"

$ ./sample.sh
++ basename ./sample.sh
+ echo sample.sh
sample.sh
+ oc get pod -A
+ egrep -v 'Running|Completed'
NAMESPACE                                          NAME                                                              READY   STATUS      RESTARTS        AGE
test-namespace-001                                 test-pod-deployment-usecase01-0001-5cd65c899d-8n4rf               0/3     Pending     0               22d
```

`PATH` 環境変数スコープ内のいずれかの Directory に Bash Script を配置します。ここでは、`/usr/local/bin` を使用してみます。
ファイル名を `kubectl-` で始まるものに変更します。ファイル名の `-` と `_` によって plugin の呼び出し方が変わる事を確認するために、以下の2ファイルを配置してみます。
```shell
$ ls -1 /usr/local/bin/kubectl-*
/usr/local/bin/kubectl-list-error-pod
/usr/local/bin/kubectl-list_error_pod
```
plugin の呼び出し方によって、Bash Script が異なる事が確認できます。
```shell
$ oc list-error-pod
++ basename /usr/local/bin/kubectl-list_error_pod
+ echo kubectl-list_error_pod
kubectl-list_error_pod
+ oc get pod -A
+ egrep -v 'Running|Completed'
NAMESPACE                                          NAME                                                              READY   STATUS      RESTARTS         AGE
test-namespace-001                                 test-pod-deployment-usecase01-0001-5cd65c899d-8n4rf               0/3     Pending     0                22d

$ oc list error pod
++ basename /usr/local/bin/kubectl-list-error-pod
+ echo kubectl-list-error-pod
kubectl-list-error-pod
+ oc get pod -A
+ egrep -v 'Running|Completed'
NAMESPACE                                          NAME                                                              READY   STATUS      RESTARTS          AGE
test-namespace-001                                 test-pod-deployment-usecase01-0001-5cd65c899d-8n4rf               0/3     Pending     0                 22d
```
なお、これらの plugin は `Krew plugin` による管理を行っていないため、`oc krew list` には表示されません。管理を行う場合は、以下の作業が必要です。

https://krew.sigs.k8s.io/docs/developer-guide/distributing-with-krew/

同様のネーミング・ルールは `oc` コマンドでも使用されており、`oc-mirror plugin` も、これによって実装されています。
```shell
$ ls -1 /usr/local/bin/oc-*
/usr/local/bin/oc-kubectl_list_error_pod
/usr/local/bin/oc-mirror

$ oc kubectl-list-error-pod
++ basename /usr/local/bin/oc-kubectl_list_error_pod
+ echo oc-kubectl_list_error_pod
oc-kubectl_list_error_pod
+ oc get pod -A
+ egrep -v 'Running|Completed'
NAMESPACE                                          NAME                                                              READY   STATUS      RESTARTS         AGE
test-namespace-001                                 test-pod-deployment-usecase01-0001-5cd65c899d-8n4rf               0/3     Pending     0                22d

$ oc mirror version
Client Version: version.Info{Major:"", Minor:"", GitVersion:"4.16.0-202409032335.p0.g819f011.assembly.stream.el9-819f011", GitCommit:"819f0115e4be3f55ed7d4edae1c38035811771ae", GitTreeState:"clean", BuildDate:"2024-09-04T05:16:07Z", GoVersion:"go1.21.11 (Red Hat 1.21.11-1.module+el8.10.0+21986+2112108a) X:strictfipsruntime", Compiler:"gc", Platform:"linux/amd64"}
```
```shell
$ oc plugin list
The following compatible plugins are available:

/root/.krew/bin/kubectl-krew
/root/.krew/bin/kubectl-view_allocations
/usr/local/bin/kubectl-list-error-pod
/usr/local/bin/kubectl-list_error_pod
/usr/local/bin/oc-kubectl_list_error_pod
/usr/local/bin/oc-mirror
```