Descheduler 設定

# 概要
 - k8sテスト環境構築<br>
<b>Descheduler 設定</b>

# 構築目次
 - [全体目次] (https://qiita.com/prodigy413/items/89fcb8cfb71e80f541d8)

# 環境
 - Rancher: v2.5.3
 - kubernetes(Client): v1.19.4
 - kubernetes(Server): v1.19.4
 - Descheduler: v0.20.0

# Descheduler
Deschedulerは設定したPolicyに該当するPodをNodeから削除する
→ 再スケジューリングはデフォルトのschedulerが行う

 - <b>Descheduler Github Page</b><br>
https://github.com/kubernetes-sigs/descheduler<br>
https://github.com/kubernetes-sigs/descheduler/releases

# 動作確認
 - ポリシーの中で「RemoveDuplicates」の動作を確認する<br>
→ Deployment/ReplicaSetなどのPodが1Node上で複数稼働中の場合はPodを削除する
 - deschedulerの実行には[<b>`Job`</b>](https://kubernetes.io/docs/concepts/workloads/controllers/job/)リソースを利用する
<br>----------<br>
 - テスト用Deployment作成

~~~yaml:test-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.19.6
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
~~~

 - manifest適用

~~~
## Manifest 適用
$ kubectl apply -f test-nginx.yaml 

## 1Pod/1Nodeを確認
$ kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE       NOMINATED NODE   READINESS GATES
nginx-76ccf9dd9d-mw58c   1/1     Running   0          15s   10.42.1.186   worker01   <none>           <none>
nginx-76ccf9dd9d-plnq8   1/1     Running   0          15s   10.42.3.110   worker03   <none>           <none>
nginx-76ccf9dd9d-s9sdx   1/1     Running   0          15s   10.42.2.113   worker02   <none>           <none>
~~~

 - ポリシー違反状態を作る

~~~
## worker03ノードからPodを退去させる
$ kubectl drain worker03 --ignore-daemonsets --delete-local-data --force

## ノードステータス確認
$ kubectl get nodes
NAME       STATUS                     ROLES               AGE   VERSION
master01   Ready                      controlplane,etcd   12d   v1.19.7
worker01   Ready                      worker              12d   v1.19.7
worker02   Ready                      worker              12d   v1.19.7
worker03   Ready,SchedulingDisabled   worker              12d   v1.19.7

## Pod確認
→ 同一deploymentのPodがworker02で複数稼働中
$ kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE    IP            NODE       NOMINATED NODE   READINESS GATES
nginx-76ccf9dd9d-mw58c   1/1     Running   0          5m1s   10.42.1.186   worker01   <none>           <none>
nginx-76ccf9dd9d-s9sdx   1/1     Running   0          5m1s   10.42.2.113   worker02   <none>           <none>
nginx-76ccf9dd9d-zvxdx   1/1     Running   0          113s   10.42.2.116   worker02   <none>           <none>

## worker03ノードをスケジューリング可能に変更
$ kubectl uncordon worker03

## ノードステータス確認
$ kubectl get nodes
NAME       STATUS   ROLES               AGE   VERSION
master01   Ready    controlplane,etcd   12d   v1.19.7
worker01   Ready    worker              12d   v1.19.7
worker02   Ready    worker              12d   v1.19.7
worker03   Ready    worker              12d   v1.19.7
~~~

 - descheduler job作成<br>
マニフェストは以下参照<br>
https://github.com/kubernetes-sigs/descheduler/tree/master/kubernetes

~~~yaml:descheduler.yaml
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: descheduler-cluster-role
rules:
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "update"]
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "watch", "list"]
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list", "delete"]
- apiGroups: [""]
  resources: ["pods/eviction"]
  verbs: ["create"]
- apiGroups: ["scheduling.k8s.io"]
  resources: ["priorityclasses"]
  verbs: ["get", "watch", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: descheduler-sa
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: descheduler-cluster-role-binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: descheduler-cluster-role
subjects:
  - name: descheduler-sa
    kind: ServiceAccount
    namespace: default
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: descheduler-policy-configmap
  namespace: default
data:
  policy.yaml: |
    apiVersion: "descheduler/v1alpha1"
    kind: "DeschedulerPolicy"
    strategies:
# 「RemoveDuplicates」を有効にする
      "RemoveDuplicates":
         enabled: true
---
apiVersion: batch/v1
kind: Job
metadata:
  name: descheduler-job
  namespace: default
spec:
  parallelism: 1
  completions: 1
  template:
    metadata:
      name: descheduler-pod
    spec:
      priorityClassName: system-cluster-critical
      containers:
        - name: descheduler
          image: k8s.gcr.io/descheduler/descheduler:v0.20.0
          volumeMounts:
          - mountPath: /policy-dir
            name: policy-volume
          command:
            - "/bin/descheduler"
          args:
            - "--policy-config-file"
            - "/policy-dir/policy.yaml"
            - "--v"
            - "3"
          resources:
            requests:
              cpu: "500m"
              memory: "256Mi"
          securityContext:
            runAsUser: 1001 # 実行ユーザエラーが出て設定追加
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
            runAsNonRoot: true
      restartPolicy: "Never"
      serviceAccountName: descheduler-sa
      volumes:
      - name: policy-volume
        configMap:
          name: descheduler-policy-configmap
~~~

 - Job実行

~~~
## Job実行
$ kubectl apply -f descheduler.yaml

## 完了確認
$ kubectl get job
NAME              COMPLETIONS   DURATION   AGE
descheduler-job   1/1           1s         18s

$ kubectl get pod
NAME                     READY   STATUS      RESTARTS   AGE
descheduler-job-qnp5r    0/1     Completed   0          59s
..........
~~~

 - Pod確認<br>
1Pod/1Nodeに戻っていることを確認

~~~
$ kubectl get pod -o wide
NAME                     READY   STATUS      RESTARTS   AGE     IP            NODE       NOMINATED NODE   READINESS GATES
nginx-76ccf9dd9d-kz8x2   1/1     Running     0          73s     10.42.3.113   worker03   <none>           <none>
nginx-76ccf9dd9d-mw58c   1/1     Running     0          7m39s   10.42.1.186   worker01   <none>           <none>
nginx-76ccf9dd9d-s9sdx   1/1     Running     0          7m39s   10.42.2.113   worker02   <none>           <none>

~~~