kubernetes インストール



まずは以下のサイトを参考に
https://www.server-world.info/query?os=CentOS_7&p=kubernetes

次に以下のサイトを参考に
https://qiita.com/yasubehe/items/51a05688d1f66d51531e

## Dockerのバージョン

```
[root@kube1 ~]# docker version
Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:33:07 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:10 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
```

## kubeadm initを実行すると以下のエラー内容

このエラーは②にかかわっていると思う。

```
  [ERROR CRI]: container runtime is not running: output: time="2023-08-14T04:20:42-04:00" level=fatal msg="validate service connection: CRI v1 runtime API is not implemented for endpoint \"unix:///var/run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
```




それ以外で修正が必要だった箇所
cat /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf



## 修正箇所 Podでデプロイするすべてで実行する

### ①10-kubeadm.conf

vi /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
以下の文字列を追加
--cgroup-driver=systemd


https://kubernetes.io/ja/docs/setup/production-environment/container-runtimes/

The approach to mitigate this instability is to use systemd as the cgroup driver for the kubelet and the container runtime when systemd is the selected init system.

systemdがinit systemで選択されているとき
この不安定を軽減するアプローチはcgroup driver（kubeletとコンテナランタイム（ここではdocker)）にsystemdを使用すること

理由：
systemd has a tight integration with cgroups and allocates a cgroup per systemd unit. As a result, if you use systemd as the init system with the cgroupfs driver, the system gets two different cgroup managers.

システムに2つの異なったcgroup managerが存在してしまう。
リソースを割り当てるcgroupsが二つ存在することを回避するため
上記設定が必要になる。
※cgroupsが２つ存在すると、リソース管理するプロセスが２つ存在することになる
→結果としてシステムが不安定になるってことを言いたいみたい。

```
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS --cgroup-driver=systemd
```
その後
systemctl daemon-reload
systemctl stop kubelet
systemctl start kubelet

### ②config.tomlの修正

どのコンテナランタイムで動作しているんだっけ？？

https://kubernetes.io/ja/docs/tasks/administer-cluster/migrating-from-dockershim/find-out-runtime-you-use/

/etc/containerd/config.toml
コメントアウトをつける

```
#disabled_plugins = ["cri"]
```
systemctl stop containerd
systemctl start containerd


### ③

事前に現在の設定を確認する

docker version
docker info | grep -i driver

https://qiita.com/NaotoHanaue/items/b9f395a63f7aae2a41b7

 /etc/docker/daemon.json
https://devlights.hatenablog.com/entry/2021/11/30/010150

```
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
```
systemctl daemon-reload
systemctl restart docker


### kubeadm initの実行

```
[root@kube1 ~]# kubeadm init --apiserver-advertise-address=192.168.2.81 --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.28.2
[preflight] Running pre-flight checks
        [WARNING FileExisting-tc]: tc not found in system path
        [WARNING Hostname]: hostname "kube1" could not be reached
        [WARNING Hostname]: hostname "kube1": lookup kube1 on 192.168.2.1:53: no such host
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W1009 01:06:41.183364   12410 checks.go:835] detected that the sandbox image "registry.k8s.io/pause:3.6" of the container runtime is inconsistent with that used by kubeadm. It is recommended that using "registry.k8s.io/pause:3.9" as the CRI sandbox image.
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kube1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.2.81]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kube1 localhost] and IPs [192.168.2.81 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kube1 localhost] and IPs [192.168.2.81 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 6.001106 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node kube1 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kube1 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: h0z32j.wzk0u4svj3atjyvc
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.2.81:6443 --token h0z32j.wzk0u4svj3atjyvc \
        --discovery-token-ca-cert-hash sha256:667c7930cc45f64fd3733b833501121d5e896714cc48a453e79099bbeb07b70c
[root@kube1 ~]#
```


### どのコンテナランタイムで動作しているかわかる。



```

[root@kube1 ~]# kubectl get nodes -o wide
NAME    STATUS     ROLES           AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE          KERNEL-VERSION          CONTAINER-RUNTIME
kube1   Ready      control-plane   55d   v1.28.2   192.168.2.81   <none>        CentOS Stream 8   4.18.0-326.el8.x86_64   containerd://1.6.24
kube2   NotReady   <none>          55d   v1.28.2   192.168.2.82   <none>        CentOS Stream 8   4.18.0-326.el8.x86_64   containerd://1.6.24
kube3   Ready      <none>          55d   v1.28.2   192.168.2.83   <none>        CentOS Stream 8   4.18.0-326.el8.x86_64   containerd://1.6.24

```