kubeadmでFlatcar Container LinuxにKubernetes 1.17

[kubeadmでFlatcar Container LinuxにKubernetes 1.18を入れる場合はこちら](https://qiita.com/rururu_kenken/items/1a568b74dd46668c6029)


# Master Node

|    EC2　　 |    OS         | Kubernetes | Docker    |
|:----------:|:-------------:|:----------:|:---------:|
| t3.small   | Flatcar Container Linux stable 2303.4.0| 1.17.3|18.06.3-ce |

Flatcar Container Linux AMI のユーザー名は core です

Master Nodeは、2CPU、メモリ2GB以下の場合、kubeadm init実行時にエラーになります。
ディスクサイズはデフォルトの8G

Master NodeとWorker Nodeは同じVPC
Master NodeのIPアドレスは172.31.14.18

# SELinux設定確認

```
$ getenforce

# SELinuxは有効だが、アクセス制限は行わず警告を出力
Permissive
```

## swap確認

```
# ないので何もしない。ある場合はswapoff -a
$ free

              total        used        free      shared  buff/cache   available
Mem:        2001376       82700     1498628      204416      420048     1572528
Swap:             0           0           0
```

# Docker設定

```
$ docker -v

Docker version 18.06.3-ce, build d7080c1

# Dockerの自動起動有効
$ sudo systemctl enable docker

# daemon.json

$ sudo mkdir -p /etc/docker
$ sudo tee /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

$ sudo mkdir -p /etc/systemd/system/docker.service.d
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
```

# kubelet、kubeadm、kubectlインストール

CNI plugins (required for most pod network)

[plugins](https://github.com/containernetworking/plugins)

```
$ CNI_VERSION="v0.8.5"
$ sudo mkdir -p /opt/cni/bin
$ curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | sudo tar -C /opt/cni/bin -xz

$ ls /opt/cni/bin

bandwidth  bridge  dhcp  firewall  flannel  host-device  host-local  ipvlan  loopback  macvlan  portmap  ptp  sbr  static  tuning  vlan
```

crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))

[cri-tools](https://github.com/kubernetes-sigs/cri-tools)

```
$ CRICTL_VERSION="v1.17.0"
$ sudo mkdir -p /opt/bin
$ curl -L "https://github.com/kubernetes-incubator/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | sudo tar -C /opt/bin -xz

$ ls /opt/bin

crictl
```

kubeadm, kubelet, kubectl

```
$ RELEASE="v1.17.3"
$ sudo mkdir -p /opt/bin
$ cd /opt/bin
$ sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
$ sudo chmod +x {kubeadm,kubelet,kubectl}

$ ls

crictl  kubeadm  kubectl  kubelet

$ cd
$ curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > ./kubelet.service
$ sudo mv ./kubelet.service /etc/systemd/system/

$ sudo mkdir -p /etc/systemd/system/kubelet.service.d
$ curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > ./10-kubeadm.conf
$ sudo mv 10-kubeadm.conf /etc/systemd/system/kubelet.service.d
```

# kubelet 起動

```
$ sudo systemctl enable --now kubelet
```

# Masterノードのセットアップ

[kubernetes 1.17.0 - kubeadm init - kube-controller-manager status is ContainerCreating #86961](https://github.com/kubernetes/kubernetes/issues/86961)

```
$ cat <<EOF > ./kubeadm-custom.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.17.3
controllerManager:
  extraArgs:
    flex-volume-plugin-dir: "/etc/kubernetes/kubelet-plugins/volume/exec"
networking:
  podSubnet: 10.244.0.0/16
EOF

$ sudo kubeadm init --config kubeadm-custom.yaml
```

## kubectl 接続設定

```
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
```

# 確認

```
$ kubectl get pod --all-namespaces

NAMESPACE     NAME                                                                      READY   STATUS    RESTARTS   AGE
kube-system   coredns-6955765f44-cd8vf                                                  0/1     Pending   0          2m5s
kube-system   coredns-6955765f44-l9ztf                                                  0/1     Pending   0          2m5s
kube-system   etcd-ip-172-31-14-18.ap-northeast-1.compute.internal                      1/1     Running   0          2m20s
kube-system   kube-apiserver-ip-172-31-14-18.ap-northeast-1.compute.internal            1/1     Running   0          2m20s
kube-system   kube-controller-manager-ip-172-31-14-18.ap-northeast-1.compute.internal   1/1     Running   0          2m20s
kube-system   kube-proxy-9md7c                                                          1/1     Running   0          2m5s
kube-system   kube-scheduler-ip-172-31-14-18.ap-northeast-1.compute.internal            1/1     Running   0          2m19s
```

# Flannelデプロイ

[Super slow access to service IP from host (& host-networked pods) with Flannel CNI #1245](https://github.com/coreos/flannel/issues/1245)

```
$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```

```
$ kubectl get pod --all-namespaces

NAMESPACE     NAME                                                                      READY   STATUS    RESTARTS   AGE
kube-system   coredns-6955765f44-cd8vf                                                  1/1     Running   0          3m30s
kube-system   coredns-6955765f44-l9ztf                                                  1/1     Running   0          3m30s
kube-system   etcd-ip-172-31-14-18.ap-northeast-1.compute.internal                      1/1     Running   0          3m45s
kube-system   kube-apiserver-ip-172-31-14-18.ap-northeast-1.compute.internal            1/1     Running   0          3m45s
kube-system   kube-controller-manager-ip-172-31-14-18.ap-northeast-1.compute.internal   1/1     Running   0          3m45s
kube-system   kube-flannel-ds-amd64-4dzrg                                               1/1     Running   0          27s
kube-system   kube-proxy-9md7c                                                          1/1     Running   0          3m30s
kube-system   kube-scheduler-ip-172-31-14-18.ap-northeast-1.compute.internal            1/1     Running   0          3m44s

$ kubectl get node

NAME                                              STATUS   ROLES    AGE     VERSION
ip-172-31-14-18.ap-northeast-1.compute.internal   Ready    master   4m25s   v1.17.3

$ kubectl get cs  

NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-0               Healthy   {"health":"true"} 
```

# WorkerNodeを構築する場合

以下はMaster Nodeの手順を参照
・SELinux設定確認
・swap確認
・Docker設定
・kubelet、kubeadm、kubectlインストール
・kubelet 起動

クラスタに参加

```
# Master Nodeで実行
$ kubeadm token create --print-join-command

# WorkerNodeで生成されたコマンドに先頭に「sudo 」をつけて実行
# 長いので「・・・」で省略
$ sudo kubeadm join ・・・
```

# 参考URL

[Flatcar Container Linux](https://www.flatcar-linux.org/)
[Flatcar Container Linux Documentation](https://docs.flatcar-linux.org/)

[kubeadmのインストール](https://kubernetes.io/ja/docs/setup/independent/install-kubeadm/)
[kubeadmのトラブルシューティング](https://kubernetes.io/ja/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/)