Kubernetesの構築

# Kubernetesの構築
(2018/11時点)
(2021/03更新)

## 環境
- OS: CentOS 7.4
- Kubernetes: v1.12
- Docker: docker-ce-18.06.1.ce
- Cluster構成: Master Node - 1台, Worker Node - n台
- Podネットワークアドオン: Flannel

## 手順
###1. Dockerのインストール (Master/Worker共通)
```sh:bash
# yum install yum-utils device-mapper-persistent-data lvm2
# yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
# yum update && yum install docker-ce-18.06.1.ce
# mkdir /etc/docker
# cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

# systemctl enable docker
# systemctl daemon-reload
# systemctl restart docker
```
###2. Swapの無効化 (Master/Worker共通)
``` bash
# swapoff -a
```
```vim:/etc/fstab
# /dev/mapper/xxx-swap swap                    swap    defaults        0 0
```
swap行をコメントアウト

###3. Firewalldの設定
- Master Node

``` bash
# firewall-cmd --add-port=6443/tcp --zone=public --permanent
# firewall-cmd --add-port=2379-2380/tcp --zone=public --permanent
# firewall-cmd --add-port=10250-10252/tcp --zone=public --permanent
# firewall-cmd --reload
```
- Worker Node

``` bash
# firewall-cmd --add-port=10250/tcp --zone=public --permanent
# firewall-cmd --add-port=30000-32767/tcp --zone=public --permanent
# firewall-cmd --reload
```
- (必要があれば)

```bash
# cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

# cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

# sysctl --system
```
###4. SELinuxの停止 (Master/Worker共通)
``` bash
# sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config
# setenforce 0
```
###5. kubeadm, kubelet, kubectlのインストール (Master/Worker共通)
``` bash
# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
# systemctl enable kubelet --now kubelet
```
###6. Cluster Master構築 (Masterのみ)
``` bash
# kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=<x.x.x.x>
```
※ --pod-network-cidr: Podに割り振られるネットワークの指定, Flannelを使用する場合は10.244.0.0/16」を指定
※ --apiserver-advertise-address: Master Nodeに設定されているIPアドレスを指定
※ kubeadm実行時に出力された"kubeadm join --toke..."を取得、Worker NodeをClusterに追加するために使用

``` bash
export KUBECONFIG=/etc/kubernetes/admin.conf
```
###7. Podネットワークアドオンのインストール (Masterのみ)
``` bash
# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```
###8. Master Nodeの隔離 (Masterのみ)
``` bash
# kubectl taint nodes --all node-role.kubernetes.io/master-
```

###9. Worker Nodeの追加 (Workerのみ)
``` bash
＃kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
```
※ 手順4で取得したコマンドを実行

###10. Cluster Nodeの確認 (Master上で)
``` bash
# kubectl get node -o wide
NAME       STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION               CONTAINER-RUNTIME
master00   Ready    master   21d   v1.12.2   x.x.x.x   <none>        CentOS Linux 7 (Core)   3.10.0-862.14.4.el7.x86_64   docker://18.6.1
worker01   Ready    <none>   21d   v1.12.2   x.x.x.y   <none>        CentOS Linux 7 (Core)   3.10.0-862.14.4.el7.x86_64   docker://18.6.1
worker02   Ready    <none>   21d   v1.12.2   x.x.x.z   <none>        CentOS Linux 7 (Core)   3.10.0-862.14.4.el7.x86_64   docker://18.6.1
```

## 参考
- https://kubernetes.io/docs/setup/independent/install-kubeadm/
- https://kubernetes.io/docs/setup/cri/