Linkerd インストール

# 概要
 - k8sテスト環境構築<br>
<b>Linkerd インストール</b><br><br>

# 構築目次
 - [全体目次] (https://qiita.com/prodigy413/items/89fcb8cfb71e80f541d8)

# 環境
 - Rancher: v2.5.3
 - kubernetes(Client): v1.19.4
 - kubernetes(Server): v1.19.4
 - Linkerd: v2.9.1

## インストール
#### 1. linkerd コマンド ダウンロード
 - 作業場所: ClientPC<br>
 - Linkerd Release Page<br>
https://github.com/linkerd/linkerd2/releases/

~~~
$ sudo wget https://github.com/linkerd/linkerd2/releases/download/stable-2.9.1/linkerd2-cli-stable-2.9.1-linux-amd64 -O /usr/local/bin/linkerd
$ sudo chmod +x /usr/local/bin/linkerd

## version 確認 ##
$ linkerd version
Client version: stable-2.9.1
Server version: unavailable
~~~

#### 2. Linkerd インストール
 - Linkerd Install Page
https://linkerd.io/2/tasks/install/
https://linkerd.io/2/reference/cli/install/

 - 事前環境チェック

~~~
$ linkerd check --pre
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

pre-kubernetes-setup
--------------------
√ control plane namespace does not already exist
W0103 18:09:21.089692    9491 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0103 18:09:21.288645    9491 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0103 18:09:21.890054    9491 warnings.go:67] admissionregistration.k8s.io/v1beta1 MutatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 MutatingWebhookConfiguration
W0103 18:09:22.488437    9491 warnings.go:67] admissionregistration.k8s.io/v1beta1 ValidatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 ValidatingWebhookConfiguration
√ can create non-namespaced resources
√ can create ServiceAccounts
√ can create Services
√ can create Deployments
√ can create CronJobs
√ can create ConfigMaps
√ can create Secrets
√ can read Secrets
√ can read extension-apiserver-authentication configmap
√ no clock skew detected

pre-kubernetes-capability
-------------------------
√ has NET_ADMIN capability
√ has NET_RAW capability

linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date

Status check results are √
~~~

 - Linkerd インストール

~~~
## Namespace(linkerd)は自動作成される
$ linkerd install | kubectl apply -f -
## replicasを変更する場合は以下のように設定
$ linkerd install --controller-replicas 2 | kubectl apply -f -

## 確認 ##
$ kubectl get deploy -n linkerd
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
linkerd-controller       2/2     2            2           2m2s
linkerd-destination      2/2     2            2           2m1s
linkerd-grafana          1/1     1            1           2m
linkerd-identity         2/2     2            2           2m2s
linkerd-prometheus       1/1     1            1           2m
linkerd-proxy-injector   2/2     2            2           2m1s
linkerd-sp-validator     2/2     2            2           2m1s
linkerd-tap              2/2     2            2           2m
linkerd-web              1/1     1            1           2m1s

$ kubectl get pod -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-controller-75c677bbb-gr2mv        2/2     Running   0          2m18s
linkerd-controller-75c677bbb-plrdb        2/2     Running   0          2m17s
linkerd-destination-759488cd56-5476f      2/2     Running   0          2m17s
linkerd-destination-759488cd56-cbfsp      2/2     Running   0          2m17s
linkerd-grafana-75fbbf55cb-lj85w          2/2     Running   0          2m16s
linkerd-identity-79b444c8b8-fp7sm         2/2     Running   0          2m18s
linkerd-identity-79b444c8b8-m2q2l         2/2     Running   0          2m18s
linkerd-prometheus-768fff8db8-pwwdc       2/2     Running   0          2m16s
linkerd-proxy-injector-8488f8fc9f-2fbqk   2/2     Running   0          2m17s
linkerd-proxy-injector-8488f8fc9f-bln9v   2/2     Running   0          2m17s
linkerd-sp-validator-69d6c7f868-rlxzt     2/2     Running   0          2m17s
linkerd-sp-validator-69d6c7f868-tl2x4     2/2     Running   0          2m17s
linkerd-tap-7578d8b6b7-4pf24              2/2     Running   0          2m16s
linkerd-tap-7578d8b6b7-65rbf              2/2     Running   0          2m16s
linkerd-web-6dbdf7b677-95s6n              2/2     Running   0          2m17s

$ kubectl get svc -n linkerd
NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
linkerd-controller-api      ClusterIP   10.43.126.44    <none>        8085/TCP            2m41s
linkerd-dst                 ClusterIP   10.43.190.104   <none>        8086/TCP            2m40s
linkerd-dst-headless        ClusterIP   None            <none>        8086/TCP            2m40s
linkerd-grafana             ClusterIP   10.43.13.127    <none>        3000/TCP            2m39s
linkerd-identity            ClusterIP   10.43.91.158    <none>        8080/TCP            2m41s
linkerd-identity-headless   ClusterIP   None            <none>        8080/TCP            2m41s
linkerd-prometheus          ClusterIP   10.43.238.35    <none>        9090/TCP            2m39s
linkerd-proxy-injector      ClusterIP   10.43.95.243    <none>        443/TCP             2m40s
linkerd-sp-validator        ClusterIP   10.43.125.35    <none>        443/TCP             2m40s
linkerd-tap                 ClusterIP   10.43.250.180   <none>        8088/TCP,443/TCP    2m40s
linkerd-web                 ClusterIP   10.43.130.61    <none>        8084/TCP,9994/TCP   2m40s
~~~

 - 事後チェック

~~~
$ linkerd check
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
√ controller pod is running
√ can initialize the client
√ can query the control plane API

linkerd-config
--------------
√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
W0103 21:50:49.836339   11834 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
√ control plane CustomResourceDefinitions exist
W0103 21:50:49.847975   11834 warnings.go:67] admissionregistration.k8s.io/v1beta1 MutatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 MutatingWebhookConfiguration
√ control plane MutatingWebhookConfigurations exist
W0103 21:50:49.859010   11834 warnings.go:67] admissionregistration.k8s.io/v1beta1 ValidatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 ValidatingWebhookConfiguration
√ control plane ValidatingWebhookConfigurations exist
√ control plane PodSecurityPolicies exist

linkerd-identity
----------------
√ certificate config is valid
√ trust anchors are using supported crypto algorithm
√ trust anchors are within their validity period
√ trust anchors are valid for at least 60 days
√ issuer cert is using supported crypto algorithm
√ issuer cert is within its validity period
√ issuer cert is valid for at least 60 days
√ issuer cert is issued by the trust anchor

linkerd-webhooks-and-apisvc-tls
-------------------------------
√ tap API server has valid cert
√ tap API server cert is valid for at least 60 days
W0103 21:50:49.933442   11834 warnings.go:67] admissionregistration.k8s.io/v1beta1 MutatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 MutatingWebhookConfiguration
√ proxy-injector webhook has valid cert
√ proxy-injector cert is valid for at least 60 days
W0103 21:50:50.240829   11834 warnings.go:67] admissionregistration.k8s.io/v1beta1 ValidatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 ValidatingWebhookConfiguration
√ sp-validator webhook has valid cert
√ sp-validator cert is valid for at least 60 days

linkerd-api
-----------
√ control plane pods are ready
√ control plane self-check
√ [kubernetes] control plane can talk to Kubernetes
√ [prometheus] control plane can talk to Prometheus
√ tap api service is running

linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date

control-plane-version
---------------------
√ control plane is up-to-date
√ control plane and cli versions match
W0103 21:50:51.247437   11834 warnings.go:67] admissionregistration.k8s.io/v1beta1 MutatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 MutatingWebhookConfiguration

linkerd-prometheus
------------------
√ prometheus add-on service account exists
√ prometheus add-on config map exists
√ prometheus pod is running

linkerd-grafana
---------------
√ grafana add-on service account exists
√ grafana add-on config map exists
√ grafana pod is running

Status check results are √
~~~

 - Dashboard 確認

~~~
$ linkerd dashboard
~~~

![1-1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/dbf1fe65-2444-e6a9-502a-3ab039cf1b19.png)

 - Grafana 確認<br>
Namespace `linkerd`のGrafanaマークをクリック<br>
![1-2.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/48839c79-602f-243a-dc18-469e3b7ae24b.png)<br>
Grafana 確認<br>
![2-1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/9074fcd3-b4fa-72ed-cda6-5ea38e8f6bb6.png)


## 動作確認
 - Namespace 設定<br>
Namespaceに`Proxy Injection`設定を追加<br>

~~~
## テスト用namespace作成
$ kubectl create ns linkerd-test

## 「linkerd-test」namespaceに「linkerd.io/inject=enabled」annotationを追加 ##
$ kubectl annotate namespace linkerd-test linkerd.io/inject=enabled

## 確認 ##
$ kubectl describe ns linkerd-test | grep -i linkerd.io
              linkerd.io/inject: enabled
~~~

 - サンプルPod作成

~~~yaml:nginx-linkerd-test.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: linkerd-test
  name: nginx-svc
spec:
  selector:
    app: nginx
  ports:
  - name: port
    port: 8080
    targetPort: 80
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
  namespace: linkerd-test
spec:
  containers:
  - name: nginx
    image: nginx:1.19.2
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
~~~

~~~
$ kubectl apply -f nginx-linkerd-test.yaml

## 確認 ##
$ kubectl get pod -n linkerd-test
NAME    READY   STATUS    RESTARTS   AGE
nginx   2/2     Running   0          20s

$ kubectl -n linkerd-test get pod nginx -o jsonpath='{.spec.containers[*].name}'
nginx1 linkerd-proxy
~~~

<br><b>Linkerdは専用Ingressを提供しないため、使用中のIngressを利用</b>

 - nginx ingress controllerにproxy追加

~~~
## ingress controller名確認
$ kubectl -n ingress-system get deploy
NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
nginx-ingress-ingress-nginx-controller   1/1     1            1           11d

## controllerにproxy追加
$ kubectl get deployment nginx-ingress-ingress-nginx-controller -n ingress-system -o yaml | linkerd inject --ingress - | kubectl apply -f -

## proxy追加後、確認
$ kubectl -n ingress-system get pod
NAME                                                      READY   STATUS    RESTARTS   AGE
nginx-ingress-ingress-nginx-controller-6d69c9499f-hz26w   2/2     Running   0          44s

$ kubectl get pod nginx-ingress-ingress-nginx-controller-6d69c9499f-hz26w -n ingress-system -o jsonpath='{.spec.containers[*].name}'
controller linkerd-proxy
~~~

 - Ingress 作成

~~~yaml:test-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: "nginx"
# linkerdの各機能を使うため、以下annotaionを追加
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
      grpc_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
  name: nginx-ingress
  namespace: linkerd-test
spec:
  rules:
  - host: nginx-linkerd.test.local
    http:
      paths:
      - path: /
        pathType: Prefix 
        backend:
          service:
            name: nginx-svc
            port:
              number: 8080
~~~

~~~
$ kubectl apply -f test-ingress.yaml
~~~

 - Nginx ingressのLoad Balancer IP(`EXTERNAL-IP`) 確認<br>

~~~
$ kubectl get svc -n ingress-system
NAME                                               TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                      AGE
nginx-ingress-ingress-nginx-controller             LoadBalancer   10.43.195.120   192.168.245.111   80:31499/TCP,443:30127/TCP   20h
nginx-ingress-ingress-nginx-controller-admission   ClusterIP      10.43.35.123    <none>            443/TCP                      20h
~~~

 - hosts 設定追加  
Load BalancerのIPとIngressで設定したhostを紐づけ

~~~
$ cat /etc/hosts
........
192.168.245.111 nginx-linkerd.test.local
........
~~~

 - 接続確認

~~~
$ curl -I http://nginx-linkerd.test.local
HTTP/1.1 200 OK
.........
.........
~~~