EFK インストール

# 概要
 - k8sテスト環境構築<br>
<b>EFK インストール</b>

# 構築目次
 - [全体目次] (https://qiita.com/prodigy413/items/89fcb8cfb71e80f541d8)

# 環境
 - Rancher: v2.4.8
 - kubernetes(Client): v1.19.1
 - kubernetes(Server): v1.18.8
 - ECK(Elastic Cloud on Kubernetes): v1.2.1
 - Elasticsearch: v7.9.2
 - Kibana: v7.9.2
 - Fluentd: v1.9.3

# ECK インストール
 - 作業場所: ClientPC
 - Elasticsearchとkibanaは<b>ECK(Elastic Cloud on Kubernetes)</b>使用  
https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html


### ECK operator インストール
※namespaceは自動作成

~~~
$ kubectl apply -f https://download.elastic.co/downloads/eck/1.2.1/all-in-one.yaml
customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created
namespace/elastic-system created
serviceaccount/elastic-operator created
secret/elastic-webhook-server-cert created
clusterrole.rbac.authorization.k8s.io/elastic-operator created
clusterrole.rbac.authorization.k8s.io/elastic-operator-view created
clusterrole.rbac.authorization.k8s.io/elastic-operator-edit created
clusterrolebinding.rbac.authorization.k8s.io/elastic-operator created
rolebinding.rbac.authorization.k8s.io/elastic-operator created
service/elastic-webhook-server created
statefulset.apps/elastic-operator created
validatingwebhookconfiguration.admissionregistration.k8s.io/elastic-webhook.k8s.elastic.co created

## 確認 ##
$ kubectl get crd | grep -i elastic
apmservers.apm.k8s.elastic.co                        2020-09-27T15:32:37Z
beats.beat.k8s.elastic.co                            2020-09-27T15:32:37Z
elasticsearches.elasticsearch.k8s.elastic.co         2020-09-27T15:32:37Z
enterprisesearches.enterprisesearch.k8s.elastic.co   2020-09-27T15:32:37Z
kibanas.kibana.k8s.elastic.co                        2020-09-27T15:32:37Z

$ kubectl get statefulset -n elastic-system
NAME               READY   AGE
elastic-operator   1/1     62s

$ kubectl get pod -n elastic-system
NAME                 READY   STATUS    RESTARTS   AGE
elastic-operator-0   1/1     Running   0          61s

$ kubectl get svc -n elastic-system
NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
elastic-webhook-server   ClusterIP   10.43.201.47   <none>        443/TCP   62s
~~~

### Elasticsearch インストール
 - Manifest作成

~~~yaml:elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: test-elastic
  namespace: elastic-system
spec:
  version: 7.9.2
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
    podTemplate:
      spec:
        volumes:
        - name: elasticsearch-data
          emptyDir: {}
~~~
 - Deploy

~~~
$ kubectl apply -f elasticsearch.yaml
elasticsearch.elasticsearch.k8s.elastic.co/test-elastic created

## 確認 ##
$ kubectl get elasticsearch -n elastic-system
NAME           HEALTH   NODES   VERSION   PHASE   AGE
test-elastic   green    1       7.9.2     Ready   3m5s

$ kubectl get statefulset -n elastic-system
NAME                      READY   AGE
..........
test-elastic-es-default   1/1     13m
..........

$ kubectl get pod -n elastic-system
NAME                        READY   STATUS    RESTARTS   AGE
..........
test-elastic-es-default-0   1/1     Running   0          2m5s
..........

$ kubectl get svc -n elastic-system
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
..........
test-elastic-es-default     ClusterIP   None           <none>        9200/TCP   5m22s
test-elastic-es-http        ClusterIP   10.43.136.65   <none>        9200/TCP   5m23s
test-elastic-es-transport   ClusterIP   None           <none>        9300/TCP   5m23s
..........
~~~

### Kibana インストール
 - Manifest作成

~~~yaml:kibana.yaml
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: test-kibana
  namespace: elastic-system
spec:
  version: 7.9.2
  count: 1
  elasticsearchRef:
    name: test-elastic
~~~
 - Deploy

~~~
$ kubectl apply -f kibana.yaml
kibana.kibana.k8s.elastic.co/test-kibana created

## 確認 ##
$ kubectl get kibana -n elastic-system
NAME          HEALTH   NODES   VERSION   AGE
test-kibana   green    1       7.9.2     2m56s

$ kubectl get deploy -n elastic-system
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
test-kibana-kb   1/1     1            1           2m25s

$ kubectl get pod -n elastic-system
NAME                             READY   STATUS    RESTARTS   AGE
..........
test-kibana-kb-c478fcc7c-rgm6z   1/1     Running   0          10m
..........

$ kubectl get svc -n elastic-system
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
..........
test-kibana-kb-http         ClusterIP   10.43.222.22   <none>        5601/TCP   2m40s
..........
~~~

### ネットワーク設定
 - Nginx Ingress 設定<br>
- KibanaにHTTPSでアクセスするため、Nginx Ingressの`passthrough`設定が必要<br>
- Ingressを`passthrough`なしでインストールした場合は以下実行

~~~
$ helm upgrade nginx-ingress ingress-nginx/ingress-nginx -n ingress-system --set "controller.extraArgs.enable-ssl-passthrough="
~~~

 - Kibanaアクセス用Ingress追加

~~~yaml:ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kibana-ingress
  namespace: elastic-system
  annotations:
    # passthrough設定追加
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
  rules:
  - host: kibana.test.local
    http:
      paths:
      - path: /
        backend:
          serviceName: test-kibana-kb-http
          servicePort: 5601
~~~

 - Deploy

~~~
$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/kibana-ingress created

## 確認 ##
$ kubectl get ingress -n elastic-system
NAME             CLASS    HOSTS               ADDRESS           PORTS   AGE
kibana-ingress   <none>   kibana.test.local   192.168.245.111   80      105s
~~~


 - hosts 設定追加  
Ingress用IPに「kibana.test.local」に追加

~~~
$ cat /etc/hosts
........
192.168.245.111 kibana.test.local
........
~~~

### Kibana アクセス
 - パスワード確認

~~~
$ kubectl get secret test-elastic-es-elastic-user -o=jsonpath='{.data.elastic}' -n elastic-system | base64 --decode; echo
35XJBcQ7i1dov0H3Q727u1aU
~~~

 - Kibana アクセス  
URL : <b>https://kibana.test.local</b><br>
Username: <b>elastic</b><br>
Password: 上記で確認したパスワード<br>
![1-1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/87138aa2-6a46-17bb-b88a-45811030dc6f.png)<br>
![2-1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/cc0ef560-7934-6060-f2ff-6f57967c380f.png)<br>
![3-1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/561025/2cc1d2bd-e310-c52b-3839-09e5f04fe489.png)

# Fluentd 設定
 - 作業場所: ClientPC
 - fluentd設定参考リンク<br>
https://github.com/fluent/fluentd-kubernetes-daemonset<br>
https://github.com/fluent/fluentd-kubernetes-daemonset/blob/master/fluentd-daemonset-elasticsearch-rbac.yaml<br>
https://github.com/joshuarobinson/elasticsearch_k8s_examples/blob/master/fluentd-daemonset-elasticsearch.yaml

### Fluentd インストール
 - インストール用manifest作成  

~~~yaml:fluentd-daemonset-elasticsearch.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: elastic-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: fluentd
  namespace: elastic-system
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - namespaces
  verbs:
  - get
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: fluentd
  namespace: elastic-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: elastic-system
  labels:
    k8s-app: fluentd-logging
    version: v1
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-logging
      version: v1
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
        version: v1
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
        env:
          - name: FLUENT_UID
            value: "0"
          - name: FLUENTD_SYSTEMD_CONF
            value: "disable"
          - name:  FLUENT_ELASTICSEARCH_HOST
            value: "test-elastic-es-http"
          - name:  FLUENT_ELASTICSEARCH_PORT
            value: "9200"
          - name: FLUENT_ELASTICSEARCH_SCHEME
            value: "https"
          - name: FLUENT_ELASTICSEARCH_SSL_VERIFY
            value: "false"
          - name: FLUENT_ELASTICSEARCH_SSL_VERSION
            value: "TLSv1_2"
          - name: FLUENT_ELASTICSEARCH_USER
            value: "elastic"
          - name: FLUENT_ELASTICSEARCH_PASSWORD
            valueFrom:
              secretKeyRef:
                name: test-elastic-es-elastic-user
                key: elastic
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
~~~

 - Deploy

~~~
$ kubectl apply -f fluentd-daemonset-elasticsearch.yaml

## 確認 ##
$ kubectl get pod -n elastic-system
NAME                              READY   STATUS    RESTARTS   AGE
......
fluentd-22crn                     1/1     Running   0          4m24s
fluentd-pcxqt                     1/1     Running   0          4m24s
fluentd-w9fvg                     1/1     Running   0          4m24s
......
~~~