Amazon EKS Workshopでk8sとIstioを勉強してみた

#０．はじめに
k8s超初心者な筆者が、k8sとistioを勉強してみるためにEKSでIstioサンプルアプリを動かして、ついでにGrafanaやらJaegerやらKialiやらの良い感じなツールを触ってみたものです。

基本は[こちらのワークショップ](https://eksworkshop.com/)と[Istio公式チュートリアル](https://istio.io/docs/tasks/)をベースに進め、気になる点や汲み取れない部分は個人的に補完するように努めましたが、もし間違った理解をしている部分があればご指摘いただけると幸いです。

#１．クラスター＆ワーカーノードの準備
eksctlを使って、EKSを立てていきます。

###eksctlの導入
```shell:eksctlの導入
$ curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
$ sudo mv /tmp/eksctl /usr/local/bin
$ eksctl version
[?]  version.Info{BuiltAt:"", GitCommit:"", GitTag:"0.1.23"}
```

###eksctlでクラスター＆ワーカーノードを作成
今回は既存のVPC/サブネットを使いたかったので、”--vpc-public-subnets”オプションを指定しています。
（サブネット指定のみでOK）

```shell:cluster&node作成
$ eksctl create cluster \
>   --name=lets-istio \
>   --region=us-west-2 \
>   --nodes=2 \
>   --nodes-min=1 \
>   --nodes-max=2 \
>   --node-type=t3.medium \
>   --version=1.11 \
>   --vpc-public-subnets=subnet-02444fd81a76e520f,subnet-04478d97b94927e41
[?]  using region us-west-2
[?]  using existing VPC (vpc-035904bf2c34af1e1) and subnets (private:[] public:[subnet-04478d97b94927e41 subnet-02444fd81a76e520f])
[!]  custom VPC/subnets will be used; if resulting cluster doesn't function as expected, make sure to review the configuration of VPC/subnets
[?]  nodegroup "ng-cae1ee27" will use "ami-0c28139856aaf9c3b" [AmazonLinux2/1.11]
[?]  creating EKS cluster "lets-istio" in "us-west-2" region
[?]  will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[?]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-west-2 --name=lets-istio'
[?]  creating cluster stack "eksctl-lets-istio-cluster"
[?]  creating nodegroup stack "eksctl-lets-istio-nodegroup-ng-cae1ee27"
[?]  all EKS cluster resource for "lets-istio" had been created
[?]  saved kubeconfig as "/home/ec2-user/.kube/config"
[?]  nodegroup "ng-cae1ee27" has 0 node(s)
[?]  waiting for at least 1 node(s) to become ready in "ng-cae1ee27"
[?]  nodegroup "ng-cae1ee27" has 2 node(s)
[?]  node "ip-10-0-0-4.us-west-2.compute.internal" is ready
[?]  node "ip-10-0-3-130.us-west-2.compute.internal" is ready
[?]  kubectl command should work with "/home/ec2-user/.kube/config", try 'kubectl get nodes'
[?]  EKS cluster "lets-istio" in "us-west-2" region is ready
```

###作成結果を確認
```shell:cluster確認
$ eksctl get cluster
NAME                    REGION
lets-istio      us-west-2
```

```shell:node確認
$ kubectl get nodes
NAME                                       STATUS    ROLES     AGE       VERSION
ip-10-0-0-4.us-west-2.compute.internal     Ready     <none>    54s       v1.11.5
ip-10-0-3-130.us-west-2.compute.internal   Ready     <none>    57s       v1.11.5
```
#２．Istioの導入
###Istioをダウンロード
```shell:Istioをダウンロード&istioctlの導入
$ curl -L https://git.io/getLatestIstio | sh -
$ cd istio-*
$ sudo mv -v bin/istioctl /usr/local/bin/
$ istioctl version
Version: 1.0.6
GitRevision: 98598f88f6ee9c1e6b3f03b652d8e0e3cd114fa2
User: root@464fc845-2bf8-11e9-b805-0a580a2c0506
Hub: docker.io/istio
GolangVersion: go1.10.4
BuildStatus: Clean
```
###Kiali用のsecretを作成
Kialiを有効化するには、事前にSecretを作成しておく必要がある。（無いとapply時にエラーとなってpodが起動しない）

```
$ NAMESPACE=istio-system
$ echo -n 'admin' | base64
YWRtaW4=
$ echo -n '1f2d1e2e67df' | base64
MWYyZDFlMmU2N2Rm
$ cat <<EOF | kubectl apply -f -
> apiVersion: v1
> kind: Secret
> metadata:
>   name: kiali
>   namespace: $NAMESPACE
>   labels:
>     app: kiali
> type: Opaque
> data:
>   username: YWRtaW4=
>   passphrase: MWYyZDFlMmU2N2Rm
> EOF
secret "kiali" created
```

###Istioをインストール
JaegerとKialiはデフォルトでは無効なので、”--set tracing.enabled=true””--set kiali.enabled=true”オプションで有効化する。

```shell:k8sへistioをインストール
$ pwd
/home/ec2-user/istio-1.0.6

#CustomResourceDefinitionのデプロイ
$ kubectl apply -f install/kubernetes/helm/istio/templates/crds.yaml

#service account & namespaceを作成
$ kubectl create -f install/kubernetes/helm/helm-service-account.yaml
$ helm init --service-account tiller
$ kubectl create namespace istio-system

#helmテンプレートでistioをデプロイ
$ helm template install/kubernetes/helm/istio \
>--name istio \
>--namespace istio-system \
>--set global.configValidation=false \
>--set sidecarInjectorWebhook.enabled=true \
>--set grafana.enabled=true \
>--set servicegraph.enabled=true \
>--set tracing.enabled=true \
>--set kiali.enabled=true> istio.yaml
$ kubectl apply -f istio.yaml
$ kubectl get pod -n istio-system
NAME                                      READY     STATUS      RESTARTS   AGE
grafana-7f6cd4bf56-tq5tp                  1/1       Running     0          39s
istio-citadel-796c94878b-s7zc4            1/1       Running     0          38s
istio-cleanup-secrets-p94tx               0/1       Completed   0          40s
istio-egressgateway-864444d6ff-cgjh4      1/1       Running     0          39s
istio-galley-6c68c5dbcf-hg9rx             1/1       Running     0          39s
istio-grafana-post-install-2pcdx          0/1       Completed   0          40s
istio-ingressgateway-694576c7bb-kwsk4     1/1       Running     0          39s
istio-pilot-79f5f46dd5-ngcm6              2/2       Running     0          38s
istio-policy-5bd5578b94-w9sq4             2/2       Running     0          38s
istio-security-post-install-chzxk         0/1       Completed   0          40s
istio-sidecar-injector-6d8f88c98f-gbhtq   1/1       Running     0          37s
istio-telemetry-5598f86cd8-7c7bc          2/2       Running     0          38s
istio-tracing-7596597bd7-t5qjg            1/1       Running     0          37s
kiali-84cd87cd5d-pl29c                    1/1       Running     0          39s
prometheus-76db5fddd5-w2477               1/1       Running     0          38s
servicegraph-df4d5f678-prxhn              1/1       Running     0          37s
```

#３．サンプルアプリケーションをデプロイ
###アプリケーションのデプロイ
ワークショップのIstio編で用意されているサンプルアプリをデプロイします。
※Istioと一緒にダウンロードされています（/home/ec2-user/istio-1.0.6/samples/bookinfo/platform/kube/bookinfo.yaml）

```
#istioctl kube-injectでサイドカー(istio-proxy)を組み込んだManifestを作成し、そのManifestを使ってデプロイする
kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml)

#デプロイ結果を確認
$ kubectl get pod,svc
NAME                                 READY     STATUS    RESTARTS   AGE
pod/details-v1-589c676b6d-pdrwc      2/2       Running   0          17s
pod/productpage-v1-8fc45744b-sxvdq   2/2       Running   0          17s
pod/ratings-v1-6f7f894cd4-hkpfr      2/2       Running   0          17s
pod/reviews-v1-767c6f8477-5x8fs      2/2       Running   0          17s
pod/reviews-v2-5f5d6b9dbd-t6z5s      2/2       Running   0          17s
pod/reviews-v3-865cd5bcc5-g4gwm      2/2       Running   0          17s

NAME                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/details       ClusterIP   172.20.136.178   <none>        9080/TCP   17s
service/kubernetes    ClusterIP   172.20.0.1       <none>        443/TCP    18h
service/productpage   ClusterIP   172.20.64.76     <none>        9080/TCP   17s
service/ratings       ClusterIP   172.20.90.58     <none>        9080/TCP   17s
service/reviews       ClusterIP   172.20.3.26      <none>        9080/TCP   17s
```

各podが2/2となっているので、それぞれpod毎にサイドカーコンテナが併せて起動しているのがわかります。

この段階で、アプリケーションはこんな感じでデプロイされた状態になっており、ポイントは以下2点かと思います。

①各podでistio-initによるiptablesの書き換えが行われ、podへのinbound/outboundが全てistio-proxyコンテナへリダイレクトされるようになっている
②istio-ingressgatewayをデプロイした段階でCLBが作成されているが、現時点ではアプリケーションとの接続はされていないためアクセスできない

![istio-sampleapp-1.png](https://qiita-image-store.s3.amazonaws.com/0/285420/04f7f83b-29c8-65c7-677a-3a36f7acdcf0.png)

※ちなみに、組み込み前後ではManifestはこんな感じに違います。（組み込み後はかなり肥大化してます）

<details><summary>もともとのbookinfo.yaml</summary><div>

```yaml:bookinfo.yaml
# Copyright 2017 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
##################################################################################################
# Details service
apiVersion: v1
kind: Service
metadata:
  name: details
  labels:
    app: details
spec:
  ports:
  - port: 9080
    app: details
kind: Deployment
metadata:
  name: details-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: details
        version: v1
    spec:
      containers:
      - name: details
        image: istio/examples-bookinfo-details-v1:1.8.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Ratings service
apiVersion: v1
kind: Service
metadata:
  name: ratings
  labels:
    app: ratings
spec:
  ports:
  - port: 9080
    app: ratings
kind: Deployment
metadata:
  name: ratings-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      containers:
      - name: ratings
        image: istio/examples-bookinfo-ratings-v1:1.8.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Reviews service
apiVersion: v1
kind: Service
metadata:
  name: reviews
  labels:
    app: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: reviews-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v1
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v1:1.8.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: reviews-v2
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v2
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v2:1.8.0
        - containerPort: 9080
kind: Deployment
metadata:
  name: reviews-v3
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v3
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v3:1.8.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Productpage services
apiVersion: v1
kind: Service
metadata:
  name: productpage
  labels:
    app: productpage
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: productpage
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: productpage-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
    spec:
      containers:
      - name: productpage
        image: istio/examples-bookinfo-productpage-v1:1.8.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
```
</div></details>

<details><summary>組み込み後のbookinfo.yaml</summary><div>

```yaml:bookinfo.yaml
# Copyright 2017 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
##################################################################################################
# Details service
apiVersion: v1
  labels:
    app: details
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: details
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: details-v1
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: details
        version: v1
    spec:
      containers:
      - image: istio/examples-bookinfo-details-v1:1.8.0
        imagePullPolicy: IfNotPresent
        name: details
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - details
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"details","version":"v1"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -d
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
          privileged: true
      volumes:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
# Ratings service
apiVersion: v1
  labels:
    app: ratings
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: ratings
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: ratings-v1
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: ratings
        version: v1
    spec:
      containers:
      - image: istio/examples-bookinfo-ratings-v1:1.8.0
        imagePullPolicy: IfNotPresent
        name: ratings
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - ratings
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"ratings","version":"v1"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -d
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
          privileged: true
      volumes:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
# Reviews service
apiVersion: v1
  labels:
    app: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: reviews-v1
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: reviews
        version: v1
    spec:
      containers:
      - image: istio/examples-bookinfo-reviews-v1:1.8.0
        imagePullPolicy: IfNotPresent
        name: reviews
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - reviews
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"reviews","version":"v1"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -b
        - "9080"
        - -d
        - ""
        image: docker.io/istio/proxy_init:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: reviews-v2
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: reviews
        version: v2
    spec:
      containers:
      - image: istio/examples-bookinfo-reviews-v2:1.8.0
        imagePullPolicy: IfNotPresent
        name: reviews
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - reviews
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"reviews","version":"v2"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -b
        - "9080"
        - -d
        - ""
        image: docker.io/istio/proxy_init:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: reviews-v3
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: reviews
        version: v3
    spec:
      containers:
      - image: istio/examples-bookinfo-reviews-v3:1.8.0
        imagePullPolicy: IfNotPresent
        name: reviews
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - reviews
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"reviews","version":"v3"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -d
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
          privileged: true
      volumes:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
# Productpage services
apiVersion: v1
  labels:
    app: productpage
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: productpage
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: productpage-v1
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      annotations:
      creationTimestamp: null
      labels:
        app: productpage
        version: v1
    spec:
      containers:
      - image: istio/examples-bookinfo-productpage-v1:1.8.0
        imagePullPolicy: IfNotPresent
        name: productpage
        ports:
        - containerPort: 9080
        resources: {}
      - args:
        - proxy
        - sidecar
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - productpage
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15007
        - --discoveryRefreshDelay
        - 1s
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        - name: ISTIO_METAJSON_LABELS
          value: |
            {"app":"productpage","version":"v1"}
        image: docker.io/istio/proxyv2:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        resources:
          requests:
            cpu: 10m
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -b
        - "9080"
        - -d
        - ""
        image: docker.io/istio/proxy_init:1.0.6
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
          privileged: true
      volumes:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---
```
</div></details>

###GatewayとVirtualServiceの作成
ここで、アプリケーションを外部に公開するために、IstioのGateway & VirtualServiceを使って、istio-ingressgatewayとアプリケーションを繋げます。

```
$ kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
$ kubectl get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' -n istio-system ; echo
a81256f2b46c011e9bec6068f6b115a8-1231799013.us-west-2.elb.amazonaws.com
```
IstioのGateway & VirtualServiceを作成するManifestを見る限り、

①Gatewayとしてistio-ingressgatewayの80ポートを構え
②トラフィックをproductpageのService⇒productpageの9080番ポートへ流す

ということをしているんだと理解しました。

```yaml:bookinfo-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080
```
この段階でこんな感じに接続されて、アプリケーションへアクセスすることができるようになりました。
CLBの80番ポートへのアクセスがproductpageのpodへトラフィック⇒istio-proxyへリダイレクト⇒9080番ポートで構えているproductpageのアプリケーションコンテナへ到達、という流れを辿っている（はず）
![istio-sampleapp-2.png](https://qiita-image-store.s3.amazonaws.com/0/285420/9e8f0ebc-3208-e19e-e4c8-625ce2bb80e7.png)

http://＜CLBのDNS＞/productpage　へアクセス
（今回はhttp://a81256f2b46c011e9bec6068f6b115a8-1231799013.us-west-2.elb.amazonaws.com/productpage）

![bookinfo.png](https://qiita-image-store.s3.amazonaws.com/0/285420/a6628c8a-4d67-e184-ae63-2ccce14f3941.png)

なお、現時点ではreviewsサービスはreviews-v1～3のすべてのpodにラウンドロビンしてトラフィックを流しているため、画面をリロードする度にBook Reviewsの表示が切り替わります。
####Version1
![reviews-v1.png](https://qiita-image-store.s3.amazonaws.com/0/285420/f64230c2-876e-91f4-56ce-eac2c78cdf08.png)
####Version2
![reviews-v2.png](https://qiita-image-store.s3.amazonaws.com/0/285420/d0b1199a-185b-af7f-e228-0fbc331ec44d.png)
####Version3
![reviews-v3.png](https://qiita-image-store.s3.amazonaws.com/0/285420/bfac365b-79ca-c210-b4f5-f2a2a1d01853.png)

#４．IstioでRouting制御
いよいよ、Istioの機能を使ってRoutingしてみます。

まずは、Routing制御のために必要なDestinationRuleを作成。
⇒ここで各versionを定義しており、それらをVirtualServiceからsubsetとして指定することでどのversionにトラフィックを流すかをRoutingします。

```
kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml
```
```yaml:destination-rule-all.yaml（抜粋）
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
---
```

###特定versionにのみRouting
versionを一つ指定し、ラウンドロビンさせずに特定versionのみにRoutingするパターン
⇒subsetに先ほど定義したDestinationRuleのversionを指定

```
kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml
```
```yaml:virtual-service-all-v1.yaml（抜粋）
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
---
```

###重みを付けて、複数versionにラウンドロビンさせてRouting
versionを2つで50:50の重みを付け、version1とversion3に50%ずつRoutingさせるパターン
⇒比率を変えてBlue/Greenデプロイをしたり、カナリアリリースしたり、A/Bテストしたり、いろいろなケースで利用できます。

```
kubectl apply -f samples/bookinfo/networking/virtual-service-all-v1.yaml
kubectl apply -f samples/bookinfo/networking/virtual-service-reviews-50-v3.yaml
```
```yaml:virtual-service-reviews-50-v3.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50
---
```

###特定ユーザからのアクセス時のみ別versionへRouting
header情報をもとにRoutingを切り替えることも可能です。
ここでは、headersのuser情報を見て、jasonユーザがログインしてアクセスしてきた場合のみversion2へRoutingし、それ以外のユーザの場合にはversion1へRoutingしています。

```
kubectl apply -f samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
```
```yaml:virtual-service-reviews-test-v2.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1
---
```

###Fault Injection
アクセスに対して、秒指定でレスポンスを遅延させたり、エラーを発生させたりすることもできます。

例えばjasonユーザからのアクセスのみレスポンスを7秒間遅延させるパターン
⇒ratingsアプリケーションは6秒間レスポンスがない場合にはエラーとなる仕様なので、Book Reviews内のraiting部分のみエラーが表示されます。

```
kubectl apply -f samples/bookinfo/networking/virtual-service-ratings-test-delay.yaml
```
```yaml:virtual-service-ratings-test-delay.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    fault:
      delay:
        percent: 100
        fixedDelay: 7s
    route:
    - destination:
        host: ratings
        subset: v1
  - route:
    - destination:
        host: ratings
        subset: v1
---
```

次は、jasonユーザからのアクセスのみ500エラーをレスポンスさせるパターン
⇒先ほどと同じように、Book Reviews内のraiting部分にエラーメッセージが表示されます。

```
kubectl apply -f samples/bookinfo/networking/virtual-service-ratings-test-abort.yaml
```
```yaml:virtual-service-ratings-test-abort.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    fault:
      abort:
        percent: 100
        httpStatus: 500
    route:
    - destination:
        host: ratings
        subset: v1
  - route:
    - destination:
        host: ratings
        subset: v1
---
```

#５．可視化ツールを使ってみる
ここで、Istioと一緒にデプロイしたGrafana、Jaeger、Kialiの画面を見てみます。

ワークショップの手順ではkubectl port-forwardを使用してlocalhostへのアクセスを行っているのですが、外部からのアクセスができるように設定してみます。

###外部アクセス可能なType:NodePortなServiceを作成
Grafana、Jaeger、KialiへアクセスするServiceはそれぞれgrafana、jaeger-query、kialiですが、現状はServiceのTypeがClusterIPのため外部アクセスできません。
そのため、TypeをNordPortに指定したサービスを追加で作成します。

```shell
$ kubectl expose service grafana --type=NodePort --name=grafana -svc -n istio-system
$ kubectl expose service jaeger-query --type=NodePort --name=jaeger-query-svc -n istio-system
$ kubectl expose service kiali --type=NodePort --name=kiali-svc -n istio-system

kubectl get pod,svc -n istio-system -o wide
NAME                                          READY     STATUS      RESTARTS   AGE       IP           NODE
pod/grafana-7f6cd4bf56-tq5tp                  1/1       Running     0          3d        10.0.0.21    ip-10-0-0-4.us-west-2.compute.internal
pod/istio-citadel-796c94878b-s7zc4            1/1       Running     0          3d        10.0.3.132   ip-10-0-3-130.us-west-2.compute.internal
pod/istio-cleanup-secrets-p94tx               0/1       Completed   0          3d        10.0.3.112   ip-10-0-3-130.us-west-2.compute.internal
pod/istio-egressgateway-864444d6ff-cgjh4      1/1       Running     0          3d        10.0.0.43    ip-10-0-0-4.us-west-2.compute.internal
pod/istio-galley-6c68c5dbcf-hg9rx             1/1       Running     0          3d        10.0.3.58    ip-10-0-3-130.us-west-2.compute.internal
pod/istio-grafana-post-install-2pcdx          0/1       Completed   0          3d        10.0.0.49    ip-10-0-0-4.us-west-2.compute.internal
pod/istio-ingressgateway-694576c7bb-kwsk4     1/1       Running     0          3d        10.0.3.143   ip-10-0-3-130.us-west-2.compute.internal
pod/istio-pilot-79f5f46dd5-ngcm6              2/2       Running     0          3d        10.0.0.28    ip-10-0-0-4.us-west-2.compute.internal
pod/istio-policy-5bd5578b94-w9sq4             2/2       Running     0          3d        10.0.0.204   ip-10-0-0-4.us-west-2.compute.internal
pod/istio-security-post-install-chzxk         0/1       Completed   0          3d        10.0.0.233   ip-10-0-0-4.us-west-2.compute.internal
pod/istio-sidecar-injector-6d8f88c98f-gbhtq   1/1       Running     1          3d        10.0.3.8     ip-10-0-3-130.us-west-2.compute.internal
pod/istio-telemetry-5598f86cd8-7c7bc          2/2       Running     0          3d        10.0.3.107   ip-10-0-3-130.us-west-2.compute.internal
pod/istio-tracing-7596597bd7-t5qjg            1/1       Running     0          3d        10.0.3.97    ip-10-0-3-130.us-west-2.compute.internal
pod/kiali-84cd87cd5d-pl29c                    1/1       Running     0          3d        10.0.3.75    ip-10-0-3-130.us-west-2.compute.internal
pod/prometheus-76db5fddd5-w2477               1/1       Running     0          3d        10.0.3.184   ip-10-0-3-130.us-west-2.compute.internal
pod/servicegraph-df4d5f678-prxhn              1/1       Running     0          3d        10.0.3.144   ip-10-0-3-130.us-west-2.compute.internal

NAME                             TYPE           CLUSTER-IP       EXTERNAL-IP                                                               PORT(S)                                                                                                                   AGE       SELECTOR
service/grafana                  ClusterIP      172.20.250.52    <none>                                                                    3000/TCP                                                                                                                  3d        app=grafana
service/grafana-svc              NodePort       172.20.169.16    <none>                                                                    3000:31179/TCP                                                                                                            18m       app=grafana
service/istio-citadel            ClusterIP      172.20.5.227     <none>                                                                    8060/TCP,9093/TCP                                                                                                         3d        istio=citadel
service/istio-egressgateway      ClusterIP      172.20.157.133   <none>                                                                    80/TCP,443/TCP                                                                                                            3d        app=istio-egressgateway,istio=egressgateway
service/istio-galley             ClusterIP      172.20.253.156   <none>                                                                    443/TCP,9093/TCP                                                                                                          3d        istio=galley
service/istio-ingressgateway     LoadBalancer   172.20.14.123    a81256f2b46c011e9bec6068f6b115a8-1231799013.us-west-2.elb.amazonaws.com   80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:31060/TCP,8060:30178/TCP,853:31935/TCP,15030:30486/TCP,15031:31441/TCP   3d        app=istio-ingressgateway,istio=ingressgateway
service/istio-pilot              ClusterIP      172.20.217.14    <none>                                                                    15010/TCP,15011/TCP,8080/TCP,9093/TCP                                                                                     3d        istio=pilot
service/istio-policy             ClusterIP      172.20.145.140   <none>                                                                    9091/TCP,15004/TCP,9093/TCP                                                                                               3d        istio-mixer-type=policy,istio=mixer
service/istio-sidecar-injector   ClusterIP      172.20.84.237    <none>                                                                    443/TCP                                                                                                                   3d        istio=sidecar-injector
service/istio-telemetry          ClusterIP      172.20.105.202   <none>                                                                    9091/TCP,15004/TCP,9093/TCP,42422/TCP                                                                                     3d        istio-mixer-type=telemetry,istio=mixer
service/jaeger-agent             ClusterIP      None             <none>                                                                    5775/UDP,6831/UDP,6832/UDP                                                                                                3d        app=jaeger
service/jaeger-collector         ClusterIP      172.20.154.91    <none>                                                                    14267/TCP,14268/TCP                                                                                                       3d        app=jaeger
service/jaeger-query             ClusterIP      172.20.132.103   <none>                                                                    16686/TCP                                                                                                                 3d        app=jaeger
service/jaeger-query-svc         NodePort       172.20.87.213    <none>                                                                    16686:31270/TCP                                                                                                           3h        app=jaeger
service/kiali                    ClusterIP      172.20.57.195    <none>                                                                    20001/TCP                                                                                                                 3d        app=kiali
service/kiali-svc                NodePort       172.20.107.2     <none>                                                                    20001:32342/TCP                                                                                                           1h        app=kiali
service/prometheus               ClusterIP      172.20.97.85     <none>                                                                    9090/TCP                                                                                                                  3d        app=prometheus
service/servicegraph             ClusterIP      172.20.23.26     <none>                                                                    8088/TCP                                                                                                                  3d        app=servicegraph
service/tracing                  ClusterIP      172.20.145.78    <none>                                                                    80/TCP                                                                                                                    3d        app=jaeger
service/zipkin                   ClusterIP      172.20.6.139     <none>                                                                    9411/TCP    
```

これで各サービスでポート31179,31270,32342が新たに割り振られましたが、現時点でノードに割り当てられたセキュリティグループが通信を許可していないため、許可をしてあげます。
![secg.png](https://qiita-image-store.s3.amazonaws.com/0/285420/103fe0e1-fdaa-8037-3581-9e8b85984398.png)



これでGrafana、Jaeger、Kialiが外部公開されました。

###Grafana
例えばGrafanaの場合、get pod,svcした情報から、

①DNS：grafana-7f6cd4bf56-tq5tpのノードIP（ip-10-0-0-4.us-west-2.compute.internal）に対応するEC2インスタンスのパブリックDNS
②PORT：追加作成したServiceに割り当てられた31179番ポート

を使用して、以下のアドレスでGrafanaのダッシュボードへアクセスできます。

http://ec2-54-188-131-63.us-west-2.compute.amazonaws.com:31179

![grafana1.png](https://qiita-image-store.s3.amazonaws.com/0/285420/d0d7046c-ec20-5670-8ee1-a19d3c282c72.png)
すでにIstio用のダッシュボードが自動で作られています。
![grafana2.png](https://qiita-image-store.s3.amazonaws.com/0/285420/b67d379d-c467-1cec-6315-a33fe31a0cc5.png)

コマンドでリクエストを投げまくってみてから、Istio Mesh Dashboardを見てみると、各Serviceへのアクセス状況が確認できます。
（reviewsへのRouting設定をv1:v3=50:50で設定中なので、v2へのアクセスがNoneとなっています）

```shell
export SMHOST=$(kubectl get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname} ' -n istio-system)
SMHOST="$(echo -e "${SMHOST}" | tr -d '[:space:]')"
while true; do curl -o /dev/null -s "${SMHOST}/productpage"; done
```
![grafana3.png](https://qiita-image-store.s3.amazonaws.com/0/285420/602da6a8-e7ae-c114-90db-50d01af68f4c.png)

リソース使用状況
![grafana4.png](https://qiita-image-store.s3.amazonaws.com/0/285420/422fd116-aa34-2e45-5fc2-94b562f8fd84.png)

###Jaeger
http://ec2-34-216-109-95.us-west-2.compute.amazonaws.com:31270

![jaeger1.png](https://qiita-image-store.s3.amazonaws.com/0/285420/95b016ff-423f-9697-0130-4a2ec9f7145a.png)
各サービスのトレース情報を表示できます
![jaeger2.png](https://qiita-image-store.s3.amazonaws.com/0/285420/73c52530-258b-1fcd-9a93-d638c9001fc0.png)


###Kiali
http://ec2-34-216-109-95.us-west-2.compute.amazonaws.com:32342

ログイン情報にはkiali用secret作成時に指定したUsername＆passを使います。
（↓ここの'admin'、'1f2d1e2e67df'）

```
$ NAMESPACE=istio-system
$ echo -n 'admin' | base64
YWRtaW4=
$ echo -n '1f2d1e2e67df' | base64
MWYyZDFlMmU2N2Rm
```

![kiali1.png](https://qiita-image-store.s3.amazonaws.com/0/285420/064d0930-a9e6-a6cd-02c2-5c788b129038.png)

Graphからはサービスメッシュを良い感じにグラフィカルに見ることができ、かつトラフィック状況をグラフで確認することができます。
![kiali2.png](https://qiita-image-store.s3.amazonaws.com/0/285420/d9f4eb17-4e3c-9f6f-edd9-66496d4323e1.png)
各設定ファイルの検査もしてくれたりします。Destinationに一部無駄な定義がある（v2がないのにv2の定義がある）のでエラーになってます。
![kiali3.png](https://qiita-image-store.s3.amazonaws.com/0/285420/6e9f10c2-3c06-dc67-f73e-d5b51946396b.png)

本当はGrafana&Jaegerとの統合ができるようで、[KialiのGithub](https://github.com/kiali/kiali#configure-external-services)の通りにConfigmapのURL追加をやってみたもののうまくいかず、Kiali画面右上にアラームが出っぱなしになってます。（Error fetching Grafana Info., Error: [ Grafana URL is not set in Kiali configuration ]）

別途情報探してみようと思います。（Configmap修正以外に必要なことがわかる方いれば教えて下さい）

#６．さいごに
基礎の基礎だけながらも記事を書きながらワークショップを進めたことで、なんとなく難しくてずっと避けてきたk8sも少し肌間隔を掴めてきたような気がします。

Istioも今回は一先ず基本的なところだけ触りましたが、まだまだCircuitBreakerやHealthCheck等々いろんな機能があるので、引き続き触ってみようと思います。
（でもやっぱりIstio難しいので、もうすぐGAされるらしいAppMeshがうまいことこのへんラッピングしてくれて、イイ感じなマネージドサービスとして君臨してくれるのを期待したいのが本音）