Kubeflow 1.0 on AWS #1 構築

# はじめに
これは、Kubeflow 1.0 をAWSで構築する記事です。
動作確認が主な目的ですので、本番環境での利用は全く想定していません。

# 環境について
- kubernetes
 - Amazon EKS v1.14(検証開始時点)
- kubeflow
 - kubeflow 1.0.1(検証開始時点での最新)
- クライアント環境
 - [DockerHub](https://hub.docker.com/r/asahi0301/eks-toolkit)のdockerイメージ上で実行(私が作成したもの)
    - kubectl, eksctl, awscli 等々検証に必要なものはだいたいインストール済み

# 構築
[GitHub](https://github.com/asahi0301/kubeflow-1.0-on-aws)に書いてあります

# 構築後にやったこと
## 環境の確認
```sh
kubectl get all -A
NAMESPACE      NAME                                                               READY   STATUS      RESTARTS   AGE
anonymous      pod/test1-0                                                        2/2     Running     0          8h
cert-manager   pod/cert-manager-564b4bffd7-svsvz                                  1/1     Running     0          8h
cert-manager   pod/cert-manager-cainjector-596986f94-rtrjn                        1/1     Running     0          8h
cert-manager   pod/cert-manager-webhook-755d75845c-qqxt4                          1/1     Running     0          8h
istio-system   pod/cluster-local-gateway-748f94c9ff-4lcbp                         1/1     Running     0          8h
istio-system   pod/grafana-67c69bb567-ckqz6                                       1/1     Running     0          8h
istio-system   pod/istio-citadel-67697b6697-dnn94                                 1/1     Running     0          8h
istio-system   pod/istio-cleanup-secrets-1.1.6-ql8mr                              0/1     Completed   0          8h
istio-system   pod/istio-egressgateway-7dbbb87698-6tgqd                           1/1     Running     0          8h
istio-system   pod/istio-galley-7bffd57ff4-hqzmk                                  1/1     Running     0          8h
istio-system   pod/istio-grafana-post-install-1.1.6-rhcpc                         0/1     Completed   0          8h
istio-system   pod/istio-ingressgateway-565b894b5f-kn9nm                          1/1     Running     0          8h
istio-system   pod/istio-pilot-6dd5b8f74c-rf2mm                                   2/2     Running     0          8h
istio-system   pod/istio-policy-7f8bb87857-bkrjj                                  2/2     Running     1          8h
istio-system   pod/istio-security-post-install-1.1.6-m48kv                        0/1     Completed   0          8h
istio-system   pod/istio-sidecar-injector-fd5875568-xlkzp                         1/1     Running     0          8h
istio-system   pod/istio-telemetry-8759dc6b7-99jqb                                2/2     Running     1          8h
istio-system   pod/istio-tracing-5d8f57c8ff-j2mf2                                 1/1     Running     0          8h
istio-system   pod/kiali-d4d886dd7-hdd4z                                          1/1     Running     0          8h
istio-system   pod/prometheus-d8d46c5b5-8xqm6                                     1/1     Running     0          8h
kube-system    pod/aws-node-9xtq9                                                 1/1     Running     0          9h
kube-system    pod/aws-node-cxw86                                                 1/1     Running     0          9h
kube-system    pod/aws-node-d55n9                                                 1/1     Running     0          9h
kube-system    pod/aws-node-fljdw                                                 1/1     Running     0          9h
kube-system    pod/aws-node-hfdzw                                                 1/1     Running     0          9h
kube-system    pod/aws-node-r56ql                                                 1/1     Running     0          9h
kube-system    pod/coredns-84549585c-5s4lv                                        1/1     Running     0          9h
kube-system    pod/coredns-84549585c-gkkx9                                        1/1     Running     0          9h
kube-system    pod/kube-proxy-27tcv                                               1/1     Running     0          9h
kube-system    pod/kube-proxy-84dws                                               1/1     Running     0          9h
kube-system    pod/kube-proxy-lbnb6                                               1/1     Running     0          9h
kube-system    pod/kube-proxy-lwchd                                               1/1     Running     0          9h
kube-system    pod/kube-proxy-t5grh                                               1/1     Running     0          9h
kube-system    pod/kube-proxy-wlwcj                                               1/1     Running     0          9h
kubeflow       pod/admission-webhook-bootstrap-stateful-set-0                     1/1     Running     0          8h
kubeflow       pod/admission-webhook-deployment-64cb96ddbf-t6k8j                  1/1     Running     0          8h
kubeflow       pod/alb-ingress-controller-8574c47cdb-8h229                        1/1     Running     0          8h
kubeflow       pod/application-controller-stateful-set-0                          1/1     Running     0          8h
kubeflow       pod/argo-ui-778676df64-8kdh5                                       1/1     Running     0          8h
kubeflow       pod/centraldashboard-7dd7dd685d-fvjl9                              1/1     Running     0          8h
kubeflow       pod/jupyter-web-app-deployment-89789fd5-b7v9m                      1/1     Running     0          8h
kubeflow       pod/katib-controller-6b789b6cb5-tlf7g                              1/1     Running     1          8h
kubeflow       pod/katib-db-manager-64f548b47c-xdh7l                              1/1     Running     0          8h
kubeflow       pod/katib-mysql-57884cb488-ghtq8                                   1/1     Running     0          8h
kubeflow       pod/katib-ui-5c5cc6bd77-rzsvf                                      1/1     Running     0          8h
kubeflow       pod/metacontroller-0                                               1/1     Running     0          8h
kubeflow       pod/metadata-db-76c9f78f77-b5rhv                                   1/1     Running     0          8h
kubeflow       pod/metadata-deployment-674fdd976b-pq2lr                           1/1     Running     0          8h
kubeflow       pod/metadata-envoy-deployment-5688989bd6-xc8p2                     1/1     Running     0          8h
kubeflow       pod/metadata-grpc-deployment-5579bdc87b-44xjf                      1/1     Running     4          8h
kubeflow       pod/metadata-ui-9b8cd699d-76876                                    1/1     Running     0          8h
kubeflow       pod/minio-755ff748b-zzkp2                                          1/1     Running     0          8h
kubeflow       pod/ml-pipeline-79b4f85cbc-24fjr                                   1/1     Running     0          8h
kubeflow       pod/ml-pipeline-ml-pipeline-visualizationserver-5fdffdc5bf-pgknt   1/1     Running     0          8h
kubeflow       pod/ml-pipeline-persistenceagent-645cb66874-vxvst                  1/1     Running     0          8h
kubeflow       pod/ml-pipeline-scheduledworkflow-6c978b6b85-hxqh5                 1/1     Running     0          8h
kubeflow       pod/ml-pipeline-ui-6995b7bccf-2wfvv                                1/1     Running     0          8h
kubeflow       pod/ml-pipeline-viewer-controller-deployment-8554dc7b9f-jppcf      1/1     Running     0          8h
kubeflow       pod/mpi-operator-5bf8b566b7-btz8t                                  1/1     Running     0          8h
kubeflow       pod/mysql-598bc897dc-5jrtd                                         1/1     Running     0          8h
kubeflow       pod/notebook-controller-deployment-7db57b9ccf-sw494                1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-bckbj                           1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-j86gk                           1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-sjrsh                           1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-sq2d9                           1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-vqjjd                           1/1     Running     0          8h
kubeflow       pod/nvidia-device-plugin-daemonset-zzh5d                           1/1     Running     0          8h
kubeflow       pod/profiles-deployment-b45dbc6f-ksf4s                             2/2     Running     0          8h
kubeflow       pod/pytorch-operator-5fd5f94bdd-wpltv                              1/1     Running     0          8h
kubeflow       pod/seldon-controller-manager-679fc777cd-cjtlj                     1/1     Running     0          8h
kubeflow       pod/spark-operatorcrd-cleanup-hbzfn                                0/2     Completed   0          8h
kubeflow       pod/spark-operatorsparkoperator-c7b64b87f-jqktr                    1/1     Running     0          8h
kubeflow       pod/spartakus-volunteer-6b79679599-5m6qg                           1/1     Running     0          8h
kubeflow       pod/tensorboard-6544748d94-c72sf                                   1/1     Running     0          8h
kubeflow       pod/tf-job-operator-7d7c8fb8bb-rpfxx                               1/1     Running     0          8h
kubeflow       pod/workflow-controller-945c84565-gt5r6                            1/1     Running     0          8h


NAMESPACE      NAME                                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                                                                                      AGE
anonymous      service/test1                                         ClusterIP   10.100.52.23     <none>        80/TCP                                                                                                                                       8h
cert-manager   service/cert-manager                                  ClusterIP   10.100.148.158   <none>        9402/TCP                                                                                                                                     8h
cert-manager   service/cert-manager-webhook                          ClusterIP   10.100.51.103    <none>        443/TCP                                                                                                                                      8h
default        service/kubernetes                                    ClusterIP   10.100.0.1       <none>        443/TCP                                                                                                                                      9h
istio-system   service/cluster-local-gateway                         ClusterIP   10.100.57.146    <none>        80/TCP,443/TCP,31400/TCP,15011/TCP,8060/TCP,15029/TCP,15030/TCP,15031/TCP,15032/TCP                                                          8h
istio-system   service/grafana                                       ClusterIP   10.100.79.107    <none>        3000/TCP                                                                                                                                     8h
istio-system   service/istio-citadel                                 ClusterIP   10.100.78.125    <none>        8060/TCP,15014/TCP                                                                                                                           8h
istio-system   service/istio-egressgateway                           ClusterIP   10.100.214.220   <none>        80/TCP,443/TCP,15443/TCP                                                                                                                     8h
istio-system   service/istio-galley                                  ClusterIP   10.100.233.242   <none>        443/TCP,15014/TCP,9901/TCP                                                                                                                   8h
istio-system   service/istio-ingressgateway                          NodePort    10.100.210.137   <none>        15020:31941/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:30395/TCP,15030:31313/TCP,15031:30738/TCP,15032:32608/TCP,15443:31210/TCP   8h
istio-system   service/istio-pilot                                   ClusterIP   10.100.16.61     <none>        15010/TCP,15011/TCP,8080/TCP,15014/TCP                                                                                                       8h
istio-system   service/istio-policy                                  ClusterIP   10.100.46.12     <none>        9091/TCP,15004/TCP,15014/TCP                                                                                                                 8h
istio-system   service/istio-sidecar-injector                        ClusterIP   10.100.161.15    <none>        443/TCP                                                                                                                                      8h
istio-system   service/istio-telemetry                               ClusterIP   10.100.98.143    <none>        9091/TCP,15004/TCP,15014/TCP,42422/TCP                                                                                                       8h
istio-system   service/jaeger-agent                                  ClusterIP   None             <none>        5775/UDP,6831/UDP,6832/UDP                                                                                                                   8h
istio-system   service/jaeger-collector                              ClusterIP   10.100.189.159   <none>        14267/TCP,14268/TCP                                                                                                                          8h
istio-system   service/jaeger-query                                  ClusterIP   10.100.233.125   <none>        16686/TCP                                                                                                                                    8h
istio-system   service/kiali                                         ClusterIP   10.100.86.41     <none>        20001/TCP                                                                                                                                    8h
istio-system   service/prometheus                                    ClusterIP   10.100.46.3      <none>        9090/TCP                                                                                                                                     8h
istio-system   service/tracing                                       ClusterIP   10.100.15.74     <none>        80/TCP                                                                                                                                       8h
istio-system   service/zipkin                                        ClusterIP   10.100.17.2      <none>        9411/TCP                                                                                                                                     8h
kube-system    service/kube-dns                                      ClusterIP   10.100.0.10      <none>        53/UDP,53/TCP                                                                                                                                9h
kubeflow       service/admission-webhook-service                     ClusterIP   10.100.92.200    <none>        443/TCP                                                                                                                                      8h
kubeflow       service/application-controller-service                ClusterIP   10.100.79.239    <none>        443/TCP                                                                                                                                      8h
kubeflow       service/argo-ui                                       NodePort    10.100.87.172    <none>        80:30385/TCP                                                                                                                                 8h
kubeflow       service/centraldashboard                              ClusterIP   10.100.32.250    <none>        80/TCP                                                                                                                                       8h
kubeflow       service/jupyter-web-app-service                       ClusterIP   10.100.153.47    <none>        80/TCP                                                                                                                                       8h
kubeflow       service/katib-controller                              ClusterIP   10.100.251.51    <none>        443/TCP,8080/TCP                                                                                                                             8h
kubeflow       service/katib-db-manager                              ClusterIP   10.100.48.46     <none>        6789/TCP                                                                                                                                     8h
kubeflow       service/katib-mysql                                   ClusterIP   10.100.196.178   <none>        3306/TCP                                                                                                                                     8h
kubeflow       service/katib-ui                                      ClusterIP   10.100.238.76    <none>        80/TCP                                                                                                                                       8h
kubeflow       service/metadata-db                                   ClusterIP   10.100.109.172   <none>        3306/TCP                                                                                                                                     8h
kubeflow       service/metadata-envoy-service                        ClusterIP   10.100.143.66    <none>        9090/TCP                                                                                                                                     8h
kubeflow       service/metadata-grpc-service                         ClusterIP   10.100.120.211   <none>        8080/TCP                                                                                                                                     8h
kubeflow       service/metadata-service                              ClusterIP   10.100.126.140   <none>        8080/TCP                                                                                                                                     8h
kubeflow       service/metadata-ui                                   ClusterIP   10.100.251.249   <none>        80/TCP                                                                                                                                       8h
kubeflow       service/minio-service                                 ClusterIP   10.100.42.199    <none>        9000/TCP                                                                                                                                     8h
kubeflow       service/ml-pipeline                                   ClusterIP   10.100.176.178   <none>        8888/TCP,8887/TCP                                                                                                                            8h
kubeflow       service/ml-pipeline-ml-pipeline-visualizationserver   ClusterIP   10.100.113.144   <none>        8888/TCP                                                                                                                                     8h
kubeflow       service/ml-pipeline-tensorboard-ui                    ClusterIP   10.100.46.114    <none>        80/TCP                                                                                                                                       8h
kubeflow       service/ml-pipeline-ui                                ClusterIP   10.100.239.18    <none>        80/TCP                                                                                                                                       8h
kubeflow       service/mysql                                         ClusterIP   10.100.235.36    <none>        3306/TCP                                                                                                                                     8h
kubeflow       service/notebook-controller-service                   ClusterIP   10.100.167.185   <none>        443/TCP                                                                                                                                      8h
kubeflow       service/profiles-kfam                                 ClusterIP   10.100.90.227    <none>        8081/TCP                                                                                                                                     8h
kubeflow       service/pytorch-operator                              ClusterIP   10.100.222.24    <none>        8443/TCP                                                                                                                                     8h
kubeflow       service/seldon-webhook-service                        ClusterIP   10.100.213.191   <none>        443/TCP                                                                                                                                      8h
kubeflow       service/tensorboard                                   ClusterIP   10.100.69.23     <none>        9000/TCP                                                                                                                                     8h
kubeflow       service/tf-job-operator                               ClusterIP   10.100.151.226   <none>        8443/TCP                                                                                                                                     8h

NAMESPACE     NAME                                            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   daemonset.apps/aws-node                         6         6         6       6            6           <none>          9h
kube-system   daemonset.apps/kube-proxy                       6         6         6       6            6           <none>          9h
kubeflow      daemonset.apps/nvidia-device-plugin-daemonset   6         6         6       6            6           <none>          8h

NAMESPACE      NAME                                                          READY   UP-TO-DATE   AVAILABLE   AGE
cert-manager   deployment.apps/cert-manager                                  1/1     1            1           8h
cert-manager   deployment.apps/cert-manager-cainjector                       1/1     1            1           8h
cert-manager   deployment.apps/cert-manager-webhook                          1/1     1            1           8h
istio-system   deployment.apps/cluster-local-gateway                         1/1     1            1           8h
istio-system   deployment.apps/grafana                                       1/1     1            1           8h
istio-system   deployment.apps/istio-citadel                                 1/1     1            1           8h
istio-system   deployment.apps/istio-egressgateway                           1/1     1            1           8h
istio-system   deployment.apps/istio-galley                                  1/1     1            1           8h
istio-system   deployment.apps/istio-ingressgateway                          1/1     1            1           8h
istio-system   deployment.apps/istio-pilot                                   1/1     1            1           8h
istio-system   deployment.apps/istio-policy                                  1/1     1            1           8h
istio-system   deployment.apps/istio-sidecar-injector                        1/1     1            1           8h
istio-system   deployment.apps/istio-telemetry                               1/1     1            1           8h
istio-system   deployment.apps/istio-tracing                                 1/1     1            1           8h
istio-system   deployment.apps/kiali                                         1/1     1            1           8h
istio-system   deployment.apps/prometheus                                    1/1     1            1           8h
kube-system    deployment.apps/coredns                                       2/2     2            2           9h
kubeflow       deployment.apps/admission-webhook-deployment                  1/1     1            1           8h
kubeflow       deployment.apps/alb-ingress-controller                        1/1     1            1           8h
kubeflow       deployment.apps/argo-ui                                       1/1     1            1           8h
kubeflow       deployment.apps/centraldashboard                              1/1     1            1           8h
kubeflow       deployment.apps/jupyter-web-app-deployment                    1/1     1            1           8h
kubeflow       deployment.apps/katib-controller                              1/1     1            1           8h
kubeflow       deployment.apps/katib-db-manager                              1/1     1            1           8h
kubeflow       deployment.apps/katib-mysql                                   1/1     1            1           8h
kubeflow       deployment.apps/katib-ui                                      1/1     1            1           8h
kubeflow       deployment.apps/metadata-db                                   1/1     1            1           8h
kubeflow       deployment.apps/metadata-deployment                           1/1     1            1           8h
kubeflow       deployment.apps/metadata-envoy-deployment                     1/1     1            1           8h
kubeflow       deployment.apps/metadata-grpc-deployment                      1/1     1            1           8h
kubeflow       deployment.apps/metadata-ui                                   1/1     1            1           8h
kubeflow       deployment.apps/minio                                         1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline                                   1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline-ml-pipeline-visualizationserver   1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline-persistenceagent                  1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline-scheduledworkflow                 1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline-ui                                1/1     1            1           8h
kubeflow       deployment.apps/ml-pipeline-viewer-controller-deployment      1/1     1            1           8h
kubeflow       deployment.apps/mpi-operator                                  1/1     1            1           8h
kubeflow       deployment.apps/mysql                                         1/1     1            1           8h
kubeflow       deployment.apps/notebook-controller-deployment                1/1     1            1           8h
kubeflow       deployment.apps/profiles-deployment                           1/1     1            1           8h
kubeflow       deployment.apps/pytorch-operator                              1/1     1            1           8h
kubeflow       deployment.apps/seldon-controller-manager                     1/1     1            1           8h
kubeflow       deployment.apps/spark-operatorsparkoperator                   1/1     1            1           8h
kubeflow       deployment.apps/spartakus-volunteer                           1/1     1            1           8h
kubeflow       deployment.apps/tensorboard                                   1/1     1            1           8h
kubeflow       deployment.apps/tf-job-operator                               1/1     1            1           8h
kubeflow       deployment.apps/workflow-controller                           1/1     1            1           8h

NAMESPACE      NAME                                                                     DESIRED   CURRENT   READY   AGE
cert-manager   replicaset.apps/cert-manager-564b4bffd7                                  1         1         1       8h
cert-manager   replicaset.apps/cert-manager-cainjector-596986f94                        1         1         1       8h
cert-manager   replicaset.apps/cert-manager-webhook-755d75845c                          1         1         1       8h
istio-system   replicaset.apps/cluster-local-gateway-748f94c9ff                         1         1         1       8h
istio-system   replicaset.apps/grafana-67c69bb567                                       1         1         1       8h
istio-system   replicaset.apps/istio-citadel-67697b6697                                 1         1         1       8h
istio-system   replicaset.apps/istio-egressgateway-7dbbb87698                           1         1         1       8h
istio-system   replicaset.apps/istio-galley-7bffd57ff4                                  1         1         1       8h
istio-system   replicaset.apps/istio-ingressgateway-565b894b5f                          1         1         1       8h
istio-system   replicaset.apps/istio-pilot-6dd5b8f74c                                   1         1         1       8h
istio-system   replicaset.apps/istio-policy-7f8bb87857                                  1         1         1       8h
istio-system   replicaset.apps/istio-sidecar-injector-fd5875568                         1         1         1       8h
istio-system   replicaset.apps/istio-telemetry-8759dc6b7                                1         1         1       8h
istio-system   replicaset.apps/istio-tracing-5d8f57c8ff                                 1         1         1       8h
istio-system   replicaset.apps/kiali-d4d886dd7                                          1         1         1       8h
istio-system   replicaset.apps/prometheus-d8d46c5b5                                     1         1         1       8h
kube-system    replicaset.apps/coredns-84549585c                                        2         2         2       9h
kubeflow       replicaset.apps/admission-webhook-deployment-64cb96ddbf                  1         1         1       8h
kubeflow       replicaset.apps/alb-ingress-controller-8574c47cdb                        1         1         1       8h
kubeflow       replicaset.apps/argo-ui-778676df64                                       1         1         1       8h
kubeflow       replicaset.apps/centraldashboard-7dd7dd685d                              1         1         1       8h
kubeflow       replicaset.apps/jupyter-web-app-deployment-89789fd5                      1         1         1       8h
kubeflow       replicaset.apps/katib-controller-6b789b6cb5                              1         1         1       8h
kubeflow       replicaset.apps/katib-db-manager-64f548b47c                              1         1         1       8h
kubeflow       replicaset.apps/katib-mysql-57884cb488                                   1         1         1       8h
kubeflow       replicaset.apps/katib-ui-5c5cc6bd77                                      1         1         1       8h
kubeflow       replicaset.apps/metadata-db-76c9f78f77                                   1         1         1       8h
kubeflow       replicaset.apps/metadata-deployment-674fdd976b                           1         1         1       8h
kubeflow       replicaset.apps/metadata-envoy-deployment-5688989bd6                     1         1         1       8h
kubeflow       replicaset.apps/metadata-grpc-deployment-5579bdc87b                      1         1         1       8h
kubeflow       replicaset.apps/metadata-ui-9b8cd699d                                    1         1         1       8h
kubeflow       replicaset.apps/minio-755ff748b                                          1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-79b4f85cbc                                   1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-ml-pipeline-visualizationserver-5fdffdc5bf   1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-persistenceagent-645cb66874                  1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-scheduledworkflow-6c978b6b85                 1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-ui-6995b7bccf                                1         1         1       8h
kubeflow       replicaset.apps/ml-pipeline-viewer-controller-deployment-8554dc7b9f      1         1         1       8h
kubeflow       replicaset.apps/mpi-operator-5bf8b566b7                                  1         1         1       8h
kubeflow       replicaset.apps/mysql-598bc897dc                                         1         1         1       8h
kubeflow       replicaset.apps/notebook-controller-deployment-7db57b9ccf                1         1         1       8h
kubeflow       replicaset.apps/profiles-deployment-b45dbc6f                             1         1         1       8h
kubeflow       replicaset.apps/pytorch-operator-5fd5f94bdd                              1         1         1       8h
kubeflow       replicaset.apps/seldon-controller-manager-679fc777cd                     1         1         1       8h
kubeflow       replicaset.apps/spark-operatorsparkoperator-c7b64b87f                    1         1         1       8h
kubeflow       replicaset.apps/spartakus-volunteer-6b79679599                           1         1         1       8h
kubeflow       replicaset.apps/tensorboard-6544748d94                                   1         1         1       8h
kubeflow       replicaset.apps/tf-job-operator-7d7c8fb8bb                               1         1         1       8h
kubeflow       replicaset.apps/workflow-controller-945c84565                            1         1         1       8h

NAMESPACE   NAME                                                        READY   AGE
anonymous   statefulset.apps/test1                                      1/1     8h
kubeflow    statefulset.apps/admission-webhook-bootstrap-stateful-set   1/1     8h
kubeflow    statefulset.apps/application-controller-stateful-set        1/1     8h
kubeflow    statefulset.apps/metacontroller                             1/1     8h

NAMESPACE      NAME                                                        REFERENCE                          TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
istio-system   horizontalpodautoscaler.autoscaling/cluster-local-gateway   Deployment/cluster-local-gateway   <unknown>/80%   1         5         1          8h
istio-system   horizontalpodautoscaler.autoscaling/istio-egressgateway     Deployment/istio-egressgateway     <unknown>/80%   1         5         1          8h
istio-system   horizontalpodautoscaler.autoscaling/istio-ingressgateway    Deployment/istio-ingressgateway    <unknown>/80%   1         5         1          8h
istio-system   horizontalpodautoscaler.autoscaling/istio-pilot             Deployment/istio-pilot             <unknown>/80%   1         5         1          8h
istio-system   horizontalpodautoscaler.autoscaling/istio-policy            Deployment/istio-policy            <unknown>/80%   1         5         1          8h
istio-system   horizontalpodautoscaler.autoscaling/istio-telemetry         Deployment/istio-telemetry         <unknown>/80%   1         5         1          8h

NAMESPACE      NAME                                          COMPLETIONS   DURATION   AGE
istio-system   job.batch/istio-cleanup-secrets-1.1.6         1/1           5s         8h
istio-system   job.batch/istio-grafana-post-install-1.1.6    1/1           7s         8h
istio-system   job.batch/istio-security-post-install-1.1.6   1/1           10s        8h
kubeflow       job.batch/spark-operatorcrd-cleanup           1/1           3s         8h
```

```sh
kubectl get ingress -n istio-system
NAME            HOSTS   ADDRESS                                                                 PORTS   AGE
istio-ingress   *       xxxxxx-istiosystem-istio-xxxxx-xxxxx.us-west-2.elb.amazonaws.com   80      8h
```

こんな感じでkubeflowがデプロイされてそうなことが確認できました

## ALBのセキュリティグループの編集
GitHubにも書きましたが、この状態だと無認証かつIP制限もかかっていないです。
幸いにも私の会社・自宅環境は固定IPのため、セキュリティグループでIP制限を設けました。
IPが固定できない場合などは、Kubeflowの公式ページに cognitoを使って認証を設ける方法も記載があるので、それをやってみる必要があるかもしれません
（本番環境であれば、認証・認可は必須なので、近々検証してみようと思います）

## kubeflow UIへのアクセス
ELBへアクセスすると namespaceを作成する画面になるので、初期値の anonymous というnamespaceをポチポチして作成すると以下のような画面に繋がります。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/107953/6a0e403b-3588-279b-adb2-f5a50ee3ea30.png)


# 次回
Kubeflowの機能を試していこうと思います