Kubeflow with Rancher

RancherのKubernetesクラスタインポート機能とカタログ機能を利用して、GKEのGPUクラスタをインポートしてKubeflowをデプロイしてみたいと思います。

以下の環境を構築してみます。

![pic01.png](https://qiita-image-store.s3.amazonaws.com/0/61606/eba70795-bc27-a567-2ac5-1c8ad3c085e2.png)

# 1.Rancher Server Setup

## 1.Instance準備

GCE(Google Compute Engine)で以下のインスタンスを作成します。

|項目               |入力概要        |
|:----------------------------------|:---------------------------------------------|
|名前|rancher-server|
|リージョン|asia-northeast1(東京)|
|ゾーン|asia-northeast1-b|
|マシンタイプ|vCPUx1|
|ブートディスク|Ubuntu 16.04 LTS<br>ディスクサイズ80GB|
|ファイアウォール|HTTP トラフィックを許可する、HTTPS トラフィックを許可するの両方をチェックします。|

## 2.Dockerのインストール

```console:コマンド
$ curl https://releases.rancher.com/install-docker/18.09.sh | sh
・
・（省略）
・
+ sudo -E sh -c docker version
Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        774a1f4
 Built:             Thu Feb 28 06:40:58 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.3
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       774a1f4
  Built:            Thu Feb 28 05:59:55 2019
  OS/Arch:          linux/amd64
  Experimental:     false

If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:

  sudo usermod -aG docker iyutaka2018

Remember that you will have to log out and back in for this to take effect!

WARNING: Adding a user to the "docker" group will grant the ability to run
         containers which can be used to obtain root privileges on the
         docker host.
         Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
         for more information.

If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:

  sudo usermod -aG docker iyutaka2018

Remember that you will have to log out and back in for this to take effect!

WARNING: Adding a user to the "docker" group will grant the ability to run
         containers which can be used to obtain root privileges on the
         docker host.
         Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
         for more information.
```

## 3.Rancher Serverのインストール

```console:コマンド
$ sudo docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher
Unable to find image 'rancher/rancher:latest' locally
latest: Pulling from rancher/rancher
38e2e6cd5626: Pull complete 
705054bc3f5b: Pull complete 
c7051e069564: Pull complete 
7308e914506c: Pull complete 
0cfcb3cfb94b: Pull complete 
49cb3551f487: Pull complete 
8856f5defe68: Pull complete 
d50abe29b623: Pull complete 
297145c80f79: Pull complete 
b6b66b1777e8: Pull complete 
7edf8d37c2b7: Pull complete 
511c877e7916: Pull complete 
Digest: sha256:924b8acaa169821c86b840c33e1d79d87db0dfbb84dae6c102cc7c196811230f
Status: Downloaded newer image for rancher/rancher:latest
c0fb3caa9f608ef25f38b60f67fcb0fb2e567e53b72c5aef58e5f800943359f6
```

## 4.ブラウザを起動して、RancherUIにアクセス

https://EXTERNAL-IP/

## 5.初回ログイン

### a.adminユーザのパスワード作成

任意のパスワードを入力
![pic02.png](https://qiita-image-store.s3.amazonaws.com/0/61606/21367d20-c637-31e6-157f-4bbc022683ce.png)

### b.Rancher Server URLの登録

「Save URL」ボタンを押下
![03pic.png](https://qiita-image-store.s3.amazonaws.com/0/61606/3bcc5713-87bd-4fd0-bf02-366fb3893e24.png)

### c.ログイン完了

![pic04.png](https://qiita-image-store.s3.amazonaws.com/0/61606/f4298cec-13c8-feee-ec5e-3dfc7c13cc81.png)

# 3.GKE GPU Cluster Set up

GCPのダッシュボードでクラスタを作成します。
![pic05.png](https://qiita-image-store.s3.amazonaws.com/0/61606/693e2e8f-2f0d-e5f9-d89e-f71b910dd4c2.png)

クラスタが完成したことを確認します。
![pic06.png](https://qiita-image-store.s3.amazonaws.com/0/61606/836ee8c3-e74d-fc09-6481-bb644b933077.png)

# 4.GKE Kubernetes Cluster Import

## 1.「Add Cluster」ボタンを押下
![pic07.png](https://qiita-image-store.s3.amazonaws.com/0/61606/42c86923-83fd-6bb3-f836-9270e6637898.png)


## 2.「Import」を選択して、Nameに任意名を入力して、「Create」ボタンを押下
![pic08.png](https://qiita-image-store.s3.amazonaws.com/0/61606/a04e5a6f-28e5-1f05-625e-2f13cdc125f2.png)

## 3.1,2をGCPのCloud Shellで実行後、「Done」ボタンを押下
![pic09.png](https://qiita-image-store.s3.amazonaws.com/0/61606/904c3443-40b2-ec6b-d534-ea788f6671d5.png)

## 4.GCPでCloud Shellを起動

「接続」ボタンを押下
![pic10.png](https://qiita-image-store.s3.amazonaws.com/0/61606/ad80b2b9-0505-b4e4-aa77-4e664e341c9c.png)

「Cloud Shellで実行」ボタンを押下
![pic11.png](https://qiita-image-store.s3.amazonaws.com/0/61606/0e2e71ca-77f1-244c-22b3-ce23591fdf46.png)

![pic12.png](https://qiita-image-store.s3.amazonaws.com/0/61606/e75adb06-089f-d81f-f57e-f545f6f302b7.png)

```console:コマンド
Your Cloud Platform project in this session is set to natural-nimbus-228405.
Use “gcloud config set project [PROJECT_ID]” to change to a different project.
$ gcloud container clusters get-credentials gpu-cluster-1 --zone asia-northeast1-b --project natural-nimbus-xxxxxxx ←Enterキーを押下
Fetching cluster endpoint and auth data.
kubeconfig entry generated for gpu-cluster-1.
```

[USER_ACCOUNT]は、GCPのアカウントになります。

```console:コマンド
$ kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user [USER_ACCOUNT]
clusterrolebinding.rbac.authorization.k8s.io "cluster-admin-binding" created
```

インポートコマンドを実行

```console:コマンド
$ curl --insecure -sfL https://34.85.63.110/v3/import/2475dq9xlw75qcztw7gztnngxvvhrjnklvdxx7t8t6xk8d8fn9kd65.yaml | kubectl apply -f -
namespace "cattle-system" created
serviceaccount "cattle" created
clusterrolebinding.rbac.authorization.k8s.io "cattle-admin-binding" created
secret "cattle-credentials-2c8898b" created
clusterrole.rbac.authorization.k8s.io "cattle-admin" created
deployment.extensions "cattle-cluster-agent" created
daemonset.extensions "cattle-node-agent" created
```

## 5.クラスタ名をクリック

![pic13.png](https://qiita-image-store.s3.amazonaws.com/0/61606/e8a6fc9b-29aa-eba9-87d2-89e590d13e1a.png)

## 6.リソース状況を確認

![pic14.png](https://qiita-image-store.s3.amazonaws.com/0/61606/0d92ab61-f3eb-bbcb-97c2-dca5d5f1bfe0.png)

# 5.Kubeflow Deploy from Catalog

## 1.上部メニュー「gke-kubernetes-cluster」-「Default」を選択

![pin15.png](https://qiita-image-store.s3.amazonaws.com/0/61606/57b538cd-5178-a61d-677e-c04dedde5ef7.png)


## 2.上部メニュー「Catalog Apps」を選択して、「Launch」ボタンを押下
![pic16.png](https://qiita-image-store.s3.amazonaws.com/0/61606/a881bd92-b407-d29c-8dd5-edcc06983f86.png)


## 3.検索フィールドで「kubeflow」と入力して、「View Details」ボタンを押下

![pic17.png](https://qiita-image-store.s3.amazonaws.com/0/61606/d458dc6b-56eb-9aff-8d30-c9ee3a9dbf2a.png)

## 4.全てデフォルト設定のまま「Launch」ボタンを押下

![pic18.png](https://qiita-image-store.s3.amazonaws.com/0/61606/e778a24e-5681-0031-a424-0c9357f28f31.png)

## 5.デプロイ完了後、ポート番号/tcpをクリック
![pic19.png](https://qiita-image-store.s3.amazonaws.com/0/61606/be19e89d-0449-36b0-bfc8-96f129fad578.png)

![pic20.png](https://qiita-image-store.s3.amazonaws.com/0/61606/4e81ff33-3958-3af0-324a-9c10742cdad0.png)

## 6.Kubeflowダッシュボード表示後、「JUPTERHUB」を選択
![pic21.png](https://qiita-image-store.s3.amazonaws.com/0/61606/4cffc88c-b73a-6777-a1d3-a76dbf5eab63.png)

## 7.「Username」,「Password」に任意のものを入力して、「Sign In」ボタンを押下
![pic22.png](https://qiita-image-store.s3.amazonaws.com/0/61606/60a06e2b-8ad2-2443-24b4-5d4e68815336.png)

## 8.「gcr.io/kubeflow-images-public/tensorflow-1.10.1-notebook-cpu:v0.3.1」を選択して「Spawn」ボタンを押下
![pic23.png](https://qiita-image-store.s3.amazonaws.com/0/61606/7504f0ea-240c-af98-4763-56bac3b79105.png)

## 9.「New」-「Python3」を選択
![pic24](https://qiita-image-store.s3.amazonaws.com/0/61606/2e80004d-a209-3250-a026-524d3cca81ac.png)

## 10.TensorFlowの公式チュートリアルを実行

```py:code
import tensorflow as tf
mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)
```

「Run」ボタンを押下
![pic25.png](https://qiita-image-store.s3.amazonaws.com/0/61606/031df2c5-348a-66e1-54e1-992425bd4675.png)

## 11.「KUBEFLOW DOCS」「TFJOB DASHBOARD」を確認
![pic26.png](https://qiita-image-store.s3.amazonaws.com/0/61606/44854938-3323-df9e-6034-43b91ccc64b2.png)

以上でRancherのカタログ機能を利用したKubeflowのデプロイからサンプルコードの実行までとなります。Kubeflowカタログは、まだEXPERIMENTALということでなのかはわかりませんが、GKEにGPUクラスタの設定はまだできないようなのでクラスタをインポートする形となりました。今後も色々と改善されるかもしれません。

ただ、公式のデプロイ手順は結構実行することが多いので、それに比べると非常に簡単であることは確かです。

正直、KubeflowやTensorflowは初心者なのでこの環境がどの程度のものかは不明です。有識者の方に試していただいてフィードバックなど頂けると幸いです。

参考資料

- [Let's Start Cloud Native with Rancher !! "Featuring Rancher2.2 Preview & Kubeflow"](https://speakerdeck.com/cyberblack28/lets-start-cloud-native-with-rancher-featuring-rancher-2-dot-2-tech-preview-and-kubeflow)
- [Rancherでkubeflow構築](https://speakerdeck.com/nakayamam/ranchertekubeflowgou-zhu)