macOSでMinishiftを起動してサンプルアプリケーションをデプロイ

macOSにMinishiftを入れてサンプルアプリケーションをデプロイしてみたメモ。Minishiftのインストール手順は以下の公式ドキュメントに書いてある通り。

- [Getting Started with Minishift](https://docs.okd.io/latest/minishift/getting-started/index.html)

## 仮想環境のセットアップ

macOSでは、仮想環境としては以下が利用できる。

- xhyve driver
- hyperkit driver
- VirtualBox

VirtualBoxのバージョン6以上がインストールされている場合、xhyve driverは上手く動かないように思われる（2019年5月時点）。そのためVirtualBoxを使ったほうが簡単と思われる。hyperkit driverは未確認。[v1.34.0から？](https://github.com/minishift/minishift/issues/1326)

### xhyve Driverの場合（失敗）

- [Setting Up the xhyve Driver](https://docs.okd.io/latest/minishift/getting-started/setting-up-virtualization-environment.html#setting-up-xhyve-driver)

`docker-machine-driver-xhyve`をインストールする。

```shell
brew install docker-machine-driver-xhyve
```

もともと入っているDockerの`docker-machine`と競合が発生した。

```shell-session
$ ls -l /usr/local/bin/docker-machine*
lrwxr-xr-x  1 sotoiwa  staff  62  1 21 01:58 /usr/local/bin/docker-machine -> /Applications/Docker.app/Contents/Resources/bin/docker-machine
lrwxr-xr-x  1 sotoiwa  admin  77  1 23 15:46 /usr/local/bin/docker-machine-driver-xhyve -> ../Cellar/docker-machine-driver-xhyve/0.3.3_1/bin/docker-machine-driver-xhyve
$
```

そのままでよいような気もするが、念のため以下コマンドでシンボリックリンクを上書きしておく。

```shell
brew link --overwrite docker-machine
```

```shell-session
$ ls -l /usr/local/bin/docker-machine*
lrwxr-xr-x  1 sotoiwa  admin  50  1 23 15:58 /usr/local/bin/docker-machine -> ../Cellar/docker-machine/0.16.1/bin/docker-machine
lrwxr-xr-x  1 sotoiwa  admin  77  1 23 15:46 /usr/local/bin/docker-machine-driver-xhyve -> ../Cellar/docker-machine-driver-xhyve/0.3.3_1/bin/docker-machine-driver-xhyve
```

以下のコマンドを実行。

```shell
sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
```

### VirtualBoxの場合

[インストーラー](https://www.virtualbox.org/wiki/Downloads)または`brew cask`でインストールする。

```shell
brew cask install virtualbox
```

## Minishiftのインストール

- [Installing Minishift](https://docs.okd.io/latest/minishift/getting-started/installing.html)

Minishiftをインストールする。

```shell
brew cask install minishift
```

## Minishiftの起動

### xhyve Driverの場合（失敗）

Minishiftを起動する。デフォルトだとxhyveが使用される。

```shell
minishift start
```

試した時点のバージョンでは失敗。

```shell-session
$ minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'xhyve' is supported on this platform ... OK
-- Checking if xhyve driver is installed ...
   Driver is available at /usr/local/bin/docker-machine-driver-xhyve
   Checking for setuid bit ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'xhyve' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Starting Minishift VM ..... FAIL E0213 23:16:15.825559   34039 start.go:494] Error starting the VM: Error creating the VM. Error with pre-create check: "Virtual Box version 4 or lower will cause a kernel panicif xhyve tries to run. You are running version: 6.0.4r128413\n\n\t Please upgrade to version 5 at https://www.virtualbox.org/wiki/Downloads". Retrying.
Error starting the VM: Error creating the VM. Error with pre-create check: "Virtual Box version 4 or lower will cause a kernel panicif xhyve tries to run. You are running version: 6.0.4r128413\n\n\t Please upgrade to version 5 at https://www.virtualbox.org/wiki/Downloads"
$
```

VirtualBoxのバージョン4以下がインストールされているとカーネルパニックが起きるため、VirtualBoxのバージョン5であるかをチェックしているが、バージョン6には対応できていないため。

- [Check for VirtualBox version fails with version 6 while xhyve is installed on macOS](https://github.com/minishift/minishift/issues/3128)
- [Driver not able to validate newer version of Virtualbox as pre flight check.](https://github.com/machine-drivers/docker-machine-driver-xhyve/issues/213)

（補足）
Issueは修正され、`docker-machine-driver-xhyve`の`0.4.0`がリリースされたので試してみたが別のエラーとなり起動しなかった。

- https://github.com/machine-drivers/docker-machine-driver-xhyve/releases

```shell-session
-- Starting Minishift VM .... FAIL E0519 23:37:54.847986   57830 start.go:494] Error starting the VM: Error creating new host: json: cannot unmarshal bool into Go struct field Driver.Virtio9p of type []string. Retrying.
Error starting the VM: Error creating new host: json: cannot unmarshal bool into Go struct field Driver.Virtio9p of type []string
```

xhyve Driverをアンインストールする場合は以下のようにする。

```shell
brew uninstall docker-machine-driver-xhyve
brew uninstall docker-machine
# docker-machineコマンドのシンボリックリンクを上書きした場合
ln -s /Applications/Docker.app/Contents/Resources/bin/docker-machine /usr/local/bin/docker-machine
```

### VirtualBoxの場合

引数で`--vm-driver=virtualbox`を指定して起動する。あるいは`minishift config set vm-driver virtualbox`で恒久的に設定することも可能。

```shell-session
$ minishift start --vm-driver=virtualbox
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'virtualbox' is supported on this platform ... OK
-- Checking if VirtualBox is installed ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'virtualbox' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Starting Minishift VM ............................ OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ...
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... OK
   Importing 'openshift/origin-control-plane:v3.11.0' . CACHE MISS
   Importing 'openshift/origin-docker-registry:v3.11.0' . CACHE MISS
   Importing 'openshift/origin-haproxy-router:v3.11.0' . CACHE MISS
-- OpenShift cluster will be configured with ...
   Version: v3.11.0
-- Pulling the OpenShift Container Image ............ OK
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster ..........................................................
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Pulling image openshift/origin-cli:v3.11.0
E0213 15:04:42.947106    5257 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously
Image pull complete
Pulling image openshift/origin-node:v3.11.0
E0213 15:04:46.120467    5257 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously
Pulled 5/6 layers, 92% complete
Pulled 6/6 layers, 100% complete
Extracting
Image pull complete
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP 192.168.99.116 as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I0213 15:05:02.558105    5257 config.go:40] Running "create-master-config"
I0213 15:05:04.580859    5257 config.go:46] Running "create-node-config"
I0213 15:05:05.103245    5257 flags.go:30] Running "create-kubelet-flags"
I0213 15:05:05.401541    5257 run_kubelet.go:49] Running "start-kubelet"
I0213 15:05:05.552931    5257 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0213 15:06:04.567287    5257 interface.go:26] Installing "kube-proxy" ...
I0213 15:06:04.568078    5257 interface.go:26] Installing "kube-dns" ...
I0213 15:06:04.568086    5257 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I0213 15:06:04.568090    5257 interface.go:26] Installing "openshift-apiserver" ...
I0213 15:06:04.568118    5257 apply_template.go:81] Installing "openshift-apiserver"
I0213 15:06:04.568262    5257 apply_template.go:81] Installing "kube-proxy"
I0213 15:06:04.569145    5257 apply_template.go:81] Installing "kube-dns"
I0213 15:06:04.569278    5257 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I0213 15:06:07.855696    5257 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I0213 15:08:00.979513    5257 run_self_hosted.go:242] openshift-apiserver available
I0213 15:08:00.980273    5257 interface.go:26] Installing "openshift-controller-manager" ...
I0213 15:08:00.980287    5257 apply_template.go:81] Installing "openshift-controller-manager"
I0213 15:08:03.607186    5257 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding router ...
Adding sample-templates ...
Adding web-console ...
Adding centos-imagestreams ...
Adding registry ...
Adding persistent-volumes ...
I0213 15:08:03.626856    5257 interface.go:26] Installing "openshift-router" ...
I0213 15:08:03.626864    5257 interface.go:26] Installing "sample-templates" ...
I0213 15:08:03.626868    5257 interface.go:26] Installing "openshift-web-console-operator" ...
I0213 15:08:03.626872    5257 interface.go:26] Installing "centos-imagestreams" ...
I0213 15:08:03.626877    5257 interface.go:26] Installing "openshift-image-registry" ...
I0213 15:08:03.626880    5257 interface.go:26] Installing "persistent-volumes" ...
I0213 15:08:03.627602    5257 interface.go:26] Installing "sample-templates/mongodb" ...
I0213 15:08:03.627611    5257 interface.go:26] Installing "sample-templates/mariadb" ...
I0213 15:08:03.627615    5257 interface.go:26] Installing "sample-templates/django quickstart" ...
I0213 15:08:03.627619    5257 interface.go:26] Installing "sample-templates/sample pipeline" ...
I0213 15:08:03.627622    5257 interface.go:26] Installing "sample-templates/mysql" ...
I0213 15:08:03.627625    5257 interface.go:26] Installing "sample-templates/postgresql" ...
I0213 15:08:03.627628    5257 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I0213 15:08:03.627632    5257 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I0213 15:08:03.627635    5257 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I0213 15:08:03.627639    5257 interface.go:26] Installing "sample-templates/rails quickstart" ...
I0213 15:08:03.627642    5257 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I0213 15:08:03.627693    5257 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I0213 15:08:03.627895    5257 apply_template.go:81] Installing "openshift-web-console-operator"
I0213 15:08:03.628044    5257 apply_list.go:67] Installing "centos-imagestreams"
I0213 15:08:03.628301    5257 apply_list.go:67] Installing "sample-templates/mongodb"
I0213 15:08:03.628377    5257 apply_list.go:67] Installing "sample-templates/mariadb"
I0213 15:08:03.628512    5257 apply_list.go:67] Installing "sample-templates/django quickstart"
I0213 15:08:03.628610    5257 apply_list.go:67] Installing "sample-templates/sample pipeline"
I0213 15:08:03.628684    5257 apply_list.go:67] Installing "sample-templates/mysql"
I0213 15:08:03.628746    5257 apply_list.go:67] Installing "sample-templates/postgresql"
I0213 15:08:03.628810    5257 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I0213 15:08:03.628871    5257 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I0213 15:08:03.628934    5257 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I0213 15:08:03.629006    5257 apply_list.go:67] Installing "sample-templates/rails quickstart"
I0213 15:08:16.318545    5257 interface.go:41] Finished installing "sample-templates/mongodb" "sample-templates/mariadb" "sample-templates/django quickstart" "sample-templates/sample pipeline" "sample-templates/mysql" "sample-templates/postgresql" "sample-templates/cakephp quickstart" "sample-templates/dancer quickstart" "sample-templates/nodejs quickstart" "sample-templates/rails quickstart" "sample-templates/jenkins pipeline ephemeral"
I0213 15:08:58.185099    5257 interface.go:41] Finished installing "openshift-router" "sample-templates" "openshift-web-console-operator" "centos-imagestreams" "openshift-image-registry" "persistent-volumes"
Login to server ...
Creating initial project "myproject" ...
Server Information ...
OpenShift server started.

The server is accessible via web console at:
    https://192.168.99.116:8443/console

You are logged in as:
    User:     developer
    Password: <any value>

To login as administrator:
    oc login -u system:admin


-- Exporting of OpenShift images is occuring in background process with pid 36439.
$
```

`dial tcp 192.168.99.124:2376: i/o timeout`のようなエラーで起動に失敗する場合、以下が参考になるかもしれない。

- [minikube startがエラーになる場合の対応方法](https://qiita.com/sotoiwa/items/18053c86ce7a47faca83#%E8%BF%BD%E8%A8%98)

configに`vm-driver`が指定されるので、次回は引数を指定しなくても`virtualbox`が使用される。`minishift config set`で事前に設定することも可能。

```
$ minishift config view
- vm-driver                          : virtualbox
$
```

## ログイン

インストール時のログメッセージに記載のURLにアクセスし、ログインする。`minishift console`コマンドでもブラウザを起動してコンソールを開くことができる。

<img width="1145" alt="image.png" src="https://qiita-image-store.s3.amazonaws.com/0/86449/3e8ba6e6-c4f5-a30f-696e-1c233f0376a5.png">

<img width="1145" alt="image.png" src="https://qiita-image-store.s3.amazonaws.com/0/86449/12bf8bf3-d3c2-876a-3e69-c68108c7d235.png">

コマンドラインの場合、まず`oc`コマンドにPATHを通す。

```shell
eval $(minishift oc-env)
```

既に`developer`でログインした状態になっているが、管理者としてログインする場合は以下のコマンドを実行する。

```shell
oc login -u system:admin
```

以下でコマンドの補完を有効にできる。

```shell
oc completion bash > bash_completion.sh
source bash_completion.sh
```

## サンプルアプリケーションをデプロイ

以下のサンプルアプリケーションをデプロイしてみる。

- https://github.com/sclorg/nodejs-ex

`developer`でログインし直す。

```shell
oc login -u developer
```

`nodejs-echo`という新しいプロジェクトを作成する。

```shell:コマンド
oc new-project nodejs-echo \
  --display-name="nodejs" \
  --description="Sample Node.js app"
```

```shell-session:実行例
$ oc new-project nodejs-echo \
>   --display-name="nodejs" \
>   --description="Sample Node.js app"
Already on project "nodejs-echo" on server "https://192.168.99.127:8443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git

to build a new example application in Ruby.
$
```

新しいアプリケーションを作成する。

```shell:コマンド
oc new-app https://github.com/sclorg/nodejs-ex -l name=myapp
```

```shell-session:実行例
$ oc new-app https://github.com/sclorg/nodejs-ex -l name=myapp
--> Found image 93de123 (7 months old) in image stream "openshift/nodejs" under tag "10" for "nodejs"

    Node.js 10.12.0
    ---------------
    Node.js  available as docker container is a base platform for building and running various Node.js  applications and frameworks. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

    Tags: builder, nodejs, nodejs-10.12.0

    * The source repository appears to match: nodejs
    * A source build using source code from https://github.com/sclorg/nodejs-ex will be created
      * The resulting image will be pushed to image stream tag "nodejs-ex:latest"
      * Use 'start-build' to trigger a new build
    * This image will be deployed in deployment config "nodejs-ex"
    * Port 8080/tcp will be load balanced by service "nodejs-ex"
      * Other containers can access this service through the hostname "nodejs-ex"

--> Creating resources with label name=myapp ...
    imagestream.image.openshift.io "nodejs-ex" created
    buildconfig.build.openshift.io "nodejs-ex" created
    deploymentconfig.apps.openshift.io "nodejs-ex" created
    service "nodejs-ex" created
--> Success
    Build scheduled, use 'oc logs -f bc/nodejs-ex' to track its progress.
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/nodejs-ex'
    Run 'oc status' to view your app.
$
```

アプリケーションの状況と作成されたリソースを確認する。

```shell-session
$ oc status
In project nodejs (nodejs-echo) on server https://192.168.99.127:8443

http://www.example.com to pod port 8080-tcp (svc/nodejs-ex)
  dc/nodejs-ex deploys istag/nodejs-ex:latest <-
    bc/nodejs-ex source builds https://github.com/sclorg/nodejs-ex on openshift/nodejs:10
    deployment #1 deployed 14 minutes ago - 1 pod


2 infos identified, use 'oc status --suggest' to see details.
$ oc get imagestream
NAME        DOCKER REPO                             TAGS      UPDATED
nodejs-ex   172.30.1.1:5000/nodejs-echo/nodejs-ex   latest    14 minutes ago
$ oc get buildconfig
NAME        TYPE      FROM      LATEST
nodejs-ex   Source    Git       1
$ oc get deploymentconfig
NAME        REVISION   DESIRED   CURRENT   TRIGGERED BY
nodejs-ex   1          1         1         config,image(nodejs-ex:latest)
$ oc get svc
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
nodejs-ex   ClusterIP   172.30.98.162   <none>        8080/TCP   16m
$ oc get po
NAME                READY     STATUS      RESTARTS   AGE
nodejs-ex-1-build   0/1       Completed   0          16m
nodejs-ex-1-hrnbd   1/1       Running     0          15m
$
```

アプリケーションを外部に公開する。

```shell
oc expose svc/nodejs-ex --hostname=www.example.com
```

確認する。

```shell-session
$ oc get route
NAME        HOST/PORT         PATH      SERVICES    PORT       TERMINATION   WILDCARD
nodejs-ex   www.example.com             nodejs-ex   8080-tcp                 None
$
```

MinishiftのIPアドレスを確認する。

```shell
minishift ip
```

`/etc/hosts`に以下を追加する。

```
<Minishiftのip> www.example.com
```

ブラウザで以下にアクセスし、アプリケーションが稼働していることを確認する。

- http://www.example.com

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/86449/9ec989c2-6148-bb5a-63d9-db0e6539a052.png)


## 参考リンク

- [OS XのネイティブHypervisorを使うxhyveと、ネイティブDockerを立ち上げるdocker-machine-driver-xhyveを作った話](https://qiita.com/zchee/items/cb4bb68a0aab12dfa2c1)
- [Uninstalling Minishift](https://docs.okd.io/latest/minishift/getting-started/uninstalling.html)
- [minishiftを動かしてみた！](https://qiita.com/Ackkeee/items/1fba9deafc5ec823e17b)
- [macOS で minishift [導入から起動まで]](https://qiita.com/ryuichi1208/items/8df4fc39392d40775e2d)