macOSでkubesprayを使いVagrant upするとAnsibleが失敗する場合の対処方法

kubernetesを触ってみようと思い，ちょっと前の雑誌ですがSoftware Design2018年 3月号の特集「Kubernetes入門」を見ながら，[kubespray](https://github.com/kubernetes-sigs/kubespray)を使って試してみています．

初っ端のVagrant環境構築でエラーが発生してしまったので，現象と対処方法を記録します．

## 環境

* macOS: 10.14 (Mojave)
* Vagrant: 2.2.4
* Ansible: 2.7.9
* Kubespray: master([e640233](https://github.com/kubernetes-sigs/kubespray/commit/e640233947e25e6baf6ddc3d1964e0626a924690))

## 問題

[kubesprayのリポジトリ](https://github.com/kubernetes-sigs/kubespray)をクローンして，次のようにVagrantの環境を立ち上げようとするとエラーになりました．

```zsh:
$ vagrant up
Bringing machine 'k8s-1' up with 'virtualbox' provider...
Bringing machine 'k8s-2' up with 'virtualbox' provider...
Bringing machine 'k8s-3' up with 'virtualbox' provider...
==> k8s-1: Cloning VM...

...

==> k8s-3: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.7.9).

Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode

    k8s-3: Running ansible-playbook...
SUDO password:
 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'


PLAY [localhost] ***************************************************************
skipping: no hosts matched
 [WARNING]: Could not match supplied host pattern, ignoring: bastion

...

PLAY RECAP *********************************************************************

Wednesday 20 March 2019  10:13:21 +0900 (0:00:00.113)       0:00:00.113 *******
===============================================================================
```

Ansibleの実行が全部skippingになってしまいました．

## 対処方法

`inventory/sample/hosts.ini`をリネームして退避することでAnsibleがskippingされずに走るようになりましたが，まだエラーが発生しました．

```zsh:
$ mv inventory/sample/{hosts.ini,hosts.ini.bak}
$ vagrant provision

...

TASK [download : container_download | copy container images to ansible host] ***
Wednesday 20 March 2019  10:28:38 +0900 (0:00:02.034)       0:07:21.903 *******
fatal: [k8s-1 -> localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh=/usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null --out-format=<<CHANGED>>%i %n%L /tmp/releases/containers/gcr.io_google_containers_cluster-proportional-autoscaler-amd64:1.3.0.tar /tmp/releases/containers/gcr.io_google_containers_cluster-proportional-autoscaler-amd64:1.3.0.tar", "msg": "rsync: link_stat \"/tmp/releases/containers/gcr.io_google_containers_cluster-proportional-autoscaler-amd64:1.3.0.tar\" failed: No such file or directory (2)\nrsync: push_dir#3 \"/tmp/releases/containers\" failed: No such file or directory (2)\nrsync error: errors selecting input/output files, dirs (code 3) at /BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-52.200.1/rsync/main.c(581) [receiver=2.6.9]\nrsync: connection unexpectedly closed (8 bytes received so far) [sender]\nrsync error: error in rsync protocol data stream (code 12) at /BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-52.200.1/rsync/io.c(453) [sender=2.6.9]\n", "rc": 12}

NO MORE HOSTS LEFT *************************************************************
        to retry, use: --limit @/Users/sakao/src/github.com/kubernetes-sigs/kubespray/cluster.retry

PLAY RECAP *********************************************************************
k8s-1                      : ok=212  changed=42   unreachable=0    failed=1
k8s-2                      : ok=130  changed=31   unreachable=0    failed=0
k8s-3                      : ok=127  changed=31   unreachable=0    failed=0

...

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
```

そこで，`Vagrantfile`の`download_run_once`を`False`に変更し，再実行することで正しくAnsibleが実行完了しました．

```zsh:
$ sed -i -e 's/"download_run_once": "True",/"download_run_once": "False",/' Vagrantfile
$ vagrant provision

...

PLAY RECAP *********************************************************************
k8s-1                      : ok=363  changed=73   unreachable=0    failed=0
k8s-2                      : ok=318  changed=68   unreachable=0    failed=0
k8s-3                      : ok=283  changed=55   unreachable=0    failed=0

...

```

## 原因

* [#3748のイシュー](https://github.com/kubernetes-sigs/kubespray/issues/3748)にあるように，Ansibleがデフォルトでiniファイルを見てしまい，Vagrantfileで指定しているインベントリが無視されるようです（たぶん）
* [#3749のイシュー](https://github.com/kubernetes-sigs/kubespray/issues/3749)にあるように，localのrsyncが悪さをしてしまうようです（たぶん）

## 留意点

* `SUDO password:`には`vagrant`と打ちました
* 関係あるかわかりませんが，Software Designの記事に倣って，Ansible設定ファイルを2点変更して実行しています

```zsh:
$ git diff -p -U0 inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml roles/network_plugin/flannel/defaults/main.yml | cat
diff --git a/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml b/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
index 09727b33..f0e8afea 100644
--- a/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
+++ b/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml
@@ -176 +176 @@ podsecuritypolicy_enabled: false
-# kubeconfig_localhost: false
+kubeconfig_localhost: true
diff --git a/roles/network_plugin/flannel/defaults/main.yml b/roles/network_plugin/flannel/defaults/main.yml
index f7f773fd..aec4c5da 100644
--- a/roles/network_plugin/flannel/defaults/main.yml
+++ b/roles/network_plugin/flannel/defaults/main.yml
@@ -9 +9 @@
-# flannel_interface:
+flannel_interface: eth1
```

* プロビジョニングには10分くらい時間がかかるので気長に待ちましょう☕