Chef検証(3) Chef Client セットアップ② ＠Cisco Nexus9000

## ユーザ設定

- Chef Node(Chef Client) である Cisco N9K にユーザを作成します。このユーザは Chef Workstation から Chef Node に SSH でログインする際に使用します。

```
[guestshell@chefn9k ~]$ sudo useradd chefuser01
[guestshell@chefn9k ~]$ sudo passwd chefuser01
Changing password for user chefuser01.
New password:
BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word
Retype new password:
passwd: all authentication tokens updated successfully.
```

- chefuser01 に sudo 経由で root を使う権限を付与します。
    - chefuser01 の2行を追加。

```
## Allow root to run any commands anywhere
root    ALL=(ALL)       ALL
chefuser01 ALL=(ALL) ALL
chefuser01 ALL=(ALL) NOPASSWD: ALL
```

[参考]この手順と同じ
http://qiita.com/inoko/items/09b9a15cb1a5c83fed34

## Bootstrap

- 予め、Chef Node にて Chef Workstation(ino-ubuntu) の名前解決ができるようにしておきます。
    - 172.22.150.100 を追加。

```
[guestshell@chefn9k ~]$ cat /etc/hosts
127.0.0.1 localhost guestshell
172.22.150.100 ino-ubuntsu.iselab.local ino-ubuntu
[guestshell@chefn9k ~]$
```

- 怒られました。

```
inohana@ino-ubuntu:~/repo/.chef$ knife bootstrap ino-ubuntu.iselab.local -x chefuser01 --sudo
Doing old-style registration with the validation key at /home/inohana/repo/.chef/ise-validator.pem...
Delete your validation key in order to use your user credentials instead

Connecting to ino-ubuntu.iselab.local
ERROR: Errno::EACCES: Permission denied @ rb_sysopen - /home/inohana/repo/.chef/ise-validator.pem

inohana@ino-ubuntu:~/repo/.chef$ ls -l
total 16
-rw------- 1 root root 1678 Mar  2 04:53 inohana.pem
-rw------- 1 root root 1678 Mar  2 04:53 ise-validator.pem
-rw-r--r-- 1 root root  667 Mar  2 05:01 knife.rb
drwxr-xr-x 2 root root 4096 Mar  2 05:01 trusted_certs
```
- ise-validator.pem の権限を変更。

```
inohana@ino-ubuntu:~/repo/.chef$ sudo chown -R inohana *
[sudo] password for inohana:
inohana@ino-ubuntu:~/repo/.chef$ ls -l
total 16
-rw------- 1 inohana root 1678 Mar  2 04:53 inohana.pem
-rw------- 1 inohana root 1678 Mar  2 04:53 ise-validator.pem
-rw-r--r-- 1 inohana root  667 Mar  2 05:01 knife.rb
drwxr-xr-x 2 inohana root 4096 Mar  2 05:01 trusted_certs
```

- Workstation の Hosts に書いているのは N9K スイッチの IP アドレスであるため、アクセスできない模様です。

```
inohana@ino-ubuntu:~/repo/.chef$ knife bootstrap ino-ubuntu.iselab.local -x chefuser01 --sudo
Doing old-style registration with the validation key at /home/inohana/repo/.chef/ise-validator.pem...
Delete your validation key in order to use your user credentials instead

Connecting to ino-ubuntu.iselab.local
chefuser01@ino-ubuntu.iselab.local's password:
chefuser01@ino-ubuntu.iselab.local's password:
chefuser01@ino-ubuntu.iselab.local's password:
Failed to authenticate chefuser01 - trying password auth

# 入れません！

ERROR: Net::SSH::AuthenticationFailed: Authentication failed for user 
```

----------------------------------
- 以下リンクの「Pre-Configured SSHD Service」をみてください。
    - [参考：Cisco Nexus 9000 Series NX-OS Programmability Guide, Release 7.x](
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/programmability/guide/b_Cisco_Nexus_9000_Series_NX-OS_Programmability_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_Programmability_Guide_7x_chapter_01100.html#task_7BC990F49E094646850F160AB0014D6A)
    - マニュアルにちょっと誤植があるので注意。
 
----------------------------------

- /usr/lib/systemd/system/sshd/service ファイルをコピー。
    - コピーしたファイル sshd-mgmt.service を編集。

```

[guestshell@chefn9k system]$ pwd
/usr/lib/systemd/system

[guestshell@chefn9k system]$ sudo cp sshd.service sshd-mgmt.service

[guestshell@chefn9k system]$ ls -la sshd-*
-rw-r--r-- 1 root root 375 Feb 27 06:13 sshd-cisco.service
-rw-r--r-- 1 root root 307 Feb 27 06:13 sshd-keygen.service
-rw-r--r-- 1 root root 316 Mar 10 02:01 sshd-mgmt.service

[guestshell@chefn9k system]$ cat sshd-mgmt.service
[Unit]
Description=OpenSSH server daemon
After=network.target sshd-keygen.service
Wants=sshd-keygen.service

[Service]
EnvironmentFile=/etc/sysconfig/sshd
ExecStart=/usr/sbin/sshd -D $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

# 以下のようにファイルを編集します。

[guestshell@chefn9k system]$ cat sshd-mgmt.service
[Unit]
Description=OpenSSH server daemon
After=network.target sshd-keygen.service
Wants=sshd-keygen.service

[Service]
EnvironmentFile=/etc/sysconfig/sshd
ExecStartPre=/usr/sbin/sshd-keygen
ExecStart=/sbin/ip netns exec management /usr/sbin/sshd -f /etc/ssh/sshd-mgmt_config -D $OPTIONS
#ExecStart=/usr/sbin/sshd -D $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target
```
- /etc/ssh/sshd_config ファイルをコピー。
    - コピーしたファイル sshd-mgmt_config を編集。

```
[guestshell@chefn9k system]$ cd /etc/ssh
[guestshell@chefn9k ssh]$ ls -la
total 275
drwxr-xr-x  2 root root       1024 Mar  2 03:22 .
drwxr-xr-x 56 root root       5120 Mar  3 06:51 ..
-rw-r--r--  1 root root     242153 Feb 27 06:13 moduli
-rw-r--r--  1 root root       2208 Feb 27 06:13 ssh_config
-rw-r-----  1 root ssh_keys    227 Feb 27 06:13 ssh_host_ecdsa_key
-rw-r--r--  1 root root        162 Feb 27 06:13 ssh_host_ecdsa_key.pub
-rw-r-----  1 root ssh_keys    387 Feb 27 06:13 ssh_host_ed25519_key
-rw-r--r--  1 root root         82 Feb 27 06:13 ssh_host_ed25519_key.pub
-rw-r-----  1 root ssh_keys   1679 Feb 27 06:13 ssh_host_rsa_key
-rw-r--r--  1 root root        382 Feb 27 06:13 ssh_host_rsa_key.pub
-rw-------  1 root root       4496 Feb 27 06:13 sshd_config
-rw-------  1 root root       4492 Feb 27 06:13 sshd_config-cisco

[guestshell@chefn9k ssh]$ sudo cp sshd_config sshd-mgmt_config

# 以下の行を追加します。
  
Port 2222
ListenAddress 172.22.90.104

```

- 次のコマンドを入力します。
    - sudo systemctl daemon-reload
    - sudo systemctl start sshd-mgmt.service
    - sudo systemctl status sshd-mgmt.service -l
    
```
[guestshell@chefn9k ssh]$ sudo systemctl daemon-reload
[guestshell@chefn9k ssh]$ sudo systemctl start sshd-mgmt.service
[guestshell@chefn9k ssh]$ sudo systemctl status sshd-mgmt.service -l
sshd-mgmt.service - OpenSSH server daemon
   Loaded: loaded (/usr/lib/systemd/system/sshd-mgmt.service; disabled)
   Active: active (running) since Fri 2017-03-10 02:17:28 UTC; 2s ago
  Process: 1447 ExecStartPre=/usr/sbin/sshd-keygen (code=exited, status=0/SUCCESS)
 Main PID: 1449 (sshd)
   CGroup: /system.slice/sshd-mgmt.service
           └─1449 /usr/sbin/sshd -f /etc/ssh/sshd-mgmt_config -D

Mar 10 02:17:28 chefn9k sshd-keygen[1447]: Executing: /usr/sbin/sshd-keygen
Mar 10 02:17:28 chefn9k ip[1449]: Executing: /sbin/ip netns exec management /usr/sbin/sshd -f /etc/ssh/sshd-mgmt_config -
Mar 10 02:17:28 chefn9k sshd[1449]: Server listening on 172.22.90.104 port 2222.

# Port 2222 が見えました。

[guestshell@chefn9k ssh]$ ss -tnldp | grep 2222
tcp    LISTEN     0      128        172.22.90.104:2222                  *:*

```

- Chef Workstation と鍵交換が必要

```
# Chef Workstation から Chef Node に SSH

user01@ino-ubuntu:~/.ssh$  ssh chefuser01@chefn9k -p 2222
The authenticity of host '[chefn9k]:2222 ([172.22.90.104]:2222)' can't be established.
ECDSA key fingerprint is b3:41:cd:e7:7a:5b:10:3a:8d:73:b5:18:4f:a4:41:bd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[chefn9k]:2222' (ECDSA) to the list of known hosts.
chefuser01@chefn9k's password:
Last login: Tue Mar 14 07:49:00 2017 from ino-ubuntu.iselab.local

# key を作成

[chefuser01@chefn9k ~]$ pwd
/home/chefuser01
[chefuser01@chefn9k ~]$ ls -la
total 14
drwx------ 3 chefuser01 chefuser01 1024 Mar 10 03:28 .
drwxr-xr-x 5 root       root       1024 Mar  3 06:26 ..
-rw------- 1 chefuser01 chefuser01  690 Mar 10 04:02 .bash_history
-rw-r--r-- 1 chefuser01 chefuser01   18 Feb 27 06:13 .bash_logout
-rw-r--r-- 1 chefuser01 chefuser01  193 Feb 27 06:13 .bash_profile
-rw-r--r-- 1 chefuser01 chefuser01  231 Feb 27 06:13 .bashrc
drwx------ 2 chefuser01 chefuser01 1024 Mar 10 03:55 .ssh
[chefuser01@chefn9k ~]$ cd .ssh

user01@ino-ubuntu:~/.ssh$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user01/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user01/.ssh/id_rsa.
Your public key has been saved in /home/user01/.ssh/id_rsa.pub.
The key fingerprint is:
xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx user01@ino-ubuntu
The key's randomart image is:

# 鍵ができた。

user01@ino-ubuntu:~/.ssh$ ls -la
total 20
drwx------ 2 user01 user01 4096 Mar  2 05:31 .
drwxr-xr-x 7 user01 user01 4096 Mar 16 15:26 ..
-rw------- 1 user01 user01 1679 Mar 16 17:32 id_rsa
-rw-r--r-- 1 user01 user01  399 Mar 16 17:32 id_rsa.pub
-rw-r--r-- 1 user01 user01  496 Mar 14 04:54 known_hosts

# 公開鍵を authorized_keys という名前でコピー。

user01@ino-ubuntu:~/.ssh$ cp id_rsa.pub authorized_keys
user01@ino-ubuntu:~/.ssh$ ls -la
total 24
drwx------ 2 user01 user01 4096 Mar 16 17:34 .
drwxr-xr-x 7 user01 user01 4096 Mar 16 15:26 ..
-rw-r--r-- 1 user01 user01  399 Mar 16 17:34 authorized_keys
-rw------- 1 user01 user01 1679 Mar 16 17:32 id_rsa
-rw-r--r-- 1 user01 user01  399 Mar 16 17:32 id_rsa.pub
-rw-r--r-- 1 user01 user01  496 Mar 14 04:54 known_hosts

# 権限を変更

user01@ino-ubuntu:~/.ssh$  sudo chmod 600 authorized_keys

# Chef Workstation に 鍵を送付
[chefuser01@chefn9k .ssh]$ scp id_rsa user01@172.22.150.100:/home/user01/.ssh/
user01@172.22.150.100's password:
id_rsa                                               100% 1675     1.6KB/s   00:00
[chefuser01@chefn9k .ssh]$

＃ SCP

[chefuser01@chefn9k .ssh]$ scp authorized_keys user01@172.22.150.100:/home/user01/.ssh/
user01@172.22.150.100's password:
authorized_keys                                      100%  400     0.4KB/s   00:00
[chefuser01@chefn9k .ssh]$
[chefuser01@chefn9k .ssh]$
[chefuser01@chefn9k .ssh]$
[chefuser01@chefn9k .ssh]$ scp id_rsa user01@172.22.150.100:/home/user01/.ssh/         id_rsa                                               100% 1675     1.6KB/s   00:00
[chefuser01@chefn9k .ssh]$ scp id_rsa.pub user01@172.22.150.100:/home/user01/.ssh/
id_rsa.pub                                           100%  400     0.4KB/s   00:00


```


### 余談：N9KのGuest ShellからCiscoコマンドを実行
- コマンドの先頭に、"sudo ip netns exec management" または ショートカットコマンド "chvrf management" をつければOKです。
    - N9KからChef ServerにPingを実行。

```
[guestshell@chefn9k ssh]$ sudo ip netns exec management ping 172.22.150.100
PING 172.22.150.100 (172.22.150.100) 56(84) bytes of data.
64 bytes from 172.22.150.100: icmp_seq=1 ttl=64 time=0.338 ms
64 bytes from 172.22.150.100: icmp_seq=2 ttl=64 time=0.310 ms
64 bytes from 172.22.150.100: icmp_seq=3 ttl=64 time=0.368 ms
64 bytes from 172.22.150.100: icmp_seq=4 ttl=64 time=0.330 ms

[guestshell@chefn9k ssh]$ chvrf management ping 172.22.150.100
PING 172.22.150.100 (172.22.150.100) 56(84) bytes of data.
64 bytes from 172.22.150.100: icmp_seq=1 ttl=64 time=0.404 ms
64 bytes from 172.22.150.100: icmp_seq=2 ttl=64 time=0.380 ms
64 bytes from 172.22.150.100: icmp_seq=3 ttl=64 time=0.406 ms
```
- Chef WorkstationからChefuser01でN9KにSSHできるようになりました。

```
inohana@ino-ubuntu:~$ ssh -p 2222 chefuser01@172.22.90.104
The authenticity of host '[172.22.90.104]:2222 ([172.22.90.104]:2222)' can't be established.
ECDSA key fingerprint is b3:41:cd:e7:7a:5b:10:3a:8d:73:b5:18:4f:a4:41:bd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[172.22.90.104]:2222' (ECDSA) to the list of known hosts.
chefuser01@172.22.90.104's password:
[chefuser01@chefn9k ~]$
```

- Chef WorkstationのHostsを編集します。
    - chefn9kを追加。

```
inohana@ino-ubuntu:~$ sudo vi /etc/hosts
inohana@ino-ubuntu:~$ cat  /etc/hosts
127.0.0.1       localhost
172.29.150.100  ino-ubuntu.iselab.local ino-ubuntu
172.22.90.104   chefn9k

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
```

## 再度Bootstrap

```
inohana@ino-ubuntu:~/chef-repo$ ls -la
total 28
drwxrwxr-x 5 inohana inohana 4096 Mar 10 04:44 .
drwxr-xr-x 8 root    root    4096 Mar 10 05:09 ..
drwxr-xr-x 3 inohana root    4096 Mar  2 05:01 .chef
drwxrwxr-x 9 inohana inohana 4096 Mar 10 04:57 cookbooks
-rw-rw-r-- 1 inohana inohana  486 Mar 10 04:43 .kitchen.yml
-rw-rw-r-- 1 inohana inohana   57 Mar 10 04:43 README.md
drwxrwxr-x 3 inohana inohana 4096 Mar 10 04:43 test

# knife.rb 定義は/chef-repo/.chef の中にあります。

inohana@ino-ubuntu:~/chef-repo$ cd .chef
inohana@ino-ubuntu:~/chef-repo/.chef$ ls
inohana.pem  ise-validator.pem  knife.rb  trusted_certs
inohana@ino-ubuntu:~/chef-repo/.chef$ ls -la
total 24
drwxr-xr-x 3 inohana root    4096 Mar  2 05:01 .
drwxrwxr-x 5 inohana inohana 4096 Mar 10 04:44 ..
-rw------- 1 inohana root    1678 Mar  2 04:53 inohana.pem
-rw------- 1 inohana root    1678 Mar  2 04:53 ise-validator.pem
-rw-r--r-- 1 inohana root     667 Mar  2 05:01 knife.rb
drwxr-xr-x 2 inohana root    4096 Mar  2 05:01 trusted_certs
```

- knifeコマンド成功しました。
    - N9Kのguestshellには同IPアドレス、ポート2222でアクセス可能になっています。

```
inohana@ino-ubuntu:~/repo$ knife bootstrap chefn9k -x chefuser01 --sudo -p 2222
Doing old-style registration with the validation key at /home/inohana/repo/.chef/ise-validator.pem...
Delete your validation key in order to use your user credentials instead

Connecting to chefn9k
chefuser01@chefn9k's password:
chefn9k -----> Existing Chef installation detected
chefn9k Starting the first Chef Client run...
chefn9k [2017-03-10T02:38:10+00:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support.
chefn9k Starting Chef Client, version 12.4.1
chefn9k Creating a new client identity for chefn9k using the validator key.
chefn9k resolving cookbooks for run list: []
chefn9k Synchronizing Cookbooks:
chefn9k Compiling Cookbooks...
chefn9k [2017-03-10T02:38:15+00:00] WARN: Node chefn9k has an empty run list.
chefn9k Converging 0 resources
chefn9k
chefn9k Running handlers:
chefn9k Running handlers complete
chefn9k Chef Client finished, 0/0 resources updated in 4.148158817 seconds
```
- 事前にHostsファイルを編集し、Chef Nodeの名前解決できるようにしておきます。

```
inohana@ino-ubuntu:~$ cat  /etc/hosts
127.0.0.1       localhost
172.29.150.100  ino-ubuntu.iselab.local ino-ubuntu
172.22.90.104   chefn9k

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

```