OpenStack仮想ネットワークの基本動作を試してみた

OpenStack仮想ネットワークの基本動作を体験してみたいと思います。


## ■ やってみたこと
OpenStaxk仮想ネットワークで、試してみたことは、

- サーバ機器２台を、HUB接続する
- サーバ機器に、IPアドレスを付与する
- サーバ機器の間で、ping通信してみる

という、非常にシンプルな内容です。

![Openstack.001.jpeg](https://qiita-image-store.s3.amazonaws.com/0/103595/2b8ef004-205d-62f2-0770-fbd295e4d943.jpeg)


## ■ OpenStack実験環境
["Nested KVM環境でのNewton版OpenStack構築メモ"](http://qiita.com/ttsubo/items/8eeb047c2a87d605a6ef)で作成したOpenStack環境を使用します。

- nova仮想マシン一覧

```
[root@newton ~(keystone_demo)]# nova list
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+
| ID                                   | Name    | Status | Task State | Power State | Networks                         |
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+
| ab70f784-385d-44a8-b010-1322e7be94a7 | server1 | ACTIVE | -          | Running     | admin_net=10.0.0.4, 10.79.5.191  |
| 06521269-bddb-4def-8369-3f92fd2f4622 | server2 | ACTIVE | -          | Running     | admin_net=10.0.0.11, 10.79.5.195 |
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+
```
- neutronフローティングIP一覧

```
[root@newton ~(keystone_demo)]# neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| affd1370-760d-4148-9d48-5afb9f97eaf5 | 10.0.0.4         | 10.79.5.191         | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c |
| cb83e8bc-47c2-438a-8c74-2f1654ddc09c | 10.0.0.11        | 10.79.5.195         | 75f7e4ac-600b-4a45-9948-d45b3ec7169b |
+--------------------------------------+------------------+---------------------+--------------------------------------+
```
- OpenStack仮想ネットワーク構成イメージ

![topology1.png](https://qiita-image-store.s3.amazonaws.com/0/103595/247e9f0d-ecee-90b6-b5e8-ce2c9d0bef0c.png)




## ■ 実験１「サーバ機器２台を、HUB接続してみる」
userネットワークに仮想マシンを配備して、ping疎通確認を行ってみます。

### (1) userネットワーク用ポートを作成する
- 仮想マシン"server1"用ポートの作成

```
[root@newton ~(keystone_demo)]# neutron port-create --name user_port1 user_net
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2017-02-03T00:29:03Z                                                               |
| description           |                                                                                    |
| device_id             |                                                                                    |
| device_owner          |                                                                                    |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "9f0e59df-d6c8-4c2d-9de5-8979798e5350", "ip_address": "172.16.0.11"} |
| id                    | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89                                               |
| mac_address           | fa:16:3e:b3:33:ae                                                                  |
| name                  | user_port1                                                                         |
| network_id            | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e                                               |
| project_id            | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| revision_number       | 4                                                                                  |
| security_groups       | bff315ad-0f08-45f8-8b14-5d19a9f95aa6                                               |
| status                | DOWN                                                                               |
| tenant_id             | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| updated_at            | 2017-02-03T00:29:04Z                                                               |
+-----------------------+------------------------------------------------------------------------------------+
```
- 仮想マシン"server2"用ポートの作成

```
[root@newton ~(keystone_demo)]# neutron port-create --name user_port2 user_net
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2017-02-03T00:29:12Z                                                               |
| description           |                                                                                    |
| device_id             |                                                                                    |
| device_owner          |                                                                                    |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "9f0e59df-d6c8-4c2d-9de5-8979798e5350", "ip_address": "172.16.0.12"} |
| id                    | e1776e0d-9f64-4684-b050-c8749f8c9a88                                               |
| mac_address           | fa:16:3e:38:dc:ef                                                                  |
| name                  | user_port2                                                                         |
| network_id            | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e                                               |
| project_id            | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| revision_number       | 4                                                                                  |
| security_groups       | bff315ad-0f08-45f8-8b14-5d19a9f95aa6                                               |
| status                | DOWN                                                                               |
| tenant_id             | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| updated_at            | 2017-02-03T00:29:13Z                                                               |
+-----------------------+------------------------------------------------------------------------------------+
```
- 登録済みのport一覧の確認

```
[root@newton ~(keystone_demo)]# neutron port-list 
+--------------------------------------+------------+-------------------+---------------------------------------------------+
| id                                   | name       | mac_address       | fixed_ips                                         |
+--------------------------------------+------------+-------------------+---------------------------------------------------+
| 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c |            | fa:16:3e:49:48:8e | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.4"}          |
| 3c173d98-c714-492d-8c6f-ef330c0e5d12 |            | fa:16:3e:6a:16:02 | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.2"}          |
| 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | user_port1 | fa:16:3e:b3:33:ae | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.11"}  |
| 5da3eb5e-d1ff-41ec-a583-0cefc19e7973 |            | fa:16:3e:4e:58:bc | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.1"}          |
| 680fcfb5-2e04-4ec5-a77c-3c6fd29d9ea9 |            | fa:16:3e:0f:0d:f1 | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.2"}   |
| 75f7e4ac-600b-4a45-9948-d45b3ec7169b |            | fa:16:3e:7c:92:60 | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.11"}         |
| e1776e0d-9f64-4684-b050-c8749f8c9a88 | user_port2 | fa:16:3e:38:dc:ef | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.12"}  |
+--------------------------------------+------------+-------------------+---------------------------------------------------+
```

### (2) 仮想マシンに、userネットワーク用ポートを取り付ける

- 仮想マシン"server1"側でのポート取り付け

```
[root@newton ~(keystone_demo)]# nova interface-attach --port-id 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 server1
```
- 仮想マシン"server2"側でのポート取り付け

```
[root@newton ~(keystone_demo)]# nova interface-attach --port-id e1776e0d-9f64-4684-b050-c8749f8c9a88 server2
```
- 仮想マシン"server1"側での登録済みのインタフェース一覧の確認

```
[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
```
- 仮想マシン"server2"側での登録済みのインタフェース一覧の確認

```
[root@newton ~(keystone_demo)]# nova interface-list server2
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 75f7e4ac-600b-4a45-9948-d45b3ec7169b | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.11    | fa:16:3e:7c:92:60 |
| ACTIVE     | e1776e0d-9f64-4684-b050-c8749f8c9a88 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.12  | fa:16:3e:38:dc:ef |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
```
### (3) 仮想マシン"server1"上で、取り付け済みインタフェースを有効にする
- 現在のインタフェース状態の確認（ **"ens6"** が新たに追加になった）

```
ubuntu@server1:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff
```
- PCI接続デバイスの確認

```
root@server1:~# lspci |grep Ether
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
```

- 暫定として、IPアドレス値を付与

```
ubuntu@server1:~$ sudo -i
root@server1:~# ip addr add 172.16.0.11/24 dev ens6
root@server1:~# ip link set ens6 up
```
- 再度、インタフェース状態の確認

```
root@server1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.11/24 scope global ens6
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feb3:33ae/64 scope link 
       valid_lft forever preferred_lft forever
```

### (4) 仮想マシン"server2"上で、取り付け済みインタフェースを有効にする
- 現在のインタフェース状態の確認（"eth1"が新たに追加になった）

```
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:7c:92:60 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::f816:3eff:fe7c:9260/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether fa:16:3e:38:dc:ef brd ff:ff:ff:ff:ff:ff
```
- 暫定として、IPアドレス値を付与

```
$ sudo -i
# ip addr add 172.16.0.12/24 dev eth1
# ip link set eth1 up
```
- 再度、インタフェース状態の確認

```
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:7c:92:60 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::f816:3eff:fe7c:9260/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:38:dc:ef brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.12/24 scope global eth1
    inet6 fe80::f816:3eff:fe38:dcef/64 scope link 
       valid_lft forever preferred_lft forever
```

### (4) 仮想マシン間での疎通性を確認してみる
- 仮想マシン"server1"上で、対向の仮想マシン宛てに、pingを打ってみる

```
root@server1:~# ping 172.16.0.12
PING 172.16.0.12 (172.16.0.12) 56(84) bytes of data.
64 bytes from 172.16.0.12: icmp_seq=1 ttl=64 time=2.28 ms
64 bytes from 172.16.0.12: icmp_seq=2 ttl=64 time=2.19 ms
64 bytes from 172.16.0.12: icmp_seq=3 ttl=64 time=0.708 ms
^C
--- 172.16.0.12 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.708/1.730/2.288/0.724 ms
```

- 最終的に、作成したトポロジ構成
![topology.png](https://qiita-image-store.s3.amazonaws.com/0/103595/46417795-f800-db69-de5c-22cf1f40e63c.png)

ここまでは、順調に、OpenStack仮想ネットワークが作成できました。

## ■ 実験２「HUB接続したサーバ機器のLANケーブルを抜き差ししてみる」
続いて、仮想マシン"server1"側のインタフェースの取り外し/取り付けを試してみます。

![Openstack.002.jpeg](https://qiita-image-store.s3.amazonaws.com/0/103595/b7851886-d8ae-f8c6-b880-689f0fddb15f.jpeg)


### (1) neuton上でのポート取り外し/取り付け
- 仮想マシン"server1"側での登録済みのインタフェース一覧の確認

```
[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
```
- 仮想マシン"server1"側のポート取り外し

```
[root@newton ~(keystone_demo)]# nova interface-detach server1 4fbca26b-95bd-4d74-8f56-20b1acfa8e89
```
- 仮想マシン"server1"側での登録済みのインタフェース一覧の確認

```
[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
```
- 再度、仮想マシン"server1"側でのポート取り付け

```
[root@newton ~(keystone_demo)]# nova interface-attach --port-id 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 server1
```
- 仮想マシン"server1"側での登録済みのインタフェース一覧の確認

```
[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
```
### (2) 仮想マシン"server1"上での確認
- 現在のインタフェース状態の確認（ **"ens7"** にNIC名が変わってしまった）

```
root@server1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
4: ens7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff
```
- PCI接続デバイスの確認

```
root@server1:~# lspci |grep Ether
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:07.0 Ethernet controller: Red Hat, Inc Virtio network device
```

[実験結果]
neuton上でのポート取り外し/取り付けを行うと、暫定で設定していたIPアドレス値が無効になってしまいました。さらに、Etherデバイスカードの接続PCIポートが変更されてしまいました。ちなみに、MACアドレスは、特に、変化しませんでした。

![OpenStack.003.jpeg](https://qiita-image-store.s3.amazonaws.com/0/103595/0f1fdda1-86e9-c189-57e4-24337545e254.jpeg)

### ■ 終わりに
今回は、実環境では、発生しえない動作を、経験することとなりました。
OpenStack仮想ネットワーク構成は、二つの異なるOpenStackコミュニティを基盤として構成されるためですが、いまいち、有効な解決方法が見つかっていない現状です。

OpenStack仮想ネットワークを活用して、クラウドオーケストレーションの自動化が一般化しつつありますし、NFVのような柔軟なトラフィック制御を実現する際には、阻害要因になりそうな気がします。みなさん、どうやって、解決しているか気になるところです。