OpenVNetを動かしてみた　最小構成

#OpenVNetとは
OpenVNetはネットワークの仮想化を実現するためのオープンソースのソフトウェアです。

#準備

環境 CentOS6.7
とりあえずインストールマニュアルをなぞっていきます。
http://openvnet.org/installation/
最終的に下の図のようなものができます。
![OVN01.jpg](https://qiita-image-store.s3.amazonaws.com/0/86684/eb91f731-7b18-bf99-7c13-052821cb69fc.jpeg)


上から

```
curl -o /etc/yum.repos.d/openvnet.repo -R https://raw.githubusercontent.com/axsh/openvnet/master/deployment/yum_repositories/stable/openvnet.repo

curl -o /etc/yum.repos.d/openvnet-third-party.repo -R https://raw.githubusercontent.com/axsh/openvnet/master/deployment/yum_repositories/stable/openvnet-third-party.repo

yum install -y epel-release
```
までは順調にいくと思います。
すると

```
yum install -y openvnet
```
で

```
Error: Package: kmod-openvswitch-2.3.1-1.el6.x86_64 (openvnet-third-party)
           Requires: kernel(openvswitch_handle_frame_hook) = 0xbe03eb4d
```
とでて失敗します。
ここはCentOS6.6ではマニュアルのままで成功する環境もありました。
そこで、こちら
https://github.com/mao172/ovs-installer
のovs-installerをお借りしてみます。

```
curl -L https://raw.githubusercontent.com/mao172/ovs-installer/master/install.sh | bash -s -- -v 2.3.1
```
そこでもう一回

```
yum install -y openvnet
```
をするとComplete!します。
素晴らしいですね。
続いて、

```
yum install -y mysql-server redis
```

を行った後、マニュアルには
```
/etc/openvnet/vnmgr.conf
/etc/openvnet/vna.conf
/etc/openvnet/webapi.conf
```

の設定に関する記述がありますが
今回はこれは変更しません。
次にbr0を作ります。

```
vi /etc/sysconfig/network-scripts/ifcfg-br0
で中を
DEVICE=br0
DEVICETYPE=ovs
TYPE=OVSBridge
ONBOOT=yes
BOOTPROTO=static
HOTPLUG=no
OVS_EXTRA="
 set bridge     ${DEVICE} protocols=OpenFlow10,OpenFlow12,OpenFlow13 --
 set bridge     ${DEVICE} other_config:disable-in-band=true --
 set bridge     ${DEVICE} other-config:datapath-id=0000020100000001 --
 set bridge     ${DEVICE} other-config:hwaddr=02:01:00:00:00:01 --
 set-fail-mode  ${DEVICE} standalone --
 set-controller ${DEVICE} tcp:127.0.0.1:6633
"
```
とします。
datapath-idはマニュアルと変えてあります。
また、hwaddrはそのままで問題ないと思いますが
同じMACアドレスを持っているものがないようにします。

次にいろいろとスタートさせます。

```
service openvswitch start
ifup br0
service redis start

```

/etc/openvnet/common.confに関しても今回は設定を変更する必要はありません。
続いてデータベースを設定していきます。

```
service mysqld start
chkconfig mysqld on
PATH=/opt/axsh/openvnet/ruby/bin:${PATH}
cd /opt/axsh/openvnet/vnet
bundle exec rake db:create
bundle exec rake db:init
```

ここまでで準備終了です。

#OpenVNet

OpenVNetのvnmgrとwebapiをスタートさせます。

```
initctl start vnet-vnmgr
initctl start vnet-webapi
```
datapathを追加します。
dpidはbr0の設定でdatapath-idに設定したものと同じで
今回はマニュアルと違うものを設定しています。

```
vnctl datapaths add --uuid dp-test1 --display-name test1 --dpid 0x0000020100000001 --node-id vna
```
次はOpenVNetで使用するネットワークの設定です。

```
vnctl networks add --uuid nw-test1 --display-name testnet1 --ipv4-network 10.100.0.0 --ipv4-prefix 24 --network-mode virtual
```

次はこの後入れるLXCをOpenVNetで接続するための
interfaceの設定をします。

```
vnctl interfaces add --uuid if-inst1 --mode vif --owner-datapath-uuid dp-test1 --mac-address 10:54:ff:00:00:01 --network-uuid nw-test1 --ipv4-address 10.100.0.10 --port-name inst1

vnctl interfaces add --uuid if-inst2 --mode vif --owner-datapath-uuid dp-test1 --mac-address 10:54:ff:00:00:02 --network-uuid nw-test1 --ipv4-address 10.100.0.11 --port-name inst2
```
そしてOpenVNetのvnaを起動します。
もし、vnmgrとwebapiが起動していなければこちらも起動します。

```
initctl start vnet-vna
```

#LXC
マニュアル通りLXCでOpenVNetを使ってみます。
まずは必要なものの準備をします。

```
yum -y install lxc lxc-templates

mkdir /cgroup
echo "cgroup /cgroup cgroup defaults 0 0" >> /etc/fstab
mount /cgroup

yum install -y rsync (既に入っていることもあります)

lxc-create -t centos -n inst1
lxc-create -t centos -n inst2
```
inst1、inst2の設定ファイルを編集します。
既にいろいろと書いてありますがすべて消して以下の設定を書きます。

```
vi /var/lib/lxc/inst1/config


lxc.network.type = veth
lxc.network.flags = up
lxc.network.veth.pair = inst1
lxc.network.ipv4 = 10.100.0.10
lxc.network.hwaddr = 10:54:FF:00:00:01
lxc.rootfs = /var/lib/lxc/inst1/rootfs
lxc.include = /usr/share/lxc/config/centos.common.conf
lxc.arch = x86_64
lxc.utsname = inst1
lxc.autodev = 0
```
同じようにinst2も

```
vi /var/lib/lxc/inst2/config

lxc.network.type = veth
lxc.network.flags = up
lxc.network.veth.pair = inst2
lxc.network.ipv4 = 10.100.0.11
lxc.network.hwaddr = 10:54:FF:00:00:02
lxc.rootfs = /var/lib/lxc/inst2/rootfs
lxc.include = /usr/share/lxc/config/centos.common.conf
lxc.arch = x86_64
lxc.utsname = inst2
lxc.autodev = 0
```
それぞれをスタートさせdatapathにさします。

```
lxc-start -d -n inst1
lxc-start -d -n inst2

ovs-vsctl add-port br0 inst1
ovs-vsctl add-port br0 inst2
```
それぞれのpasswordを確認します。

```
cat /var/lib/lxc/inst1/tmp_root_pass
cat /var/lib/lxc/inst2/tmp_root_pass
```

inst1に入ります。

```
lxc-console -n inst1
```

これで10.100.0.11にpingが通れば成功です。

#Flowの確認
最後にフローを見てみます。

```
ovs-ofctl -O OpenFlow13 dump-flows br0

OFPST_FLOW reply (OF1.3) (xid=0x2):
 cookie=0x900000000000000, duration=325.525s, table=0, n_packets=1, n_bytes=90, priority=1,tun_id=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=0, n_packets=0, n_bytes=0, priority=0 actions=write_metadata:0x80000000000/0xc0000000000,goto_table:3
 cookie=0x900000000000000, duration=325.525s, table=0, n_packets=0, n_bytes=0, priority=2,in_port=CONTROLLER actions=write_metadata:0x4040000000000/0x40c0000000000,goto_table:7
 cookie=0x500000000000001, duration=130.748s, table=0, n_packets=17, n_bytes=3158, priority=2,in_port=1 actions=write_metadata:0x700040000000001/0xff000c007fffffff,goto_table:15
 cookie=0x5000000fffffffe, duration=325.467s, table=0, n_packets=0, n_bytes=0, priority=2,in_port=LOCAL actions=write_metadata:0x40000000000/0xc0000000000,goto_table:6
 cookie=0x500000000000002, duration=130.024s, table=0, n_packets=17, n_bytes=3170, priority=2,in_port=2 actions=write_metadata:0x700040000000002/0xff000c007fffffff,goto_table:15
 cookie=0x900000000000000, duration=325.526s, table=3, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=4, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x400000000000001, duration=130.240s, table=4, n_packets=0, n_bytes=0, priority=30,tun_id=0x80000001 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:20
 cookie=0x900000000000000, duration=325.526s, table=6, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=7, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=8, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=9, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=10, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=11, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xc00001300000001, duration=130.314s, table=11, n_packets=0, n_bytes=0, priority=50,dl_src=10:54:ff:00:00:01 actions=drop
 cookie=0xc00001300000002, duration=129.963s, table=11, n_packets=0, n_bytes=0, priority=50,dl_src=10:54:ff:00:00:02 actions=drop
 cookie=0xc00001200000002, duration=129.963s, table=11, n_packets=0, n_bytes=0, priority=30,arp,dl_dst=10:54:ff:00:00:02,arp_tpa=10.100.0.11 actions=write_metadata:0x8000000180000000/0xffffffff80000000,goto_table:12
 cookie=0xc00001200000002, duration=129.963s, table=11, n_packets=0, n_bytes=0, priority=30,ip,dl_dst=10:54:ff:00:00:02,nw_dst=10.100.0.11 actions=write_metadata:0x8000000180000000/0xffffffff80000000,goto_table:12
 cookie=0xc00001200000001, duration=130.166s, table=11, n_packets=0, n_bytes=0, priority=30,ip,dl_dst=10:54:ff:00:00:01,nw_dst=10.100.0.10 actions=write_metadata:0x8000000180000000/0xffffffff80000000,goto_table:12
 cookie=0xc00001200000001, duration=130.166s, table=11, n_packets=0, n_bytes=0, priority=30,arp,dl_dst=10:54:ff:00:00:01,arp_tpa=10.100.0.10 actions=write_metadata:0x8000000180000000/0xffffffff80000000,goto_table:12
 cookie=0xc00001300000001, duration=130.314s, table=11, n_packets=0, n_bytes=0, priority=50,arp,arp_sha=10:54:ff:00:00:01 actions=drop
 cookie=0xc00001300000002, duration=129.963s, table=11, n_packets=0, n_bytes=0, priority=50,arp,arp_sha=10:54:ff:00:00:02 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=12, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xc00001200000002, duration=129.963s, table=12, n_packets=0, n_bytes=0, priority=90,arp,metadata=0x8000000100000000/0xffffffff00000000,arp_spa=10.100.0.11 actions=drop
 cookie=0xc00001200000001, duration=130.166s, table=12, n_packets=0, n_bytes=0, priority=90,ip,metadata=0x8000000100000000/0xffffffff00000000,nw_src=10.100.0.10 actions=drop
 cookie=0xc00001200000001, duration=130.166s, table=12, n_packets=0, n_bytes=0, priority=90,arp,metadata=0x8000000100000000/0xffffffff00000000,arp_spa=10.100.0.10 actions=drop
 cookie=0xc00001200000002, duration=129.963s, table=12, n_packets=0, n_bytes=0, priority=90,ip,metadata=0x8000000100000000/0xffffffff00000000,nw_src=10.100.0.11 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=13, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=15, n_packets=8, n_bytes=588, priority=0 actions=drop
 cookie=0xc00001200000001, duration=130.166s, table=15, n_packets=2, n_bytes=84, priority=30,arp,metadata=0x700000000000001/0xff0000007fffffff,dl_src=10:54:ff:00:00:01,arp_spa=10.100.0.10,arp_sha=10:54:ff:00:00:01 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0xc00001200000002, duration=129.963s, table=15, n_packets=2, n_bytes=84, priority=30,arp,metadata=0x700000000000002/0xff0000007fffffff,dl_src=10:54:ff:00:00:02,arp_spa=10.100.0.11,arp_sha=10:54:ff:00:00:02 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0xc00001200000001, duration=130.166s, table=15, n_packets=4, n_bytes=392, priority=30,ip,metadata=0x700000000000001/0xff0000007fffffff,dl_src=10:54:ff:00:00:01,nw_src=10.100.0.10 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0xc00001200000002, duration=129.963s, table=15, n_packets=4, n_bytes=392, priority=30,ip,metadata=0x700000000000002/0xff0000007fffffff,dl_src=10:54:ff:00:00:02,nw_src=10.100.0.11 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0xc00001200000002, duration=129.963s, table=15, n_packets=7, n_bytes=2394, priority=30,ip,metadata=0x700000000000002/0xff0000007fffffff,dl_src=10:54:ff:00:00:02,nw_src=0.0.0.0 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0xc00001200000001, duration=130.166s, table=15, n_packets=7, n_bytes=2394, priority=30,ip,metadata=0x700000000000001/0xff0000007fffffff,dl_src=10:54:ff:00:00:01,nw_src=0.0.0.0 actions=write_metadata:0x300000000000001/0xff0000007fffffff,goto_table:18
 cookie=0x900000000000000, duration=325.526s, table=16, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=17, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=18, n_packets=26, n_bytes=5740, priority=0 actions=goto_table:20
 cookie=0x400000000000001, duration=130.240s, table=20, n_packets=0, n_bytes=0, priority=40,arp,metadata=0x300080000000001/0xff000c007fffffff actions=goto_table:21
 cookie=0x900000000000000, duration=325.526s, table=20, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x400000000000001, duration=130.240s, table=20, n_packets=26, n_bytes=5740, priority=30,metadata=0x300000000000001/0xff0000007fffffff actions=goto_table:30
 cookie=0x400000000000001, duration=130.226s, table=21, n_packets=0, n_bytes=0, priority=1,arp,metadata=0x300080000000001/0xff000c007fffffff actions=learn(table=43,idle_timeout=36000,priority=35,cookie=0x400000000000001,metadata=0x300040000000001,NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),goto_table:42
 cookie=0x900000000000000, duration=325.525s, table=21, n_packets=0, n_bytes=0, priority=0 actions=goto_table:42
 cookie=0x400000000000001, duration=130.233s, table=21, n_packets=0, n_bytes=0, priority=3,arp,tun_id=0,metadata=0x300080000000001/0xff000c007fffffff actions=learn(table=43,idle_timeout=36000,priority=35,cookie=0x400000000000001,metadata=0x300040000000001,NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],output:NXM_OF_IN_PORT[]),goto_table:42
 cookie=0x900000000000000, duration=325.525s, table=21, n_packets=0, n_bytes=0, priority=2,arp,tun_id=0 actions=goto_table:42
 cookie=0x900000000000000, duration=325.525s, table=30, n_packets=26, n_bytes=5740, priority=0 actions=goto_table:42
 cookie=0x900000000000000, duration=325.526s, table=31, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=32, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=33, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=34, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=35, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=36, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=37, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=40, n_packets=0, n_bytes=0, priority=0 actions=goto_table:41
 cookie=0xc00001200000001, duration=130.166s, table=40, n_packets=0, n_bytes=0, priority=40,ip,metadata=0x300000000000001/0xff0000007fffffff,nw_dst=10.100.0.10 actions=set_field:10:54:ff:00:00:01->eth_dst,goto_table:42
 cookie=0xc00001200000002, duration=129.963s, table=40, n_packets=0, n_bytes=0, priority=40,ip,metadata=0x300000000000001/0xff0000007fffffff,nw_dst=10.100.0.11 actions=set_field:10:54:ff:00:00:02->eth_dst,goto_table:42
 cookie=0x900000000000000, duration=325.526s, table=41, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=42, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x400000000000001, duration=130.240s, table=42, n_packets=26, n_bytes=5740, priority=30,metadata=0x300000000000001/0xff0000007fffffff actions=goto_table:43
 cookie=0xc00001200000001, duration=130.166s, table=43, n_packets=6, n_bytes=476, priority=60,metadata=0x300000000000001/0xff0000007fffffff,dl_dst=10:54:ff:00:00:01 actions=write_metadata:0x700000000000001/0xff0000007fffffff,goto_table:45
 cookie=0xc00001200000002, duration=129.963s, table=43, n_packets=5, n_bytes=434, priority=60,metadata=0x300000000000001/0xff0000007fffffff,dl_dst=10:54:ff:00:00:02 actions=write_metadata:0x700000000000002/0xff0000007fffffff,goto_table:45
 cookie=0x900000000000000, duration=325.526s, table=43, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=43, n_packets=15, n_bytes=4830, priority=30,dl_dst=ff:ff:ff:ff:ff:ff actions=goto_table:50
 cookie=0x900000000000000, duration=325.525s, table=45, n_packets=0, n_bytes=0, priority=0 actions=goto_table:46
 cookie=0xc00007100000002, duration=130.024s, table=45, n_packets=4, n_bytes=392, priority=90,metadata=0x700000000000002/0xff0000007fffffff actions=goto_table:90
 cookie=0xc00007100000001, duration=130.586s, table=45, n_packets=4, n_bytes=392, priority=90,metadata=0x700000000000001/0xff0000007fffffff actions=goto_table:90
 cookie=0xe00001100000000, duration=325.733s, table=45, n_packets=3, n_bytes=126, priority=90,arp actions=goto_table:90
 cookie=0x900000000000000, duration=325.526s, table=46, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=50, n_packets=15, n_bytes=4830, priority=0 actions=goto_table:51
 cookie=0x900000000000000, duration=325.526s, table=51, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x400000000000001, duration=129.963s, table=51, n_packets=15, n_bytes=4830, priority=1,metadata=0x300000000000001/0xff0000007fffffff actions=output:1,output:2,goto_table:52
 cookie=0x900000000000000, duration=325.525s, table=52, n_packets=15, n_bytes=4830, priority=0 actions=goto_table:53
 cookie=0x900000000000000, duration=325.525s, table=52, n_packets=0, n_bytes=0, priority=10,metadata=0x80000000000/0xc0000000000 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=53, n_packets=15, n_bytes=4830, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=70, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=71, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=72, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=73, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=74, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=80, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=80, n_packets=0, n_bytes=0, priority=2,dl_dst=ff:ff:ff:ff:ff:ff actions=drop
 cookie=0x900000000000000, duration=325.526s, table=81, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=82, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=84, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=85, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.525s, table=85, n_packets=0, n_bytes=0, priority=1,tun_id=0x10000001 actions=goto_table:86
 cookie=0x900000000000000, duration=325.525s, table=85, n_packets=0, n_bytes=0, priority=1,tun_id=0x80000000/0x80000000 actions=goto_table:86
 cookie=0x900000000000000, duration=325.526s, table=86, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=90, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x500000000000002, duration=130.024s, table=90, n_packets=5, n_bytes=434, priority=10,metadata=0x700000000000002/0xff0000007fffffff actions=output:2
 cookie=0x500000000000001, duration=130.748s, table=90, n_packets=6, n_bytes=476, priority=10,metadata=0x700000000000001/0xff0000007fffffff actions=output:1
 cookie=0x900000000000000, duration=325.526s, table=91, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x900000000000000, duration=325.526s, table=92, n_packets=0, n_bytes=0, priority=0 actions=drop
```

これが下のようだと失敗です。

```
OFPST_FLOW reply (OF1.3) (xid=0x2):
 cookie=0x0, duration=3351.584s, table=0, n_packets=46, n_bytes=7068, priority=0 actions=NORMAL
```

私の場合このようになった原因はwebapiを起動してからデータベースを作成したことでした。
この場合、

```
initctl restart vnet-webapi
```

でwebapiを再起動した後、
データベースの登録からやり直すと正しく動くようになりました。