FreeBSD12　qjailでジャンボフレームの設定とLR-LINK製のNIC「LREC6860BT」による作動

# はじめに
qjailでジャンボフレームの設定の方法が分からず、トライ・アンド・エラーでqjailの設定ファイルを手動で設定し可能になった。今回報告する方法は、なんの根拠もなく、また問題が潜んでいる可能性がある。本来のやり方があってドキュメントの見落としをしてるかもしれなく、わかる方教示いただければと思います。
次に、LREC6860BTは以下の報告で熱の問題がありましたが、解消でき今回のジャンボフレーム設定を行った。しかし、問題が発生し解決に至らなく、その内容を報告する。結果としてbridge接続で使用できない（単独での使用は可能）ので、windowsで使用していたintelのX540-T1と交換することにした。
* [「FreeBSD12にLR-LINK製のNIC「LREC6860BT」(10Gbase-T)をインストール」@Qiita](https://qiita.com/Chun3/items/810d64cf3911b479b66f)
* [「FreeBSD 12で自宅サーバの構築　基本システム編その３」@Qiita](https://qiita.com/Chun3/items/e5cc6a69542153649dc0)

# qjailでジャンボフレームの設定
「FreeBSD 12で自宅サーバの構築　基本システム編その３」で以下のように簡単に設定ファイルを作成できた（インタフェースを機器に合わせem0からre0に変更）。

```
# qjail create -4 192.168.1.41 jail1
Successfully created  jail1
# qjail config -w re0 -v none jail1
Successfully enabled vnet.interface for jail1
Successfully enabled vnet for jail1
```

この場合re0に"mtu 9014"のようにジャンボフレームを設定するとepair1aとブリッジ接続するときにエラーが発生する。epair1aに"mtu 9014"が設定できれば問題ないと思われる。
そこで考えたのが、予めブリッジ接続をしてからqjail startすることにする

## 手動でブリッジ接続設定
rc.confを設定する

```sh:/etc/rc.conf
cloned_interfaces="epair0 epair1 bridge0"
ifconfig_epair0a="up mtu 9014"
ifconfig_epair1a="up mtu 9014"
ifconfig_re0="up mtu 9014"
ifconfig_bridge0="addm re0 addm epair0a addm epair1a up mtu 9014"
ifconfig_bridge0_alias0="inet 192.168.1.31/24"
```

この設定で立ち上げると以下が確認できる

```
# ifconfig
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

## qjailの設定
qjailの設定がよく分からず、トライ・アンド・エラーで結果以下で落ち着いた。

```/usr/local/etc/qjail.config/jqil1
jail1 {
allow.sysvipc       =  "1";
host.hostname       =  "jail1.example.jp";
path                =  "/usr/jails/jail1";
mount.fstab         =  "/usr/local/etc/qjail.fstab/jail1";
exec.consolelog     =  "/var/log/qjail.jail1.console.log";
mount.devfs;
devfs_ruleset       =  "4";
vnet                =  "new";
vnet.interface      =  "epair0b";
exec.start   = "ifconfig epair0b 192.168.1.24";
exec.start  += "route add default 192.168.1.1";
exec.start  += "/bin/echo "epair0b" > /etc/epair";
exec.start  += "/bin/sh /etc/rc";
exec.stop    = "/bin/sh /etc/rc.shutdown";
}
```

```/usr/local/etc/qjail.global/jail1
name="jail1"
ip4="192.168.1.24"
ip6=""
path="/usr/jails/jail1"
interface="bridge0"
fstab="/usr/local/etc/qjail.fstab/jail1"
securelevel=""
cpuset=""
fib=""
vnet=""
vinterface=""
rsockets=""
ruleset=""
sysvipc=""
quotas=""
nullfs=""
tmpfs=""
zfs=""
poststartssh=""
deffile="/usr/local/etc/qjail.local/jail1"
image=""
imagetype=""
imageblockcount=""
imagedevice=""
```

```/usr/local/etc/qjail.local/jail1
name="jail1"
ip4="192.168.1.24"
ip6=""
path="/usr/jails/jail1"
interface="bridge0"
fstab="/usr/local/etc/qjail.fstab/jail1"
securelevel=""
cpuset=""
fib=""
vnet=""
vinterface=""
rsockets=""
ruleset=""
sysvipc=""
quotas=""
nullfs=""
tmpfs=""
zfs=""
poststartssh=""
deffile="/usr/local/etc/qjail.local/jail1"
image=""
imagetype=""
imageblockcount=""
imagedevice=""
```

```/usr/local/etc/qjail.vnetctl
count="0"
```

## 確認
以下のようにqjailを開始すると２つのjailは正常に作動する。
rc.confでqjail_enable="YES"と設定することでパワーオンでjailが正常に作動する
jail作動後のネットワークは以下になる。

```
# qjail start
Jail successfully started  jail1
Jail successfully started  jail2
# ifconfig
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

## 問題
まだ問題が少し残っているが、通常運用には差し支えがないのでこれで良しとしている。
問題は、qjailを停止してもepairxbが復活しないことがある。その場合は停止後にepairを作成し直さないと開始ができない。
（少し待つと復活する場合もある）

# LREC6860BTの使用について
ネットワークインターフェースをre0からtn400に変更するだけでできるはず

## ネットワーク設定と確認
qjailの設定ファイルからネットワークの実インタフェース名がないので、rc.confの変更のみで行ける。

```sh:/etc/rc.conf
cloned_interfaces="epair0 epair1 bridge0"
ifconfig_epair0a="up mtu 9014"
ifconfig_epair1a="up mtu 9014"
ifconfig_tn400="up mtu 9014"
# ifconfig_re0="up mtu 9014"
ifconfig_bridge0="addm tn400 addm epair0a addm epair1a up mtu 9014"
# ifconfig_bridge0="addm re0 addm epair0a addm epair1a up mtu 9014"
ifconfig_bridge0_alias0="inet 192.168.1.31/24"
```

起動すると先ほどと同様の内容が確認できる。

```
# ifconfig
tn400: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8d01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10baseT/UTP <half-duplex>)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: tn400 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

ただ、sshで接続するときに時間がかかっている。この症状は、初回のみで2回目からは問題ない。

#### 問題点
他のホスト（192.168.1.30）にピングを飛ばしても届かない

```
# ping 192.168.1.30
PING 192.168.1.30 (192.168.1.30): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
```

しかし、他のホスト（192.168.1.30）からのピングには応答し、その後は他のホスト（192.168.1.30）へのピングが通るようになる。

```
# ping 192.168.1.30
PING 192.168.1.30 (192.168.1.30): 56 data bytes
64 bytes from 192.168.1.30: icmp_seq=0 ttl=64 time=0.127 ms
64 bytes from 192.168.1.30: icmp_seq=1 ttl=64 time=0.108 ms
64 bytes from 192.168.1.30: icmp_seq=2 ttl=64 time=0.110 ms
64 bytes from 192.168.1.30: icmp_seq=3 ttl=64 time=0.108 ms
```

tcpdumpで確認すると自ホスト(bridge0)ではarpが送信されているが、他のホストではarpが届いていないことが分かった。加えて、bridgeを組まない場合はarpが他のホストまで届き、ifconfigの内容が以下のようになり、tn400のflagsにRUNNINGが示されているが、bridgeを組むとRUNNINGがない。

```
# ifconfig
tn400: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8d01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10baseT/UTP <half-duplex>)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
```

## エラーログ
5/26追記
kernel logにエラーが残っていたので書き残す

```sh:/var/log/messages
May 26 07:38:56 host kernel: tn40xx: tn400 Link Up 10G
May 26 07:38:56 host kernel: tn400: link state changed to UP
May 26 07:38:56 host kernel: tn40xx: tn400 Link Up 10G
May 26 07:38:56 host kernel: tn400: promiscuous mode enabled
May 26 07:38:56 host kernel: bridge0: link state changed to UP
May 26 07:38:56 host kernel: bridge0: can't disable some capabilities on tn400: 0x102
May 26 07:38:56 host kernel: epair0a: promiscuous mode enabled
May 26 07:38:56 host kernel: bridge0: can't disable some capabilities on tn400: 0x102
May 26 07:38:56 host kernel: epair1a: promiscuous mode enabled
```

どうも、bridge0がtn400の機能を無効にできないとエラーを吐いているようです。

## qjailの確認
ネットワークに問題があるが、qjailを開始し確認する。

```
# qjail start
Jail successfully started  jail1
Jail successfully started  jail2
# ifconfig
tn400: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8d01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE>
        ether xx:xx:xx:x:xx:xx
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10baseT/UTP <half-duplex>)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: tn400 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

re0に比べ少し時間がかかったが正常に起動したようだ。
## 結論
LREC6860BTとbridgeを組み合わせるとLREC6860BTのflagsのRUNNINGが消えarpが出ていかないため、接続に問題が発生する。
bridge接続を組まず、LREC6860BT単体の場合はRUNNINGとなりarpも出るため正常に接続ができる。

# intelのX540-T1
LREC6860BTを購入時に最悪windowsに使用するが当たってしまった。
## 設定

```sh:/etc/rc.conf
# ifconfig_ix0="inet 192.168.1.31 netmask 255.255.255.0 mtu 9014"
cloned_interfaces="epair0 epair1 bridge0"
ifconfig_epair0a="up mtu 9014"
ifconfig_epair1a="up mtu 9014"
ifconfig_ix0="up mtu 9014"
# ifconfig_re0="up mtu 9014"
ifconfig_bridge0="addm ix0 addm epair0a addm epair1a up mtu 9014"
# ifconfig_bridge0="addm re0 addm epair0a addm epair1a up mtu 9014"
ifconfig_bridge0_alias0="inet 192.168.1.31/24"
```

立ち上がると以下のように表示され問題が全くありませんでした。

```
# ifconfig
ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=a538b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10baseT/UTP <half-duplex>)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0b
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

## qjail起動

```
# qjail start
Jail successfully started  jail1
Jail successfully started  jail2
# ifconfig
ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=a538b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect (10baseT/UTP <half-duplex>)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:36:e1:3c:df:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9014
        options=8<VLAN_MTU>
        ether 02:22:1e:c9:a7:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9014
        ether 02:a4:7a:e0:5c:00
        inet 192.168.1.31 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
```

全く問題なく作動した。

# 転送速度
以下３台で実施する。中間のX540-T2は2ch(ix0,ix1)をブリッジ接続している
Windows10(LREC6860BT) - FreeBSD12(X540-T2) - FreeBSD12(X540-T1)
約9GByteのファイルをWindows10とFreeBSD(X540-T1)でコピーして速度を観測した。

+ Windows10からFreeBSD12へ

![Windows2FreeBSD.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/233460/1860ca58-6e56-ff76-1331-c0f027e563d4.png)

+ FreeBSD12からWindows10へ

![FreeBSD2Windows.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/233460/9ba33031-f4ba-39e8-fe69-a820401197ac.png)

FreeBSD12のディスクは古いSSDであることからこのような差が出たと思うが、まずまずですね。