SoftLayer での Vyatta による IPsec VPN 接続 (BGP単一構成)

## やりたいこと

- データセンター間をIPsec VPNで接続
- 各データセンターでは、デフォルトで割り当てられるPrimary IPサブネットに加えて、独自のIPアドレスのサブネットを作成(BYOIPサブネット)
- BYOIPサブネットのルーティング情報を、データセンター間でBGPで交換する

![image](https://qiita-image-store.s3.amazonaws.com/0/65167/547162aa-0fe3-ccf2-5b97-06813207ab9f.png)

\# IPアドレスはサンプルです。


## 途中経過

vyatta#1側で、vyatta#2から192.168.2.0/24のルートを受けっとってはいるが、ルーティングには出てこない状況。

```
(vyatta#1)$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       > - selected route, * - FIB route, p - stale info

Gateway of last resort is 1.1.1.14 to network 0.0.0.0

S    *> 0.0.0.0/0 [1/0] via 1.1.1.14, bond1
S    *> 10.0.0.0/8 [1/0] via 10.140.24.1, bond0
C    *  10.140.24.0/26 is directly connected, bond0v1
C    *> 10.140.24.0/26 is directly connected, bond0
C    *> 127.0.0.0/8 is directly connected, lo
C    *  1.1.1.0/28 is directly connected, bond1v1
C    *> 1.1.1.0/28 is directly connected, bond1
C    *> 192.168.1.0/24 is directly connected, bond0
C    *> 192.168.10.0/30 is directly connected, vti0
```

```
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 received-routes
BGP table version is 534, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.2.0      192.168.10.2             1             0 65002 i

Total number of prefixes 1
```

同じ症状の事例

- [networking - Simple Vyatta configuration / BGP routes not injecting into routing table - Server Fault](http://serverfault.com/questions/570307/simple-vyatta-configuration-bgp-routes-not-injecting-into-routing-table)
- [BGP route is not installed (IPSec + BGP) - Brocade Community Forums - 58130](http://community.brocade.com/t5/Virtual-Router-Firewall-VPN/BGP-route-is-not-installed-IPSec-BGP/td-p/58130)


## Vyatta config

追加部分のconfigのみ記載

### Vyatta #1
```
# IPsec configuration

## IKE/ESP policy
set vpn ipsec esp-group ESP1 lifetime 3600
set vpn ipsec esp-group ESP1 proposal 1 encryption aes256
set vpn ipsec esp-group ESP1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 lifetime 28800
set vpn ipsec ike-group IKE1 proposal 1 dh-group 5
set vpn ipsec ike-group IKE1 proposal 1 encryption aes256
set vpn ipsec ike-group IKE1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 dead-peer-detection action restart
set vpn ipsec ike-group IKE1 dead-peer-detection interval 15
set vpn ipsec ike-group IKE1 dead-peer-detection timeout 30
set vpn ipsec ipsec-interfaces interface bond1
set vpn ipsec nat-traversal enable

## IPsec tunnel configuration
set vpn ipsec site-to-site peer 2.2.2.2 authentication pre-shared-secret !secret!
set vpn ipsec site-to-site peer 2.2.2.2 ike-group IKE1
set vpn ipsec site-to-site peer 2.2.2.2 local-address 1.1.1.1
set vpn ipsec site-to-site peer 2.2.2.2 vti bind vti0
set vpn ipsec site-to-site peer 2.2.2.2 vti esp-group ESP1

## Virtual tunnel interface configuration
set interfaces vti vti0 address 192.168.10.1/30
set interfaces vti vti0 mtu 1436


# BYOIPサブネット用のGWアドレス
set interface bonding bond0 address 192.168.1.1/24


# BGP configuration

## BGP peering configuration
set protocols bgp 65001 parameters router-id 192.168.10.1
set protocols bgp 65001 neighbor 192.168.10.2 remote-as 65002
set protocols bgp 65001 neighbor 192.168.10.2 soft-reconfiguration 'inbound'
set protocols bgp 65001 neighbor 192.168.10.2 timers holdtime '30'
set protocols bgp 65001 neighbor 192.168.10.2 timers keepalive '30'

## Originating network configuration
set protocols bgp 65001 network 192.168.1.0/24
```


### Vyatta #2

```
# IPsec configuration

## IKE/ESP policy
set vpn ipsec esp-group ESP1 lifetime 3600
set vpn ipsec esp-group ESP1 proposal 1 encryption aes256
set vpn ipsec esp-group ESP1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 lifetime 28800
set vpn ipsec ike-group IKE1 proposal 1 dh-group 5
set vpn ipsec ike-group IKE1 proposal 1 encryption aes256
set vpn ipsec ike-group IKE1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 dead-peer-detection action restart
set vpn ipsec ike-group IKE1 dead-peer-detection interval 15
set vpn ipsec ike-group IKE1 dead-peer-detection timeout 30
set vpn ipsec ipsec-interfaces interface eth1
set vpn ipsec nat-traversal enable

## IPsec tunnel configuration
set vpn ipsec site-to-site peer 1.1.1.1 authentication pre-shared-secret !secret!
set vpn ipsec site-to-site peer 1.1.1.1 ike-group IKE1
set vpn ipsec site-to-site peer 1.1.1.1 local-address 2.2.2.2
set vpn ipsec site-to-site peer 1.1.1.1 vti bind vti0
set vpn ipsec site-to-site peer 1.1.1.1 vti esp-group ESP1

## Virtual tunnel interface configuration
set interfaces vti vti0 address 192.168.10.2/30
set interfaces vti vti0 mtu 1436


# BYOIPサブネット用のGWアドレス
set interface ethernet eth0 address 192.168.2.1/24


# BGP configuration

## BGP peering configuration
set protocols bgp 65002 parameters router-id 192.168.10.2
set protocols bgp 65002 neighbor 192.168.10.1 remote-as 65001
set protocols bgp 65002 neighbor 192.168.10.1 soft-reconfiguration 'inbound'
set protocols bgp 65002 neighbor 192.168.10.1 timers holdtime '30'
set protocols bgp 65002 neighbor 192.168.10.1 timers keepalive '30'

## Originating network configuration
set protocols bgp 65002 network 192.168.2.0/24
```


## 確認



### ルーティング 

`show ip route`

```
(vyatta#1)$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       > - selected route, * - FIB route, p - stale info

Gateway of last resort is 1.1.1.14 to network 0.0.0.0

S    *> 0.0.0.0/0 [1/0] via 1.1.1.14, bond1
S    *> 10.0.0.0/8 [1/0] via 10.140.24.1, bond0
C    *  10.140.24.0/26 is directly connected, bond0v1
C    *> 10.140.24.0/26 is directly connected, bond0
C    *> 127.0.0.0/8 is directly connected, lo
C    *  1.1.1.0/28 is directly connected, bond1v1
C    *> 1.1.1.0/28 is directly connected, bond1
C    *> 192.168.1.0/24 is directly connected, bond0
C    *> 192.168.10.0/30 is directly connected, vti0
```

```
(vyatta#2)$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

S>* 0.0.0.0/0 [1/0] via 2.2.2.6, eth1
S>* 10.0.0.0/8 [1/0] via 10.64.234.65, eth0
C>* 10.64.234.64/26 is directly connected, eth0
C>* 2.2.2.0/29 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
B>* 192.168.1.0/24 [20/0] via 192.168.10.1, vti0, 00:09:00
C>* 192.168.2.0/24 is directly connected, eth0
C>* 192.168.10.0/30 is directly connected, vti0
```


### Neighbor情報 

`show ip bgp neighbors`

```
(vyatta#1)$ show ip bgp neighbors
BGP neighbor is 192.168.10.2, remote AS 65002, local AS 65001, external link
  BGP version 4, remote router ID 192.168.10.2
  BGP state = Established, up for 03:23:54
  Last read 03:23:54, hold time is 30, keepalive interval is 10 seconds
  Configured hold time is 30, keepalive interval is 30 seconds
  Neighbor capabilities:
    Route refresh: advertised and received (old and new)
    Address family IPv4 Unicast: advertised and received
  Received 3257 messages, 2 notifications, 0 in queue
  Sent 3371 messages, 71 notifications, 0 in queue
  Route refresh request: received 0, sent 0
  Minimum time between advertisement runs is 30 seconds
 For address family: IPv4 Unicast
  BGP table version 532, neighbor version 532
  Index 1, Offset 0, Mask 0x2
  Inbound soft reconfiguration allowed
  Community attribute sent to this neighbor (both)
  0 accepted prefixes
  1 announced prefixes

 Connections established 74; dropped 73
Local host: 192.168.10.1, Local port: 179
Foreign host: 192.168.10.2, Foreign port: 58991
Nexthop: 192.168.10.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 03:23:59, due to BGP Notification received
Notification Error Message: (Cease/Other Configuration Change.)
```

```
(vyatta#2)$ show ip bgp neighbors
BGP neighbor is 192.168.10.1, remote AS 65001, local AS 65002, external link
  BGP version 4, remote router ID 192.168.10.1
  BGP state = Established, up for 03:24:17
  Last read 14:40:31, hold time is 30, keepalive interval is 10 seconds
  Configured hold time is 30, keepalive interval is 10 seconds
  Neighbor capabilities:
    4 Byte AS: advertised
    Route refresh: advertised and received(old & new)
    Address family IPv4 Unicast: advertised and received
  Message statistics:
    Inq depth is 0
    Outq depth is 0
                         Sent       Rcvd
    Opens:                 74         73
    Notifications:          5         68
    Updates:              142         77
    Keepalives:          3251       3080
    Route Refresh:          0          0
    Capability:             0          0
    Total:               3472       3298
  Minimum time between advertisement runs is 30 seconds

 For address family: IPv4 Unicast
  Inbound soft reconfiguration allowed
  Community attribute sent to this neighbor(both)
  1 accepted prefixes

  Connections established 74; dropped 73
  Last reset 03:24:28, due to Update source change
Local host: 192.168.10.2, Local port: 58991
Foreign host: 192.168.10.1, Foreign port: 179
Nexthop: 192.168.10.2
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on  Write thread: off
```

### Advertiseしたルート情報

`show ip bgp neighbors <neighbor's IP> advertised-routes`

```
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 advertised-routes
BGP table version is 536, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.1.0      192.168.10.1                  100  32768 i

Total number of prefixes 1
```

```
(vyatta#2)$ show ip bgp neighbors 192.168.10.1 advertised-routes
BGP table version is 0, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.2.0      192.168.10.2             1         32768 i

Total number of prefixes 1
```


### 受信したルート情報

`show ip bgp neighbors <neighbor's IP> received-routes`

```
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 received-routes
BGP table version is 534, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.2.0      192.168.10.2             1             0 65002 i

Total number of prefixes 1
```

```
(vyatta#2)$ show ip bgp neighbors 192.168.10.1 received-routes
BGP table version is 0, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 192.168.1.0      192.168.10.1                           0 65001 i

Total number of prefixes 1
```

## 参考

- [VyOSでBGPを設定したみた - ぽぽぽぽーんのネットワークとOSS](http://komeiy.hatenablog.com/entry/2014/11/27/001750)
- [AWSでのリージョン間接続 その2 BGP単一構成パターン(VPN/VyOS) | cloudpack技術情報サイト](http://blog.cloudpack.jp/2015/01/16/connect-between-aws-regions-with-vpn-vyos-vol2-bgp-pattern/)
- [AWSでのリージョン間接続 その3 BGP冗長構成(VPN/VyOS) - Qiita](http://qiita.com/SatoHiroyuki/items/4769e858df061a16a62c)
- [VPCのVGWに複数のVPNコネクションを作るときの注意点 - すこしふしぎ](http://ssff.hateblo.jp/entry/2014/04/11/223058)
- [How-To: Setting up BGP on Vyatta — page 3 — Setting BGP | Debuntu](http://www.debuntu.org/how-to-setting-up-bgp-on-vyatta-page-3-setting-bgp/)
- [BGP | NetworkNoob](http://www.networknoob.net/category/bgp/)
- [Understanding BGP Convergence](http://blog.ine.com/2010/11/22/understanding-bgp-convergence/)