Oracle Cloud: VRRP+BGPでIPSec VPN Routerを冗長構成してみてみた

#■構成
![NW構成.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/109087/8538bb75-f654-79f9-d2d5-6ccdc9ce56d8.png)

#■目的
Oracle Cloud Infrastructure(OCI)からCPEへの同一CIDRへのパスが複数ある場合、非対称ルーティングとなり、往路と復路でパケットが同一CIDRの複数ある別インターフェース、別CPEを通る動作になります
そのため、CPEを2台使用した冗長構成ではOCIからルータへのパケットは全パスが使用されるのでWAN側をActive-Active構成にします。
CPEのWAN側Active-Active構成での注意点は、LAN側インターフェースに障害が発生した場合、そのCPEのWAN側のインターフェースからパケットが来ないようDownさせるようなルート制御が必要です。
Static Routing(静的ルーティング)の場合は、VRRPのLAN側インターフェース障害発生したら、watch-groupでWAN側IPSec TunnelをDownさせ、VRRPがフェールした生存ルーターのみでパケットが通るようにします
BGP(動的ルーティング）の場合は、LAN側インターフェース障害も検知してLAN側ルート情報がなくなったことを隣接ルーター(neighbors)に伝搬してくれます。
ということで、実装が簡単そうなBGPとVRRPで冗長構成を行い動作検証してみます。
※UNIVERGE IX ルーターのwatch-groupは、CISCOのObject Tracking機能に相当します。





#■構築設定
NEC IXのIPSec VPNとBGPの設定は以前の記事を参考に構成します。
ここではVRRPとBGPの設定を行います。

●参考
・[UNIVERGE IXルーターとOracle CloudをIPSec接続してAS-Path Prependで経路制御してみてみた](https://qiita.com/shirok/items/be7d6b06d7345831d84c)
・[Oracle Cloud:NEC UNIVERGE IXルーターでOCIへIPsec VPN接続してみてみた](https://qiita.com/shirok/items/6501abbc63175bc68ce3)

##●VRRP設定
VRRPはLAN側デフォルトゲートウェイのパスを冗長化するためのVIPで、2台のRouterを使用してActive-Standby構成にします。このVIPをLAN側のクライアントはGatwey IPに設定します。
ここではRouter01をActive(Master)、Router02をStandby(Backup)にするため、Router01のプライオリティを高くして設定します

・Router01設定

```shell-session
vrrp enable
interface GigaEthernet1.0
  vrrp 254 ip 192.168.0.254
  vrrp 254 priority 100
  vrrp 254 ip virtual-host
```

・Router02設定

```shell-session
vrrp enable
interface GigaEthernet1.0
  vrrp 254 ip 192.168.0.254
  vrrp 254 priority 95
  vrrp 254 ip virtual-host
```

##●VRRP設定確認
・Router01設定確認
    
```shell-session
Router01(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254 100 P   master     192.168.0.211
```

・Router02設定確認

```shell-session
Router01(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254  95 P   backup     192.168.0.211
```

##●VRRP動作確認
Router02のプライオリティを高くすることで、Router02がMasterとなりVRRPが遷移し、Router01がBackupになることを確認

・Router02設定と設定確認

```shell-session
Router02(config)# interface GigaEthernet1.0
Router(config-GigaEthernet1.0)# vrrp 254 priority 205

Router02(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254 205 P   master     192.168.0.212
```

・Router01状態確認

```shell-session
Router01(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254 105 P   backup     192.168.0.212
```


##●BGPの設定
・Router01

```shell-session
router bgp 65000
  neighbor 192.168.100.202 remote-as 31898
  neighbor 192.168.100.202 timers 10 30
  neighbor 192.168.100.206 remote-as 31898
  neighbor 192.168.100.206 timers 10 30
  address-family ipv4 unicast
    network 192.168.100.0/24
    network 192.168.0.0/24
```

・Router02

```shell-session
router bgp 65000
  neighbor 192.168.100.210 remote-as 31898
  neighbor 192.168.100.210 timers 10 30
  neighbor 192.168.100.214 remote-as 31898
  neighbor 192.168.100.214 timers 10 30
  address-family ipv4 unicast
    network 192.168.100.0/24
    network 192.168.0.0/24
```

    
##●BGP設定確認

・Router01

```shell-session
Router01(config)# show ip bgp summary
    BGP router ID 192.168.0.211, local AS number 65000
    2 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    192.168.100.202 4    31898 3820    3484    9:24:36       ESTABLISHED
    192.168.100.206 4    31898 3813    3483    9:24:36       ESTABLISHED

    Total number of neighbors 2

Router01(config)# show ip bgp
    BGP table version is 251, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        192.168.100.206                        31898 i
    *>                    192.168.100.202                        31898 i
    192.168.100.0/24   0.0.0.0                  0             i
    *> 192.168.0.0/24   0.0.0.0                  1             i

    Total number of prefixes 6

Router01(config)# show ip bgp neighbors 192.168.100.202 advertised-routes
    BGP table version is 195, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   192.168.100.201          1             i

    Total number of prefixes 1

Router01(config)# show ip bgp neighbors 192.168.100.202 received-routes
    BGP table version is 195, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        192.168.100.202                        31898 i

    Total number of prefixes 1

Router01(config)# show ip bgp neighbors 192.168.100.206 advertised-routes
    BGP table version is 195, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   192.168.100.205          1             i

    Total number of prefixes 1

Router01(config)# show ip route
    IP Routing Table - 9 entries, 3 hidden, 2036 frees
    Entries: 5 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 3:44:56
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 192.168.100.202, Tunnel0.0, 2:45:23
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.200/30 [0/1] is directly connected, Tunnel0.0, 3:00:26
    C      192.168.100.204/30 [0/1] is directly connected, Tunnel1.0, 3:00:26
        192.168.0.0/24 is subnetted, 2 subnets
    C      192.168.0.0/24 [0/1] is directly connected, GigaEthernet1.0, 2:46:52
    C      192.168.0.254/32 [0/1] is directly connected, Virtual254, 2:46:48
```


・Router02

```shell-session
Router02(config)# show ip bgp summary
    BGP router ID 192.168.0.212, local AS number 65000
    4 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    192.168.100.210 4    31898 4482    4084    9:19:47       ESTABLISHED
    192.168.100.214 4    31898 4477    4086    9:19:46       ESTABLISHED

    Total number of neighbors 2


Router02(config)# show ip bgp
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        192.168.100.214                        31898 i
    *                     192.168.100.210                        31898 i
    192.168.100.0/24   0.0.0.0                  0             i
    *> 192.168.0.0/24   0.0.0.0                  1             i

    Total number of prefixes 6


Router02(config-bgp)# show ip bgp neighbors 192.168.100.210 advertised-routes
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   192.168.100.209          1             i

    Total number of prefixes 1

Router02(config-bgp)# show ip bgp neighbors 192.168.100.210 received-routes
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        192.168.100.210                        31898 i

    Total number of prefixes 1

Router02(config-bgp)# show ip bgp neighbors 192.168.100.214 advertised-routes
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   192.168.100.213          1             i

    Total number of prefixes 1

Router02(config-bgp)# show ip bgp neighbors 192.168.100.214 received-routes
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        192.168.100.214                        31898 i

    Total number of prefixes 1


Router02(config-bgp)# show ip route
    IP Routing Table - 8 entries, 3 hidden, 2037 frees
    Entries: 4 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 3:42:27
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 192.168.100.214, Tunnel1.0, 0:12:21
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.208/30 [0/1] is directly connected, Tunnel0.0, 2:57:44
    C      192.168.100.212/30 [0/1] is directly connected, Tunnel1.0, 2:57:44
        192.168.0.0/24 is subnetted, 1 subnets
    C      192.168.0.0/24 [0/1] is directly connected, GigaEthernet1.0, 3:42:33
        219.98.239.0/24 is subnetted, 1 subnets
    C      219.98.239.247/32 [0/1] is directly connected, GigaEthernet0.1, 3:42:27
```



##●OCI IPSec 設定確認
・Router01
  <img width="1226" alt="IPSec01.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/109087/7ea7c7f1-54d4-771e-94cb-7dd181ec3da1.png">

・Router02
<img width="1230" alt="IPSec02.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/109087/34733900-4815-ff4d-9e3f-12d72a1dc991.png">
  



#■Router01：LAN抜線障害検証
Router01のLANを抜線するので、Router01のコマンド確認はコンソールケーブル接続してコマンド確認します

##●Router01 LAN抜線
![IMG_5869.JPG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/109087/1f73187d-f674-8d79-6e75-84b3911a1184.jpeg)


##●ping疎通確認
通信断がないことを確認

・OCI -> OnP ping疎通確認

```shell-session
[opc@oci-inst01 ~]$ ping 192.168.100.250
    PING 192.168.100.250 (192.168.100.250) 56(84) bytes of data.
    64 bytes from 192.168.100.250: icmp_seq=1 ttl=62 time=9.12 ms
    64 bytes from 192.168.100.250: icmp_seq=2 ttl=62 time=15.3 ms
    64 bytes from 192.168.100.250: icmp_seq=3 ttl=62 time=16.8 ms
    64 bytes from 192.168.100.250: icmp_seq=4 ttl=62 time=43.2 ms
    64 bytes from 192.168.100.250: icmp_seq=5 ttl=62 time=8.85 ms
    64 bytes from 192.168.100.250: icmp_seq=6 ttl=62 time=12.1 ms
    64 bytes from 192.168.100.250: icmp_seq=7 ttl=62 time=39.9 ms
    64 bytes from 192.168.100.250: icmp_seq=8 ttl=62 time=128 ms
    64 bytes from 192.168.100.250: icmp_seq=9 ttl=62 time=87.2 ms
    64 bytes from 192.168.100.250: icmp_seq=10 ttl=62 time=85.0 ms
    64 bytes from 192.168.100.250: icmp_seq=11 ttl=62 time=22.5 ms
    64 bytes from 192.168.100.250: icmp_seq=12 ttl=62 time=73.9 ms
    64 bytes from 192.168.100.250: icmp_seq=13 ttl=62 time=42.0 ms
    64 bytes from 192.168.100.250: icmp_seq=14 ttl=62 time=68.0 ms
    64 bytes from 192.168.100.250: icmp_seq=15 ttl=62 time=31.5 ms

    64 bytes from 192.168.100.250: icmp_seq=21 ttl=62 time=22.7 ms
    64 bytes from 192.168.100.250: icmp_seq=22 ttl=62 time=13.3 ms
    64 bytes from 192.168.100.250: icmp_seq=23 ttl=62 time=51.2 ms
    ・・・
```
==> 抜線後5秒間フリーズするが通信断はなかった


・OnP -> OCI ping疎通確認

```shell-session
[user@onp-inst01 ~]$ ping 10.0.0.2
    PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
    64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=95.3 ms
    64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=4.27 ms
    64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.45 ms
    64 bytes from 10.0.0.2: icmp_seq=4 ttl=62 time=108 ms
    64 bytes from 10.0.0.2: icmp_seq=5 ttl=62 time=88.5 ms
    64 bytes from 10.0.0.2: icmp_seq=6 ttl=62 time=88.8 ms
    64 bytes from 10.0.0.2: icmp_seq=7 ttl=62 time=77.2 ms
    64 bytes from 10.0.0.2: icmp_seq=8 ttl=62 time=40.4 ms
    64 bytes from 10.0.0.2: icmp_seq=9 ttl=62 time=48.0 ms
    64 bytes from 10.0.0.2: icmp_seq=10 ttl=62 time=43.1 ms
    64 bytes from 10.0.0.2: icmp_seq=11 ttl=62 time=27.8 ms

    64 bytes from 10.0.0.2: icmp_seq=16 ttl=62 time=7.34 ms
    64 bytes from 10.0.0.2: icmp_seq=17 ttl=62 time=7.45 ms
    64 bytes from 10.0.0.2: icmp_seq=18 ttl=62 time=43.0 ms
    64 bytes from 10.0.0.2: icmp_seq=19 ttl=62 time=76.4 ms
    64 bytes from 10.0.0.2: icmp_seq=20 ttl=62 time=11.5 ms
    64 bytes from 10.0.0.2: icmp_seq=21 ttl=62 time=5.87 ms
    ・・・
```
==> 抜線後5秒間フリーズするが通信断はなかった


##●VRRP確認
Router01からRouter02へVRRPがフェールオーバーしたことを確認

・Router01

```shell-session
Router01(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254 105 P   initialize 0.0.0.0
```

・Router02

```shell-session
Router02(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254  95 P   master     192.168.0.212
```

##●Route確認
Router01のOCIルート10.0.0.0/24とLAN側ルート192.168.0.0/24がなくなっていることを確認

・Router01 

```shell-session
Router01(config)# show ip route
    IP Routing Table - 4 entries, 2 hidden, 2042 frees
    Entries: 3 Connected, 1 Static, 0 RIP, 0 OSPF, 0 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 4:29:26
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.200/30 [0/1] is directly connected, Tunnel0.0, 3:44:57
    C      192.168.100.204/30 [0/1] is directly connected, Tunnel1.0, 3:44:57
```

・ノRouter02

```shell-session
Router02(config)# show ip route
    IP Routing Table - 9 entries, 3 hidden, 2036 frees
    Entries: 5 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 4:39:46
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 192.168.100.214, Tunnel1.0, 1:09:40
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.208/30 [0/1] is directly connected, Tunnel0.0, 3:55:03
    C      192.168.100.212/30 [0/1] is directly connected, Tunnel1.0, 3:55:03
        192.168.0.0/24 is subnetted, 2 subnets
    C      192.168.0.0/24 [0/1] is directly connected, GigaEthernet1.0, 4:39:52
    C      192.168.0.254/32 [0/1] is directly connected, Virtual254, 0:13:38
        219.98.239.0/24 is subnetted, 1 subnets
    C      219.98.239.247/32 [0/1] is directly connected, GigaEthernet0.1, 4:39:46
```

##●BGP Route確認
Router01のOCIルート10.0.0.0/24がなくなっていることを確認

・Router01

```shell-session
Router01(config)# show ip bgp
    BGP table version is 258, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    192.168.100.0/24   0.0.0.0                  0             i
    192.168.0.0/24   0.0.0.0                  0             i

    Total number of prefixes 2
```

・Router02

```shell-session
Router02(config)# show ip bgp
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        192.168.100.214                        31898 i
    *                     192.168.100.210                        31898 i
    192.168.100.0/24   0.0.0.0                  0             i
    *> 192.168.0.0/24   0.0.0.0                  1             i

    Total number of prefixes 6
```

##●隣接ルーターに広告しているLAN側経路192.168.0.0/24は無いことを確認

```shell-session
Router01(config)# show ip  bgp neighbors 169.254.200.202 advertised-routes
    BGP table version is 205, local router ID is 192.168.100.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path

    Total number of prefixes 0

Router01(config)# show ip  bgp neighbors 169.254.200.206 advertised-routes
    BGP table version is 205, local router ID is 192.168.100.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path

    Total number of prefixes 0
```


#■Router01：LAN接続によるフェール・バック確認

##●Router01 LAN抜線切り戻し接続
![IMG_5886.JPG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/109087/c89fe538-a095-07dc-64e5-3e6388a7502f.jpeg)




##●ping疎通確認
通信断がないことを確認

・OCI -> OnP ping疎通確認

```shell-session
[opc@oci-inst01 ~]$ ping 192.168.100.250
    ・・・
    64 bytes from 192.168.100.250: icmp_seq=965 ttl=62 time=13.2 ms
    64 bytes from 192.168.100.250: icmp_seq=966 ttl=62 time=11.4 ms
    64 bytes from 192.168.100.250: icmp_seq=967 ttl=62 time=106 ms
    64 bytes from 192.168.100.250: icmp_seq=968 ttl=62 time=20.8 ms
    64 bytes from 192.168.100.250: icmp_seq=969 ttl=62 time=4.27 ms

    64 bytes from 192.168.100.250: icmp_seq=991 ttl=62 time=13.2 ms
    64 bytes from 192.168.100.250: icmp_seq=992 ttl=62 time=11.4 ms
    64 bytes from 192.168.100.250: icmp_seq=993 ttl=62 time=106 ms
    64 bytes from 192.168.100.250: icmp_seq=994 ttl=62 time=20.8 ms
    64 bytes from 192.168.100.250: icmp_seq=995 ttl=62 time=4.27 ms
```
==> 抜線後22秒間フリーズするが通信断はなかった


・OnP -> OCI ping疎通確認

```shell-session
[user@onp-inst01 ~]$ ping 10.0.0.2
    ・・・
    64 bytes from 10.0.0.2: icmp_seq=960 ttl=62 time=16.7 ms
    64 bytes from 10.0.0.2: icmp_seq=961 ttl=62 time=13.8 ms
    64 bytes from 10.0.0.2: icmp_seq=962 ttl=62 time=31.1 ms
    64 bytes from 10.0.0.2: icmp_seq=963 ttl=62 time=4.66 ms
    64 bytes from 10.0.0.2: icmp_seq=964 ttl=62 time=4.95 ms

    64 bytes from 10.0.0.2: icmp_seq=987 ttl=62 time=16.7 ms
    64 bytes from 10.0.0.2: icmp_seq=989 ttl=62 time=13.8 ms
    64 bytes from 10.0.0.2: icmp_seq=990 ttl=62 time=31.1 ms
    64 bytes from 10.0.0.2: icmp_seq=991 ttl=62 time=4.66 ms
    64 bytes from 10.0.0.2: icmp_seq=992 ttl=62 time=4.95 ms
```
==> 抜線後23秒間フリーズするが通信断はなかった


##●VRRP確認
Router02からRouter01へVRRPがフェールバックしたことを確認

・Router01

```shell-session
Router01(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254 105 P   master     192.168.0.211
```

・Router02

```shell-session
Router02(config)# show vrrp
    Interface            VRID Pri Pre State      Master addr
    GigaEthernet1.0       254  95 P   backup     192.168.0.211
```


##●Route確認
Router01のOCIルート10.0.0.0/24とLAN側ルート192.168.0.0/24が追加されていることを確認

・Router01

```shell-session
Router01(config)# show ip route
    IP Routing Table - 9 entries, 3 hidden, 2036 frees
    Entries: 5 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 4:50:44
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 192.168.100.202, Tunnel0.0, 0:05:54
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.200/30 [0/1] is directly connected, Tunnel0.0, 4:06:15
    C      192.168.100.204/30 [0/1] is directly connected, Tunnel1.0, 4:06:15
        192.168.0.0/24 is subnetted, 2 subnets
    C      192.168.0.0/24 [0/1] is directly connected, GigaEthernet1.0, 0:07:49
    C      192.168.0.254/32 [0/1] is directly connected, Virtual254, 0:07:45
```

・Router02

```shell-session
Router02(config)# show ip route
    IP Routing Table - 8 entries, 3 hidden, 2037 frees
    Entries: 4 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1, 5:00:38
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 192.168.100.214, Tunnel1.0, 1:30:31
        192.168.0.0/16 is subnetted, 2 subnets
    C      192.168.100.208/30 [0/1] is directly connected, Tunnel0.0, 4:15:54
    C      192.168.100.212/30 [0/1] is directly connected, Tunnel1.0, 4:15:54
        192.168.0.0/24 is subnetted, 1 subnets
    C      192.168.0.0/24 [0/1] is directly connected, GigaEthernet1.0, 5:00:43
        219.98.239.0/24 is subnetted, 1 subnets
    C      219.98.239.247/32 [0/1] is directly connected, GigaEthernet0.1, 5:00:38
```

##●BGP Route確認

```shell-session
Router01(config)# show ip bgp
    BGP table version is 262, local router ID is 192.168.0.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        192.168.100.206                        31898 i
    *>                    192.168.100.202                        31898 i
*>                    192.168.100.202                        31898 i
   192.168.100.0/24   0.0.0.0                  0             i
*> 192.168.0.0/24   0.0.0.0                  1             i

Total number of prefixes 6
```


・Router02

```shell-session
Router02(config)# show ip bgp
    BGP table version is 169, local router ID is 192.168.0.212
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        192.168.100.214                        31898 i
    *                     192.168.100.210                        31898 i
    192.168.100.0/24   0.0.0.0                  0             i
    *> 192.168.0.0/24   0.0.0.0                  1             i

    Total number of prefixes 6
    Router02(config)#
    Router02(config)# show ip bgp summary
    BGP router ID 192.168.0.212, local AS number 65000
    2 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    192.168.100.210 4    31898 1851    1693    4:15:53       ESTABLISHED
    192.168.100.214 4    31898 599     547     1:30:43       ESTABLISHED

    Total number of neighbors 2
```

##●隣接ルーターに広告しているLAN側経路192.168.0.0/24が有ることを確認

```shell-session
Router01(config)# show ip  bgp neighbors 169.254.200.202  advertised-routes
    BGP table version is 251, local router ID is 192.168.100.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   169.254.200.201          1             i

    Total number of prefixes 1

Router01(config)# show ip  bgp neighbors 169.254.200.206 advertised-routes
    BGP table version is 251, local router ID is 192.168.100.211
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.0.0/24   169.254.200.205          1             i

    Total number of prefixes 1
```