CML で Arista Fabric の構築と Nexus Fabric との Border 間接続を試す

## はじめに

CML (Cisco Modeling Labs) では KVM でサポートされているイメージに対して「独自のイメージを持ち込む」モデルが許可されているため、Cisco 以外のベンダのイメージを動作させることが可能 ([CML Community, Node and Image Definition Files](https://github.com/CiscoDevNet/cml-community?tab=readme-ov-file#node-and-image-definition-files))

自宅ラボで試しに Arista vEOS を CML に登録して、EVPN VXLAN の Fabric 構成を動作させたので、構築メモを記載する

また、合わせて、Cisco の Nexus 9K の Fabric も構築して、Border 間接続も試してみたので合わせてメモ記載する

## 実施環境

以前に記載した([記事1](https://qiita.com/suzuyui/items/f2ed1738eaa1c8f64408), [記事2](https://qiita.com/suzuyui/items/55349efbcc9ab44a7132)) で記載した自宅ラボ環境

メモリが 96GB (CML2割り当ては92GB)と CML 利用としては多くないので節約して動作させる必要がある (個人環境の限界)


|    項目    |                         スペック                          |
| :--------: | :------------------------------------------------------: |
|   サーバ   |　                       NUC14RVHI7                        |
|    CPU     | Intel Core Ultra 7 155H <br> 16Core(P6/E8/LPE2)/22Thread (CML 割り当て 20vCPU)|
|   Memory   |        96GiB (DDR5-5600 48GiB x2) (CML 割り当て 92GiB)    |
|    Disk    |            1TB PCIe Gen4 (CML 割り当て 512GB)             |
|  拡張 NIC  |          USB NIC 1G x1 <br> (LUA5-U3-AGTE-NBK)           |
|  Host OS  |          Ubuntu24.02 LTS          |
|  HyperVisor  |          KVM          |
|  CML Verion  |          CML-Personal 2.8.1           |



## Arista 構築準備

Arista は CML に標準では含まれてないので、vEOS のダウンロードや CML へのノード定義・インポートなどを実施する

### Arista vEOS イメージ ダウンロード

https://github.com/CiscoDevNet/cml-community/tree/master/node-definitions/arista/veos

```
A valid vEOS image can be downloaded the through Arista support portal. https://www.arista.com/en/support/software-download
```

とあるので、まずはユーザ登録がない場合はユーザ登録をして上記リンクからダウンロードする

#### ユーザ登録

下記サイトでユーザ登録をする  (メールアドレスが`gmail.com`だと制限があり取得できなかったので注意)

https://www.arista.com/en/user-registration

#### vEOS ダウンロード

Version は `4.28` 以降でないと `qcow2` イメージはない (2025.03.08時点では最新は`4.33`)

`vEOS-lab` をクリックしてダウンロードしたいバージョン内の `qcow2` イメージをダウンロードする

以下は、4.32の場合 (2025.03.08時点で最新のメンテナンス(M)バージョン)

![スクリーンショット 2025-03-08 12.06.42.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/a9e97d05-e819-4254-9cb2-7b6b97cacd8a.png)

以下は、4.22の場合

![スクリーンショット 2025-03-08 12.05.00.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/1995db4a-251a-43b3-ad63-a8efb9e72e8d.png)

4.27以前の場合は vmdk しかないので、qcow2 にコンバートする

```sh:vmdkの場合は変換を実施する(v4.22の場合)
qemu-img convert -f vmdk -O qcow2 vEOS-lab-4.22.13M.vmdk vEOS-lab-4.22.13M.qcow2
```

```sh:qemu-imgコマンドが無い場合は、下記コマンドでインストール(Ubuntu)
apt -y install qemu-utils
```
### Arista vEOS Import

vEOS のノード定義とイメージの登録を実施する

`TOOLS` の `Node and Image Definitions` をクリックする

![スクリーンショット 2025-03-08 12.19.24.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/bb29ae13-262d-41ab-9701-bdedad05eaf2.png)

`IMPORT` をクリックする

![スクリーンショット 2025-03-08 12.21.33.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/04c3fa47-50f7-4c44-88d8-03e7390d1dde.png)

下記リンクの`vEOS.yaml`をダウンロードする

https://github.com/CiscoDevNet/cml-community/blob/master/node-definitions/arista/veos/vEOS.yaml

※ Management1 の Interface 設定が違っているので後ほど修正する

※ Provisioning 設定がないがこの記事では深追いしない。なので Config は CML 設定に出力できないので別出しで記載する

アップロードして `GO TO NODE DEFINITON` をクリックする

![スクリーンショット 2025-03-08 12.24.23.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/f5a8365b-b0b3-45d9-95f0-7decdd94864d.png)

Interface 定義が Management1 などが定義されてないので修正する

![スクリーンショット 2025-03-08 13.42.46.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/280f3481-80fb-4876-bfbb-73e336252432.png)

下記のように修正したら下にある`UPDATE`をクリックする

![スクリーンショット 2025-03-08 14.03.19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/9c3e7e29-ab70-4fe8-ae9e-35c1c1735e6c.png)


次に `IMAGE DEFINITIONS` から `MANAGE` をクリックする

![スクリーンショット 2025-03-08 12.28.55.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/c8e091c0-347a-43f5-affb-ead6be816f1b.png)

`Image File` を指定して`UPLOAD IMAGE` をクリックする

![スクリーンショット 2025-03-08 12.35.38.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/a3dbef50-7752-4016-a3d1-259b2f93e82a.png)

アップロード完了後に `+ CREATE NEW IMAGE DEFINITION` をクリックする

![スクリーンショット 2025-03-08 12.36.17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/af6fc24e-0ea6-4968-b3c4-0c40d88fdd7d.png)

`ID` と `Label` を入力して、`Disk Image` をアップロードしたイメージを指定して、下にある`+ CREATE IMAGE DEFINITION`をクリックする

![スクリーンショット 2025-03-08 12.39.13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/97594f2c-0248-4857-8e32-062e6fe49be3.png)

完了するとワークベンチで`vEOS`が選択できる

![スクリーンショット 2025-03-08 12.41.40.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/9289e202-d096-4547-b56d-5261278acc1a.png)

複数バージョンをインポートするとバージョン指定が可能

![スクリーンショット 2025-03-08 12.42.03.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/163aef6b-4c41-4b6f-9f0d-00bdaa2d5a50.png)

以上で、CML で vEOS ノードが構築可能となった

## Arista Fabric (EVPN+VXLAN) の構築

### 構成

Underlay を OSPF、MP-BGP は iBGP で Spine を BGP-RR にして構成する

Spine は 2台、Leaf は3台で 1台目がシングルで、2~3台目が MLAG 試験用の冗長構成 (概説の冗長試験用にするので Border Leaf として使用予定) で構築する

サーバは3台で、2台が同サブネット所属で、1台が別サブネット所属で L2/L3 確認用として構築する

概要図は下記の通り

![arista_fabric_構成概要図.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/fbb5bd4a-14f2-4e4c-9965-11e7a88d555c.png)


CML 上の表示は下記の通り

![スクリーンショット 2025-03-09 1.44.03.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/926ab14f-36ab-4300-80b9-6b83ea97529d.png)

### 設定

CML 設定は Qiita で書くと編集で重くなるので、下記リンク先の GitHub に公開した

[CML 設定 YAML (GitHub)](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/Arista_EOS_fabric.yaml)

Leaf 側設定は CML に保存できるところまでノード定義できてないので、ワークベンチ起動後に機器へ手動で設定をする必要がある

設定方法を下記リンク先で公開している

- [f1-spsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/arista_config/f1-spsw01.cfg.md)
- [f1-spsw02](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/arista_config/f1-spsw02.cfg.md)
- [f1-lfsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/arista_config/f1-lfsw01.cfg.md)
- [f1-bdsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/arista_config/f1-bdsw01.cfg.md)
- [f1-bdsw02](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista/arista_config/f1-bdsw02.cfg.md)

### 動作確認

各ノードで構成が組めていることを確認するために、下記コマンドを実施してログを記載する

#### Spine

アンダーレイと BGP-RR としての動作なので、OSPF のネイバー状態と BGP のサマリ表示を実施する

```sh
show ip ospf neighbor
show bgp evpn summary
```

<details><summary>　f1-spsw01　</summary><div>

```
f1-spsw01#show ip ospf neighbor
Neighbor ID     Instance VRF      Pri State                  Dead Time   Address         Interface
192.168.0.1     1        default  0   FULL                   00:00:31    192.168.3.1     Ethernet1
192.168.0.3     1        default  0   FULL                   00:00:29    192.168.3.9     Ethernet3
192.168.0.4     1        default  0   FULL                   00:00:29    192.168.3.13    Ethernet4
f1-spsw01#show bgp evpn summary
BGP summary information for VRF default
Router identifier 192.168.0.254, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor    V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.1 4 65001           2992      3083    0    0 06:15:00 Estab   3      3
  192.168.0.2 4 65001              0         0    0    0 07:38:58 Active
  192.168.0.3 4 65001           3079      3094    0    0 06:14:25 Estab   2      2
  192.168.0.4 4 65001           2982      3055    0    0 01:48:21 Estab   2      2
```

</div></details>

<details><summary>　f1-spsw02　</summary><div>

```
f1-spsw02#show ip ospf neighbor
Neighbor ID     Instance VRF      Pri State                  Dead Time   Address         Interface
192.168.0.4     1        default  0   FULL                   00:00:38    192.168.4.13    Ethernet4
192.168.0.3     1        default  0   FULL                   00:00:32    192.168.4.9     Ethernet3
192.168.0.1     1        default  0   FULL                   00:00:29    192.168.4.1     Ethernet1
f1-spsw02#show bgp evpn summary
BGP summary information for VRF default
Router identifier 192.168.0.253, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor    V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.1 4 65001            581       581    0    0 01:20:42 Estab   3      3
  192.168.0.2 4 65001              0         0    0    0 01:20:57 Active
  192.168.0.3 4 65001            580       584    0    0 01:20:44 Estab   2      2
  192.168.0.4 4 65001            576       583    0    0 01:20:37 Estab   2      2
```

</div></details>

#### Leaf

VTEP となる箇所なので VXLAN と EVPN の情報を確認する。MLAG もしているので MLAG 状態も確認する

```sh
show vxlan vtep
show vxlan vni
show bgp evpn summary 
show bgp evpn
show bgp evpn mac 
show bgp evpn arp
show mlag
show ip route vrf all
```

<details><summary>　f1-lfsw01　</summary><div>

```
f1-lfsw01#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.3       flood, unicast
192.168.1.4       flood         

Total number of remote VTEPS:  2
f1-lfsw01#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface       802.1Q Tag
----------- ---------- ------------ --------------- ----------
10100       100        static       Ethernet1       100       
                                    Ethernet2       100       
                                    Vxlan1          100       
10200       200        static       Ethernet2       200       
                                    Vxlan1          200       

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4098       TENANT1       evpn         

f1-lfsw01#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.1, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001           4652      4621    0    0 10:53:55 Estab   4      4
  192.168.0.254 4 65001           7154      7039    0    0 15:50:07 Estab   4      4
f1-lfsw01#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.1, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >      RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 -                     -       -       0       i
 * >      RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.1
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 -                     -       -       0       i
f1-lfsw01#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
0                  5254.0064.9ead                        -   
f1-lfsw01#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
0                  172.16.0.1         5254.0064.9ead                     -   
100   10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        -   
100   10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        -   
f1-lfsw01#show mlag
MLAG Configuration:              
domain-id                          :                   
local-interface                    :                   
peer-address                       :             0.0.0.0
peer-link                          :                   
hb-peer-address                    :             0.0.0.0
peer-config                        :                   
                                                       
MLAG Status:                     
state                              :            Disabled
negotiation status                 :                   
peer-link status                   :                   
local-int status                   :                   
system-id                          :   00:00:00:00:00:00
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   0

f1-lfsw01#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 C        192.168.0.1/32
           directly connected, Loopback0
 O        192.168.0.3/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.0.4/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.0.253/32 [110/20]
           via 192.168.4.2, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.2, Ethernet4
 C        192.168.1.1/32
           directly connected, Loopback1
 O        192.168.1.3/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.1.4/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 C        192.168.3.0/30
           directly connected, Ethernet4
 O        192.168.3.8/30 [110/20]
           via 192.168.3.2, Ethernet4
 O        192.168.3.12/30 [110/20]
           via 192.168.3.2, Ethernet4
 C        192.168.4.0/30
           directly connected, Ethernet3
 O        192.168.4.8/30 [110/20]
           via 192.168.4.2, Ethernet3
 O        192.168.4.12/30 [110/20]
           via 192.168.4.2, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.2/32 [200/0]
           via VTEP 192.168.1.3 VNI 19001 router-mac 52:54:00:7a:0e:9c local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan100
 C        172.16.1.0/24
           directly connected, Vlan200

```

</div></details>

<details><summary>　f1-bdsw01　</summary><div>

```
f1-bdsw01#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.1       flood, unicast
192.168.1.4       flood         

Total number of remote VTEPS:  2
f1-bdsw01#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface           802.1Q Tag
----------- ---------- ------------ ------------------- ----------
10100       100        static       Ethernet1           100       
                                    Port-Channel1       100       
                                    Vxlan1              100       

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4097       TENANT1       evpn         

f1-bdsw01#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.3, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001           4684      4681    0    0 10:58:35 Estab   6      6
  192.168.0.254 4 65001           7186      7202    0    0 15:54:10 Estab   6      6
f1-bdsw01#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.3, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 -                     -       -       0       i
 * >      RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.3
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
f1-bdsw01#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
0                  7ac1.d895.b089                        -   
f1-bdsw01#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
100   10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
100   10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
0                  172.16.0.2         7ac1.d895.b089                     -   
f1-bdsw01#show mlag
MLAG Configuration:              
domain-id                          :             domain2
local-interface                    :               Vlan2
peer-address                       :         192.168.2.2
peer-link                          :      Port-Channel99
hb-peer-address                    :             0.0.0.0
peer-config                        :          consistent
                                                       
MLAG Status:                     
state                              :              Active
negotiation status                 :           Connected
peer-link status                   :                  Up
local-int status                   :                  Up
system-id                          :   02:00:1a:d9:4b:92
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   1

f1-bdsw01#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 O        192.168.0.1/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.0.3/32
           directly connected, Loopback0
 O        192.168.0.4/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 O        192.168.0.253/32 [110/20]
           via 192.168.4.10, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.10, Ethernet4
 O        192.168.1.1/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.1.3/32
           directly connected, Loopback1
 O        192.168.1.4/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.2.0/30
           directly connected, Vlan2
 O        192.168.3.0/30 [110/20]
           via 192.168.3.10, Ethernet4
 C        192.168.3.8/30
           directly connected, Ethernet4
 O        192.168.3.12/30 [110/20]
           via 192.168.3.10, Ethernet4
 O        192.168.4.0/30 [110/20]
           via 192.168.4.10, Ethernet3
 C        192.168.4.8/30
           directly connected, Ethernet3
 O        192.168.4.12/30 [110/20]
           via 192.168.4.10, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.1/32 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan100
 B I      172.16.1.0/24 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1

```

</div></details>

<details><summary>　f1-bdsw02　</summary><div>

```
f1-bdsw02#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.1       unicast, flood
192.168.1.3       flood         

Total number of remote VTEPS:  2
f1-bdsw02#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface           802.1Q Tag
----------- ---------- ------------ ------------------- ----------
10100       100        static       Ethernet1           100       
                                    Port-Channel1       100       
                                    Vxlan1              100       

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4097       TENANT1       evpn         

f1-bdsw02#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.4, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001           4700      4688    0    0 11:01:16 Estab   8      8
  192.168.0.254 4 65001           4898      4911    0    0 11:30:53 Estab   8      8
f1-bdsw02#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.4, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *        RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *        RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.4
                                 -                     -       -       0       i
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
f1-bdsw02#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
100   10100  VXLAN 7ac1.d895.b089     192.168.1.3        -   
0                  7ac1.d895.b089                        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
100   10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
f1-bdsw02#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
100   10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
100   10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
100   10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        -   
100   10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        -   
0                  172.16.0.2         7ac1.d895.b089                     -   
f1-bdsw02#show mlag
MLAG Configuration:              
domain-id                          :             domain2
local-interface                    :               Vlan2
peer-address                       :         192.168.2.1
peer-link                          :      Port-Channel99
hb-peer-address                    :             0.0.0.0
peer-config                        :          consistent
                                                       
MLAG Status:                     
state                              :              Active
negotiation status                 :           Connected
peer-link status                   :                  Up
local-int status                   :                  Up
system-id                          :   02:00:1a:d9:4b:92
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   1

f1-bdsw02#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 O        192.168.0.1/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 O        192.168.0.3/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 C        192.168.0.4/32
           directly connected, Loopback0
 O        192.168.0.253/32 [110/20]
           via 192.168.4.14, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.14, Ethernet4
 O        192.168.1.1/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 O        192.168.1.3/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 C        192.168.1.4/32
           directly connected, Loopback1
 C        192.168.2.0/30
           directly connected, Vlan2
 O        192.168.3.0/30 [110/20]
           via 192.168.3.14, Ethernet4
 O        192.168.3.8/30 [110/20]
           via 192.168.3.14, Ethernet4
 C        192.168.3.12/30
           directly connected, Ethernet4
 O        192.168.4.0/30 [110/20]
           via 192.168.4.14, Ethernet3
 O        192.168.4.8/30 [110/20]
           via 192.168.4.14, Ethernet3
 C        192.168.4.12/30
           directly connected, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.1/32 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1
 B I      172.16.0.2/32 [200/0]
           via VTEP 192.168.1.3 VNI 19001 router-mac 52:54:00:7a:0e:9c local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan100
 B I      172.16.1.0/24 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1

```

</div></details>


#### Server

各サーバの Interface 設定 (`ifconfig` or `ip addr show`) と ping 疎通を確認した

```
ifconfig
ip addr show
ping 172.16.0.1
ping 172.16.0.2
ping 172.16.1.1
```

<details><summary>　tenant1-server01　</summary><div>

```
cisco@tenant1-server01:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:64:9E:AD  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:883 errors:0 dropped:0 overruns:0 frame:0
          TX packets:962 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:90319 (88.2 KiB)  TX bytes:121257 (118.4 KiB)

eth0.100  Link encap:Ethernet  HWaddr 52:54:00:64:9E:AD  
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:883 errors:0 dropped:0 overruns:0 frame:0
          TX packets:869 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:74425 (72.6 KiB)  TX bytes:85975 (83.9 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

cisco@tenant1-server01:~$ ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2): 56 data bytes
64 bytes from 172.16.0.2: seq=0 ttl=64 time=8.305 ms
64 bytes from 172.16.0.2: seq=1 ttl=64 time=7.877 ms
64 bytes from 172.16.0.2: seq=2 ttl=64 time=10.378 ms
^C
--- 172.16.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 7.877/8.853/10.378 ms
cisco@tenant1-server01:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=63 time=5.704 ms
64 bytes from 172.16.1.1: seq=1 ttl=63 time=2.351 ms
64 bytes from 172.16.1.1: seq=2 ttl=63 time=2.496 ms
^C
--- 172.16.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 2.351/3.517/5.704 ms
cisco@tenant1-server01:~$ 
```

</div></details>

<details><summary>　tenant1-server02　</summary><div>

```
cisco@tenant1-server02:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:4b:31:62
    altname enp0s2
3: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:47:5a:29
    altname enp0s3
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::78c1:d8ff:fe95:b089/64 scope link 
       valid_lft forever preferred_lft forever
5: bond0.100@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.2/24 brd 172.16.0.255 scope global bond0.100
       valid_lft forever preferred_lft forever
    inet6 fe80::78c1:d8ff:fe95:b089/64 scope link 
       valid_lft forever preferred_lft forever
cisco@tenant1-server02:~$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=5.90 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=5.75 ms
64 bytes from 172.16.0.1: icmp_seq=3 ttl=64 time=7.75 ms
^C
--- 172.16.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 5.749/6.464/7.748/0.909 ms
cisco@tenant1-server02:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_seq=1 ttl=62 time=6.21 ms
64 bytes from 172.16.1.1: icmp_seq=2 ttl=62 time=5.91 ms
64 bytes from 172.16.1.1: icmp_seq=3 ttl=62 time=4.00 ms
^C
--- 172.16.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 3.996/5.371/6.211/0.980 ms
cisco@tenant1-server02:~$ 
```

</div></details>

<details><summary>　tenant1-server03　</summary><div>

```
cisco@tenant1-server03:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:FE:69:7D  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2450 errors:0 dropped:1 overruns:0 frame:0
          TX packets:7716 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:257477 (251.4 KiB)  TX bytes:2023462 (1.9 MiB)

eth0.200  Link encap:Ethernet  HWaddr 52:54:00:FE:69:7D  
          inet addr:172.16.1.1  Bcast:172.16.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2449 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2586 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:213276 (208.2 KiB)  TX bytes:258658 (252.5 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:504 (504.0 B)  TX bytes:504 (504.0 B)

cisco@tenant1-server03:~$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1): 56 data bytes
64 bytes from 172.16.0.1: seq=0 ttl=63 time=2.734 ms
64 bytes from 172.16.0.1: seq=1 ttl=63 time=2.675 ms
64 bytes from 172.16.0.1: seq=2 ttl=63 time=2.826 ms
^C
--- 172.16.0.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 2.675/2.745/2.826 ms
cisco@tenant1-server03:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=64 time=0.029 ms
64 bytes from 172.16.1.1: seq=1 ttl=64 time=0.044 ms
64 bytes from 172.16.1.1: seq=2 ttl=64 time=0.063 ms
^C
--- 172.16.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.029/0.045/0.063 ms
cisco@tenant1-server03:~$ 
```

</div></details>


以上で CML 上での Arista vEOS で Fabric 構築を完了

## Nexus Fabric との接続試験構成構築

Fabric 間の接続・移行は実施可否含めて検討範囲だと思うが、一旦 Border 間での接続を MLAG + vPC を利用して構築して試してみる

### 試験構成

上記で試した Arista Fabric 構成を拡張して下記の構成で構築する

- Cisco Fabric を `NX-OS 9000` のノードイメージを使用して構築する
    - [過去記事](https://qiita.com/suzuyui/items/665507e992becf682f5a)で構築した構成・設定のノード数を変更したもの (leafを削減して、border-leafは冗長試験のため2つにしている。設定は基本踏襲)
- Border 間を MLAG(Arista) と vPC (Cisco) で接続する
- 今回は、上記でサーバを接続していた VLAN を L2 でそのまま延伸するところだけ試す
    - tenant1-server02 を Fabric1 から Fabric2 の Leaf へ移動して疎通確認をしている

![arista_n9k_baric_bd_connect_構成概要図.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/0edc3221-c50f-4d3e-bc70-a37fb296dafa.png)

CML 上の表示は下記の通り

![arista_n9k_baric_bd_connect_cml図.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/00a94c60-a55f-42c1-93b7-8cdd0d827e63.png)

### 構築

構築自体は上記から拡張で実施した。設定方法は長くなるので GitHub 側に記載している

https://github.com/suzuyu/nxos-terraform/tree/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con#cml-%E6%A7%8B%E7%AF%89

機器の設定自体は下記の CML YAML で Cicso の設定済みにして入れている (長くなるので GitHub 先で共有)

[CML YAML (GitHub先リンク)](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/Arista_EOS_fabric_And_Cisco_NX9K_fabric.yaml)

Arista の機器設定は下記の通り

- [f1-spsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/arista_config/config_back/f1-spsw01.cfg)
- [f1-spsw02](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/arista_config/config_back/f1-spsw02.cfg)
- [f1-lfsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/arista_config/config_back/f1-lfsw01.cfg)
- [f1-bdsw01](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/arista_config/config_back/f1-bdsw01.cfg)
- [f1-bdsw02](https://github.com/suzuyu/nxos-terraform/blob/v1.1.0/env/cml/cml_yaml/arista_n9k_fabric_bd_con/arista_config/config_back/f1-bdsw02.cfg)


### 試験構成確認

各機器の状態確認を実施する


#### サーバ MAC アドレス / Ping 確認

各サーバの Interface 設定(MACアドレス) (ifconfig or ip addr show) と ping 疎通を確認した

```sh
ifconfig
ip addr show
ping 172.16.0.1
ping 172.16.0.2
ping 172.16.1.1
```

<details><summary>　tenant1-server01 (fabric1)　</summary><div>

```
cisco@tenant1-server01:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:64:9E:AD  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5211 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6324 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:501787 (490.0 KiB)  TX bytes:748485 (730.9 KiB)

eth0.100  Link encap:Ethernet  HWaddr 52:54:00:64:9E:AD  
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5211 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5742 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:407989 (398.4 KiB)  TX bytes:526473 (514.1 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:188 errors:0 dropped:0 overruns:0 frame:0
          TX packets:188 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:21056 (20.5 KiB)  TX bytes:21056 (20.5 KiB)

cisco@tenant1-server01:~$ ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2): 56 data bytes
64 bytes from 172.16.0.2: seq=0 ttl=64 time=14.797 ms
64 bytes from 172.16.0.2: seq=1 ttl=64 time=15.443 ms
64 bytes from 172.16.0.2: seq=2 ttl=64 time=22.952 ms
^C
--- 172.16.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 14.797/17.730/22.952 ms
cisco@tenant1-server01:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: seq=0 ttl=63 time=3.025 ms
64 bytes from 172.16.1.1: seq=1 ttl=63 time=4.890 ms
^C
--- 172.16.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 3.025/3.957/4.890 ms
cisco@tenant1-server01:~$ 
```

</div></details>

<details><summary>　tenant1-server02 (fabric2)　</summary><div>

```
cisco@tenant1-server02:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:4b:31:62
    altname enp0s2
3: ens3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:47:5a:29
    altname enp0s3
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::78c1:d8ff:fe95:b089/64 scope link 
       valid_lft forever preferred_lft forever
5: bond0.100@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:c1:d8:95:b0:89 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.2/24 brd 172.16.0.255 scope global bond0.100
       valid_lft forever preferred_lft forever
    inet6 fe80::78c1:d8ff:fe95:b089/64 scope link 
       valid_lft forever preferred_lft forever
cisco@tenant1-server02:~$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=11.5 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=14.1 ms
64 bytes from 172.16.0.1: icmp_seq=3 ttl=64 time=17.1 ms
^C
--- 172.16.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 11.546/14.263/17.112/2.274 ms
cisco@tenant1-server02:~$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_seq=1 ttl=62 time=12.5 ms
64 bytes from 172.16.1.1: icmp_seq=2 ttl=62 time=12.3 ms
64 bytes from 172.16.1.1: icmp_seq=3 ttl=62 time=13.2 ms
^C
--- 172.16.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 12.253/12.650/13.170/0.384 ms
```

</div></details>

<details><summary>　tenant1-server03 (fabric1) </summary><div>

```
cisco@tenant1-server03:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 52:54:00:FE:69:7D  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4519 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5863 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:463583 (452.7 KiB)  TX bytes:719379 (702.5 KiB)

eth0.200  Link encap:Ethernet  HWaddr 52:54:00:FE:69:7D  
          inet addr:172.16.1.1  Bcast:172.16.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4519 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5272 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:382241 (373.2 KiB)  TX bytes:496169 (484.5 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

cisco@tenant1-server03:~$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1): 56 data bytes
64 bytes from 172.16.0.1: seq=0 ttl=63 time=3.690 ms
64 bytes from 172.16.0.1: seq=1 ttl=63 time=2.813 ms
^C
--- 172.16.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 2.813/3.251/3.690 ms
cisco@tenant1-server03:~$ ping 172.16.0.2
PING 172.16.0.2 (172.16.0.2): 56 data bytes
64 bytes from 172.16.0.2: seq=0 ttl=62 time=14.677 ms
64 bytes from 172.16.0.2: seq=1 ttl=62 time=27.135 ms
64 bytes from 172.16.0.2: seq=2 ttl=62 time=17.423 ms
^C
--- 172.16.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 14.677/19.745/27.135 ms
```

</div></details>


#### 各 Leaf 状態確認

構成図の左側の機器から確認していく

```sh:arista
show vxlan vtep
show vxlan vni
show bgp evpn summary 
show bgp evpn
show bgp evpn mac 
show bgp evpn arp
show mlag
show ip route vrf all
```

<details><summary>　f1-lfsw01　</summary><div>

```
f1-lfsw01#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.3       flood, unicast
192.168.1.4       flood, unicast

Total number of remote VTEPS:  2
f1-lfsw01#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface       802.1Q Tag
----------- ---------- ------------ --------------- ----------
10100       100        static       Ethernet1       100       
                                    Ethernet2       100       
                                    Vxlan1          100       
10200       200        static       Ethernet2       200       
                                    Vxlan1          200       

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4097       TENANT1       evpn         

f1-lfsw01#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.1, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001            717       669    0    0 01:34:12 Estab   5      5
  192.168.0.254 4 65001            719       666    0    0 01:34:28 Estab   5      5
f1-lfsw01#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.1, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >      RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 -                     -       -       0       i
 * >      RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.1
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 -                     -       -       0       i
f1-lfsw01#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
0     10100  VXLAN 52cd.4475.1b08     192.168.1.4        1   
0     10100  VXLAN 52cd.4475.1b08     192.168.1.4        1   
0                  5254.0064.9ead                        -   
0     10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
0     10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
0     10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
0     10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
f1-lfsw01#
f1-lfsw01#
f1-lfsw01#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
0                  172.16.0.1         5254.0064.9ead                     -   
0     10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        1   
0     10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        1   
f1-lfsw01#show mlag
MLAG Configuration:              
domain-id                          :                   
local-interface                    :                   
peer-address                       :             0.0.0.0
peer-link                          :                   
hb-peer-address                    :             0.0.0.0
peer-config                        :                   
                                                       
MLAG Status:                     
state                              :            Disabled
negotiation status                 :                   
peer-link status                   :                   
local-int status                   :                   
system-id                          :   00:00:00:00:00:00
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   0

f1-lfsw01#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 C        192.168.0.1/32
           directly connected, Loopback0
 O        192.168.0.3/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.0.4/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.0.253/32 [110/20]
           via 192.168.4.2, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.2, Ethernet4
 C        192.168.1.1/32
           directly connected, Loopback1
 O        192.168.1.3/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 O        192.168.1.4/32 [110/30]
           via 192.168.4.2, Ethernet3
           via 192.168.3.2, Ethernet4
 C        192.168.3.0/30
           directly connected, Ethernet4
 O        192.168.3.8/30 [110/20]
           via 192.168.3.2, Ethernet4
 O        192.168.3.12/30 [110/20]
           via 192.168.3.2, Ethernet4
 C        192.168.4.0/30
           directly connected, Ethernet3
 O        192.168.4.8/30 [110/20]
           via 192.168.4.2, Ethernet3
 O        192.168.4.12/30 [110/20]
           via 192.168.4.2, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.2/32 [200/0]
           via VTEP 192.168.1.3 VNI 19001 router-mac 52:54:00:7a:0e:9c local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan100
 C        172.16.1.0/24
           directly connected, Vlan200

```

</div></details>

<details><summary>　f1-bdsw01　</summary><div>

```
f1-bdsw01#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.1       flood, unicast
192.168.1.4       flood         

Total number of remote VTEPS:  2
f1-bdsw01#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface           802.1Q Tag
----------- ---------- ------------ ------------------- ----------
10100       2010       static       Ethernet1           2010      
                                    Port-Channel1       2010      
                                    Vxlan1              2010      

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4097       TENANT1       evpn         

f1-bdsw01#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.3, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001            707       706    0    0 01:35:06 Estab   7      7
  192.168.0.254 4 65001            705       702    0    0 01:35:06 Estab   7      7
f1-bdsw01#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.3, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *        RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 -                     -       -       0       i
 * >      RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 -                     -       -       0       i
 * >      RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.3
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.4
                                 192.168.1.4           -       100     0       i Or-ID: 192.168.0.4 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
f1-bdsw01#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
2010  10100  VXLAN 52cd.4475.1b08     192.168.1.4        1   
2010  10100  VXLAN 52cd.4475.1b08     192.168.1.4        1   
0                  52cd.4475.1b08                        -   
0                  7ac1.d895.b089                        1   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
f1-bdsw01#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
2010  10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
2010  10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
0                  172.16.0.2         7ac1.d895.b089                     1   
f1-bdsw01#show mlag
MLAG Configuration:              
domain-id                          :             domain2
local-interface                    :               Vlan2
peer-address                       :         192.168.2.2
peer-link                          :      Port-Channel99
hb-peer-address                    :             0.0.0.0
peer-config                        :          consistent
                                                       
MLAG Status:                     
state                              :              Active
negotiation status                 :           Connected
peer-link status                   :                  Up
local-int status                   :                  Up
system-id                          :   02:00:1a:d9:4b:92
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   1

f1-bdsw01#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 O        192.168.0.1/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.0.3/32
           directly connected, Loopback0
 O        192.168.0.4/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 O        192.168.0.253/32 [110/20]
           via 192.168.4.10, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.10, Ethernet4
 O        192.168.1.1/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.1.3/32
           directly connected, Loopback1
 O        192.168.1.4/32 [110/30]
           via 192.168.4.10, Ethernet3
           via 192.168.3.10, Ethernet4
 C        192.168.2.0/30
           directly connected, Vlan2
 O        192.168.3.0/30 [110/20]
           via 192.168.3.10, Ethernet4
 C        192.168.3.8/30
           directly connected, Ethernet4
 O        192.168.3.12/30 [110/20]
           via 192.168.3.10, Ethernet4
 O        192.168.4.0/30 [110/20]
           via 192.168.4.10, Ethernet3
 C        192.168.4.8/30
           directly connected, Ethernet3
 O        192.168.4.12/30 [110/20]
           via 192.168.4.10, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.1/32 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan2010
 B I      172.16.1.0/24 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1

```

</div></details>

<details><summary>　f1-bdsw02 </summary><div>

```
f1-bdsw02#show vxlan vtep
Remote VTEPS for Vxlan1:

VTEP              Tunnel Type(s)
----------------- --------------
192.168.1.1       flood, unicast
192.168.1.3       flood         

Total number of remote VTEPS:  2
f1-bdsw02#show vxlan vni
VNI to VLAN Mapping for Vxlan1
VNI         VLAN       Source       Interface           802.1Q Tag
----------- ---------- ------------ ------------------- ----------
10100       2010       static       Ethernet1           2010      
                                    Port-Channel1       2010      
                                    Vxlan1              2010      

VNI to dynamic VLAN Mapping for Vxlan1
VNI         VLAN       VRF           Source       
----------- ---------- ------------- ------------ 
19001       4097       TENANT1       evpn         

f1-bdsw02#show bgp evpn summary 
BGP summary information for VRF default
Router identifier 192.168.0.4, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor      V AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.0.253 4 65001            717       720    0    0 01:36:52 Estab   8      8
  192.168.0.254 4 65001            722       710    0    0 01:36:54 Estab   8      8
f1-bdsw02#show bgp evpn
BGP routing table information for VRF default
Router identifier 192.168.0.4, local AS number 65001
Route status codes: * - valid, > - active, S - Stale, E - ECMP head, e - ECMP
                    c - Contributing to ECMP, % - Pending best path selection
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

          Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 5254.0064.9ead 172.16.0.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 mac-ip 10100 52cd.4475.1b08
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *        RD: 65001:19000 mac-ip 10100 7ac1.d895.b089
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *        RD: 65001:19000 mac-ip 10100 7ac1.d895.b089 172.16.0.2
                                 -                     -       -       0       i
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.1
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >Ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 *  ec    RD: 65001:19000 imet 10100 192.168.1.3
                                 192.168.1.3           -       100     0       i Or-ID: 192.168.0.3 C-LST: 0.0.0.1 
 * >      RD: 65001:19000 imet 10100 192.168.1.4
                                 -                     -       -       0       i
 * >      RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 -                     -       -       0       i
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.0.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 * >      RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
 *        RD: 65001:19001 ip-prefix 172.16.1.0/24
                                 192.168.1.1           -       100     0       i Or-ID: 192.168.0.1 C-LST: 0.0.0.1 
f1-bdsw02#show bgp evpn mac 
VLAN  Label  Encap MAC                Tunnel Endpoint    Seq#
----- ------ ----- ------------------ ------------------ ----
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 5254.0064.9ead     192.168.1.1        -   
2010  10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
2010  10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
2010  10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
2010  10100  VXLAN 7ac1.d895.b089     192.168.1.3        1   
0                  7ac1.d895.b089                        -   
0                  52cd.4475.1b08                        1   
f1-bdsw02#show bgp evpn arp
VLAN  Label  Encap IP                 MAC             Tunnel Endpoint    Seq#
----- ------ ----- ------------------ --------------- ------------------ ----
2010  10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        1   
2010  10100  VXLAN 172.16.0.2         7ac1.d895.b089  192.168.1.3        1   
0                  172.16.0.2         7ac1.d895.b089                     -   
2010  10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
2010  10100  VXLAN 172.16.0.1         5254.0064.9ead  192.168.1.1        -   
f1-bdsw02#show mlag
MLAG Configuration:              
domain-id                          :             domain2
local-interface                    :               Vlan2
peer-address                       :         192.168.2.1
peer-link                          :      Port-Channel99
hb-peer-address                    :             0.0.0.0
peer-config                        :          consistent
                                                       
MLAG Status:                     
state                              :              Active
negotiation status                 :           Connected
peer-link status                   :                  Up
local-int status                   :                  Up
system-id                          :   02:00:1a:d9:4b:92
dual-primary detection             :            Disabled
dual-primary interface errdisabled :               False
                                                       
MLAG Ports:                      
Disabled                           :                   0
Configured                         :                   0
Inactive                           :                   0
Active-partial                     :                   0
Active-full                        :                   1

f1-bdsw02#show ip route vrf all

VRF: default
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 O        192.168.0.1/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 O        192.168.0.3/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 C        192.168.0.4/32
           directly connected, Loopback0
 O        192.168.0.253/32 [110/20]
           via 192.168.4.14, Ethernet3
 O        192.168.0.254/32 [110/20]
           via 192.168.3.14, Ethernet4
 O        192.168.1.1/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 O        192.168.1.3/32 [110/30]
           via 192.168.4.14, Ethernet3
           via 192.168.3.14, Ethernet4
 C        192.168.1.4/32
           directly connected, Loopback1
 C        192.168.2.0/30
           directly connected, Vlan2
 O        192.168.3.0/30 [110/20]
           via 192.168.3.14, Ethernet4
 O        192.168.3.8/30 [110/20]
           via 192.168.3.14, Ethernet4
 C        192.168.3.12/30
           directly connected, Ethernet4
 O        192.168.4.0/30 [110/20]
           via 192.168.4.14, Ethernet3
 O        192.168.4.8/30 [110/20]
           via 192.168.4.14, Ethernet3
 C        192.168.4.12/30
           directly connected, Ethernet3


VRF: MGMT
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort:
 S        0.0.0.0/0 [1/0]
           via 192.168.129.254, Management1

 C        192.168.129.0/24
           directly connected, Management1


VRF: TENANT1
Source Codes:
       C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - Other BGP Routes,
       B I - iBGP, B E - eBGP, R - RIP, I L1 - IS-IS level 1,
       I L2 - IS-IS level 2, O3 - OSPFv3, A B - BGP Aggregate,
       A O - OSPF Summary, NG - Nexthop Group Static Route,
       V - VXLAN Control Service, M - Martian,
       DH - DHCP client installed default route,
       DP - Dynamic Policy Route, L - VRF Leaked,
       G  - gRIBI, RC - Route Cache Route,
       CL - CBF Leaked Route

Gateway of last resort is not set

 B I      172.16.0.1/32 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1
 B I      172.16.0.2/32 [200/0]
           via VTEP 192.168.1.3 VNI 19001 router-mac 52:54:00:7a:0e:9c local-interface Vxlan1
 C        172.16.0.0/24
           directly connected, Vlan2010
 B I      172.16.1.0/24 [200/0]
           via VTEP 192.168.1.1 VNI 19001 router-mac 52:54:00:ce:b3:b8 local-interface Vxlan1

```

</div></details>

```sh:cisco
show nve peers
show nve vni
show bgp l2vpn evpn summary
show bgp l2vpn evpn
show l2route evpn mac all
show l2route evpn mac-ip all
show vpc brief
show ip route vrf all
```

<details><summary>　f2-bdsw01　</summary><div>

```
f2-bdsw01# show nve peers
Interface Peer-IP                                 State LearnType Uptime   Router-Mac       
--------- --------------------------------------  ----- --------- -------- -----------------
nve1      10.0.2.2                                Up    CP        01:34:28 n/a              

f2-bdsw01# show nve vni
Codes: CP - Control Plane        DP - Data Plane          
       UC - Unconfigured         SA - Suppress ARP        
       S-ND - Suppress ND        
       SU - Suppress Unknown Unicast 
       Xconn - Crossconnect      
       MS-IR - Multisite Ingress Replication 
       HYB - Hybrid IRB mode
    
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      20100    UnicastBGP        Up    CP   L2 [2010]               

f2-bdsw01# show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.0.0.1, local AS number 65001
BGP table version is 20, L2VPN EVPN config peers 2, capable peers 1
10 network entries and 10 paths using 2808 bytes of memory
BGP attribute entries [7/2576], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [1/4]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.0.253      4     0          0          0        0    0    0 01:37:43 Idle     
10.0.0.254      4 65001        110        101       20    0    0 01:36:13 2         

Neighbor        T    AS PfxRcd     Type-2     Type-3     Type-4     Type-5     Type-12   
10.0.0.253      I     0 Idle       0          0          0          0          0         
10.0.0.254      I 65001 2          1          1          0          0          0         
f2-bdsw01# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 20, Local Router ID is 10.0.0.1
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 10.0.0.1:34777    (L2VNI 20100)
*>l[2]:[0]:[0]:[48]:[001c.7300.0999]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5254.0064.9ead]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5254.007a.0e9c]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5267.5475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[52cd.4475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>i[2]:[0]:[0]:[48]:[7ac1.d895.b089]:[0]:[0.0.0.0]/216
                      10.0.2.2                          100          0 i
*>l[3]:[0]:[32]:[10.0.2.1]/88
                      10.0.2.1                          100      32768 i
*>i[3]:[0]:[32]:[10.0.2.2]/88
                      10.0.2.2                          100          0 i

Route Distinguisher: 10.0.0.3:32867
*>i[2]:[0]:[0]:[48]:[7ac1.d895.b089]:[0]:[0.0.0.0]/216
                      10.0.2.2                          100          0 i
*>i[3]:[0]:[32]:[10.0.2.2]/88
                      10.0.2.2                          100          0 i

f2-bdsw01# show l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole, (Dum):Dummy
(Pf):Permanently-Frozen, (Orp): Orphan

(PipOrp): Directly connected Orphan to PIP based vPC BGW 
(PipPeerOrp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Prod   Flags              Seq No     Next-Hops                              
----------- -------------- ------ ------------------ ---------- ---------------------------------------------------------
2010        001c.7300.0999 Local  L,                 0          Po11
2010        5254.0064.9ead Local  L,                 0          Po11
2010        5254.007a.0e9c Local  L,                 0          Po11
2010        5267.5475.1b08 VXLAN  Stt,Nho,Dum,       0          10.0.2.1
2010        52cd.4475.1b08 Local  L,                 0          Po11
2010        7ac1.d895.b089 BGP    Rcv                0          10.0.2.2 (Label: 20100)
f2-bdsw01# show l2route evpn mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated (Orp):Orphan (Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole
(Piporp): Directly connected Orphan to PIP based vPC BGW 
(Pipporp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Host IP                                 Prod   Flags              Seq No     Next-Hops                              
----------- -------------- --------------------------------------- ------ ----------------- ---------- ----------------------------------------------------
-----
f2-bdsw01# show vpc brief
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1   
Peer status                       : peer adjacency formed ok      
vPC keep-alive status             : peer is alive                 
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : primary                       
Number of vPCs configured         : 1   
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Disabled
Delay-restore status              : Timer is off.(timeout = 30s)
Delay-restore SVI status          : Timer is off.(timeout = 10s)
Delay-restore Orphan-port status  : Timer is off.(timeout = 0s)
Operational Layer3 Peer-router    : Disabled
Virtual-peerlink mode             : Enabled

vPC Peer-link status
---------------------------------------------------------------------
id    Port   Status Active vlans    
--    ----   ------ -------------------------------------------------
1     Po1    up     1,2010                                                               

vPC status
----------------------------------------------------------------------------
Id    Port          Status Consistency Reason                Active vlans
--    ------------  ------ ----------- ------                ---------------
11    Po11          up     success     success               2010                        
                                                                                         

Please check "show vpc consistency-parameters vpc <vpc-num>" for the 
consistency reason of down vpc and for type-2 consistency reasons for 
any vpc.

f2-bdsw01# show ip route vrf all
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.0.0.1/32, ubest/mbest: 2/0, attached
    *via 10.0.0.1, Lo0, [0/0], 01:37:56, local
    *via 10.0.0.1, Lo0, [0/0], 01:37:56, direct
10.0.0.2/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/81], 01:36:26, ospf-1, intra
10.0.0.3/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/81], 01:36:38, ospf-1, intra
10.0.0.254/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/41], 01:36:38, ospf-1, intra
10.0.1.1/32, ubest/mbest: 2/0, attached
    *via 10.0.1.1, Lo1, [0/0], 01:33:22, local
    *via 10.0.1.1, Lo1, [0/0], 01:33:22, direct
10.0.1.2/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/81], 01:33:22, ospf-1, intra
10.0.1.3/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/81], 01:36:38, ospf-1, intra
10.0.2.1/32, ubest/mbest: 2/0, attached
    *via 10.0.2.1, Lo1, [0/0], 01:33:22, local
    *via 10.0.2.1, Lo1, [0/0], 01:33:22, direct
10.0.2.2/32, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/81], 01:36:38, ospf-1, intra
10.0.3.0/31, ubest/mbest: 1/0, attached
    *via 10.0.3.0, Eth1/4, [0/0], 01:36:49, direct
10.0.3.0/32, ubest/mbest: 1/0, attached
    *via 10.0.3.0, Eth1/4, [0/0], 01:36:49, local
10.0.3.2/31, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/80], 01:36:38, ospf-1, intra
10.0.3.4/31, ubest/mbest: 1/0
    *via 10.0.3.1, Eth1/4, [110/80], 01:36:38, ospf-1, intra

IP Route Table for VRF "management"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

0.0.0.0/0, ubest/mbest: 1/0
    *via 192.168.129.254, [1/0], 01:37:56, static
192.168.129.0/24, ubest/mbest: 1/0, attached
    *via 192.168.129.67, mgmt0, [0/0], 01:37:57, direct
192.168.129.67/32, ubest/mbest: 1/0, attached
    *via 192.168.129.67, mgmt0, [0/0], 01:37:57, local

IP Route Table for VRF "egress-loadbalance-resolution-"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

```

</div></details>

<details><summary>　f2-bdsw02 </summary><div>

```
f2-bdsw02# show nve peers
Interface Peer-IP                                 State LearnType Uptime   Router-Mac       
--------- --------------------------------------  ----- --------- -------- -----------------
nve1      10.0.2.2                                Up    CP        01:35:30 n/a              

f2-bdsw02# show nve vni
Codes: CP - Control Plane        DP - Data Plane          
       UC - Unconfigured         SA - Suppress ARP        
       S-ND - Suppress ND        
       SU - Suppress Unknown Unicast 
       Xconn - Crossconnect      
       MS-IR - Multisite Ingress Replication 
       HYB - Hybrid IRB mode
    
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      20100    UnicastBGP        Up    CP   L2 [2010]               

f2-bdsw02# show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.0.0.2, local AS number 65001
BGP table version is 20, L2VPN EVPN config peers 2, capable peers 1
10 network entries and 10 paths using 2808 bytes of memory
BGP attribute entries [7/2576], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [1/4]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.0.253      4     0          0          0        0    0    0 01:38:50 Idle     
10.0.0.254      4 65001        111        102       20    0    0 01:37:07 2         

Neighbor        T    AS PfxRcd     Type-2     Type-3     Type-4     Type-5     Type-12   
10.0.0.253      I     0 Idle       0          0          0          0          0         
10.0.0.254      I 65001 2          1          1          0          0          0         
f2-bdsw02# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 20, Local Router ID is 10.0.0.2
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 10.0.0.2:34777    (L2VNI 20100)
*>l[2]:[0]:[0]:[48]:[001c.7300.0999]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5254.0064.9ead]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5254.007a.0e9c]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[5275.d033.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>l[2]:[0]:[0]:[48]:[52cd.4475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100      32768 i
*>i[2]:[0]:[0]:[48]:[7ac1.d895.b089]:[0]:[0.0.0.0]/216
                      10.0.2.2                          100          0 i
*>l[3]:[0]:[32]:[10.0.2.1]/88
                      10.0.2.1                          100      32768 i
*>i[3]:[0]:[32]:[10.0.2.2]/88
                      10.0.2.2                          100          0 i

Route Distinguisher: 10.0.0.3:32867
*>i[2]:[0]:[0]:[48]:[7ac1.d895.b089]:[0]:[0.0.0.0]/216
                      10.0.2.2                          100          0 i
*>i[3]:[0]:[32]:[10.0.2.2]/88
                      10.0.2.2                          100          0 i

f2-bdsw02# show l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole, (Dum):Dummy
(Pf):Permanently-Frozen, (Orp): Orphan

(PipOrp): Directly connected Orphan to PIP based vPC BGW 
(PipPeerOrp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Prod   Flags              Seq No     Next-Hops                              
----------- -------------- ------ ------------------ ---------- ---------------------------------------------------------
2010        001c.7300.0999 Local  L,                 0          Po11
2010        5254.0064.9ead Local  L,                 0          Po11
2010        5254.007a.0e9c Local  L,                 0          Po11
2010        5275.d033.1b08 VXLAN  Stt,Nho,Dum,       0          10.0.2.1
2010        52cd.4475.1b08 Local  L,                 0          Po11
2010        7ac1.d895.b089 BGP    Rcv                0          10.0.2.2 (Label: 20100)
f2-bdsw02# show l2route evpn mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated (Orp):Orphan (Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole
(Piporp): Directly connected Orphan to PIP based vPC BGW 
(Pipporp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Host IP                                 Prod   Flags              Seq No     Next-Hops                              
----------- -------------- --------------------------------------- ------ ----------------- ---------- ----------------------------------------------------
-----
f2-bdsw02# show vpc brief
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1   
Peer status                       : peer adjacency formed ok      
vPC keep-alive status             : peer is alive                 
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : secondary                     
Number of vPCs configured         : 1   
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Disabled
Delay-restore status              : Timer is off.(timeout = 30s)
Delay-restore SVI status          : Timer is off.(timeout = 10s)
Delay-restore Orphan-port status  : Timer is off.(timeout = 0s)
Operational Layer3 Peer-router    : Disabled
Virtual-peerlink mode             : Enabled

vPC Peer-link status
---------------------------------------------------------------------
id    Port   Status Active vlans    
--    ----   ------ -------------------------------------------------
1     Po1    up     1,2010                                                               

vPC status
----------------------------------------------------------------------------
Id    Port          Status Consistency Reason                Active vlans
--    ------------  ------ ----------- ------                ---------------
11    Po11          up     success     success               2010                        
                                                                                         

Please check "show vpc consistency-parameters vpc <vpc-num>" for the 
consistency reason of down vpc and for type-2 consistency reasons for 
any vpc.

f2-bdsw02# show ip route vrf all
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.0.0.1/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/81], 01:37:36, ospf-1, intra
10.0.0.2/32, ubest/mbest: 2/0, attached
    *via 10.0.0.2, Lo0, [0/0], 01:38:59, local
    *via 10.0.0.2, Lo0, [0/0], 01:38:59, direct
10.0.0.3/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/81], 01:37:36, ospf-1, intra
10.0.0.254/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/41], 01:37:36, ospf-1, intra
10.0.1.1/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/81], 01:34:32, ospf-1, intra
10.0.1.2/32, ubest/mbest: 2/0, attached
    *via 10.0.1.2, Lo1, [0/0], 01:34:32, local
    *via 10.0.1.2, Lo1, [0/0], 01:34:32, direct
10.0.1.3/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/81], 01:37:36, ospf-1, intra
10.0.2.1/32, ubest/mbest: 2/0, attached
    *via 10.0.2.1, Lo1, [0/0], 01:34:32, local
    *via 10.0.2.1, Lo1, [0/0], 01:34:32, direct
10.0.2.2/32, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/81], 01:37:36, ospf-1, intra
10.0.3.0/31, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/80], 01:37:36, ospf-1, intra
10.0.3.2/31, ubest/mbest: 1/0, attached
    *via 10.0.3.2, Eth1/4, [0/0], 01:37:49, direct
10.0.3.2/32, ubest/mbest: 1/0, attached
    *via 10.0.3.2, Eth1/4, [0/0], 01:37:49, local
10.0.3.4/31, ubest/mbest: 1/0
    *via 10.0.3.3, Eth1/4, [110/80], 01:37:36, ospf-1, intra

IP Route Table for VRF "management"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

0.0.0.0/0, ubest/mbest: 1/0
    *via 192.168.129.254, [1/0], 01:38:59, static
192.168.129.0/24, ubest/mbest: 1/0, attached
    *via 192.168.129.68, mgmt0, [0/0], 01:39:00, direct
192.168.129.68/32, ubest/mbest: 1/0, attached
    *via 192.168.129.68, mgmt0, [0/0], 01:39:00, local

IP Route Table for VRF "egress-loadbalance-resolution-"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

```

</div></details>

<details><summary>　f2-lfsw01　</summary><div>

```
f2-lfsw01# show nve peers
Interface Peer-IP                                 State LearnType Uptime   Router-Mac       
--------- --------------------------------------  ----- --------- -------- -----------------
nve1      10.0.2.1                                Up    CP        01:35:01 n/a              

f2-lfsw01# show nve vni
Codes: CP - Control Plane        DP - Data Plane          
       UC - Unconfigured         SA - Suppress ARP        
       S-ND - Suppress ND        
       SU - Suppress Unknown Unicast 
       Xconn - Crossconnect      
       MS-IR - Multisite Ingress Replication 
       HYB - Hybrid IRB mode
    
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      20100    UnicastBGP        Up    CP   L2 [100]                

f2-lfsw01# show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.0.0.3, local AS number 65001
BGP table version is 52, L2VPN EVPN config peers 2, capable peers 1
21 network entries and 26 paths using 5628 bytes of memory
BGP attribute entries [14/5152], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [2/8]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.0.253      4     0          0          0        0    0    0 01:39:58 Idle     
10.0.0.254      4 65001        118        103       52    0    0 01:38:25 12        

Neighbor        T    AS PfxRcd     Type-2     Type-3     Type-4     Type-5     Type-12   
10.0.0.253      I     0 Idle       0          0          0          0          0         
10.0.0.254      I 65001 12         10         2          0          0          0         
f2-lfsw01# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 52, Local Router ID is 10.0.0.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 10.0.0.1:34777
*>i[2]:[0]:[0]:[48]:[001c.7300.0999]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5254.0064.9ead]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5254.007a.0e9c]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5267.5475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[52cd.4475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[3]:[0]:[32]:[10.0.2.1]/88
                      10.0.2.1                          100          0 i

Route Distinguisher: 10.0.0.2:34777
*>i[2]:[0]:[0]:[48]:[001c.7300.0999]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5254.0064.9ead]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5254.007a.0e9c]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5275.d033.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[52cd.4475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[3]:[0]:[32]:[10.0.2.1]/88
                      10.0.2.1                          100          0 i

Route Distinguisher: 10.0.0.3:32867    (L2VNI 20100)
*>i[2]:[0]:[0]:[48]:[001c.7300.0999]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
* i                   10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5254.0064.9ead]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
* i                   10.0.2.1                          100          0 i
* i[2]:[0]:[0]:[48]:[5254.007a.0e9c]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i                   10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5267.5475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[5275.d033.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
*>i[2]:[0]:[0]:[48]:[52cd.4475.1b08]:[0]:[0.0.0.0]/216
                      10.0.2.1                          100          0 i
* i                   10.0.2.1                          100          0 i
*>l[2]:[0]:[0]:[48]:[7ac1.d895.b089]:[0]:[0.0.0.0]/216
                      10.0.2.2                          100      32768 i
*>i[3]:[0]:[32]:[10.0.2.1]/88
                      10.0.2.1                          100          0 i
* i                   10.0.2.1                          100          0 i
*>l[3]:[0]:[32]:[10.0.2.2]/88
                      10.0.2.2                          100      32768 i

f2-lfsw01# show l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole, (Dum):Dummy
(Pf):Permanently-Frozen, (Orp): Orphan

(PipOrp): Directly connected Orphan to PIP based vPC BGW 
(PipPeerOrp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Prod   Flags              Seq No     Next-Hops                              
----------- -------------- ------ ------------------ ---------- ---------------------------------------------------------
100         001c.7300.0999 BGP    Rcv                0          10.0.2.1 (Label: 20100)
100         5254.0064.9ead BGP    Rcv                0          10.0.2.1 (Label: 20100)
100         5254.007a.0e9c BGP    Rcv                0          10.0.2.1 (Label: 20100)
100         52cd.4475.1b08 BGP    Rcv                0          10.0.2.1 (Label: 20100)
100         7ac1.d895.b089 Local  L,                 0          Po11
f2-lfsw01# show l2route evpn mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote 
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated (Orp):Orphan (Asy):Asymmetric (Gw):Gateway
(Bh):Blackhole
(Piporp): Directly connected Orphan to PIP based vPC BGW 
(Pipporp): Orphan connected to peer of PIP based vPC BGW 
Topology    Mac Address    Host IP                                 Prod   Flags              Seq No     Next-Hops                              
----------- -------------- --------------------------------------- ------ ----------------- ---------- ----------------------------------------------------
-----
f2-lfsw01# show vpc brief
                    ^
% Invalid command at '^' marker.
f2-lfsw01# show ip route vrf all
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.0.0.1/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/81], 01:39:31, ospf-1, intra
10.0.0.2/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/81], 01:39:21, ospf-1, intra
10.0.0.3/32, ubest/mbest: 2/0, attached
    *via 10.0.0.3, Lo0, [0/0], 01:40:51, local
    *via 10.0.0.3, Lo0, [0/0], 01:40:51, direct
10.0.0.254/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/41], 01:39:32, ospf-1, intra
10.0.1.1/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/81], 01:36:16, ospf-1, intra
10.0.1.2/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/81], 01:36:16, ospf-1, intra
10.0.1.3/32, ubest/mbest: 2/0, attached
    *via 10.0.1.3, Lo1, [0/0], 01:41:00, local
    *via 10.0.1.3, Lo1, [0/0], 01:41:00, direct
10.0.2.1/32, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/81], 01:36:16, ospf-1, intra
10.0.2.2/32, ubest/mbest: 2/0, attached
    *via 10.0.2.2, Lo1, [0/0], 01:41:00, local
    *via 10.0.2.2, Lo1, [0/0], 01:41:00, direct
10.0.3.0/31, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/80], 01:39:32, ospf-1, intra
10.0.3.2/31, ubest/mbest: 1/0
    *via 10.0.3.5, Eth1/4, [110/80], 01:39:32, ospf-1, intra
10.0.3.4/31, ubest/mbest: 1/0, attached
    *via 10.0.3.4, Eth1/4, [0/0], 01:39:39, direct
10.0.3.4/32, ubest/mbest: 1/0, attached
    *via 10.0.3.4, Eth1/4, [0/0], 01:39:39, local

IP Route Table for VRF "management"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

0.0.0.0/0, ubest/mbest: 1/0
    *via 192.168.129.254, [1/0], 01:40:52, static
192.168.129.0/24, ubest/mbest: 1/0, attached
    *via 192.168.129.69, mgmt0, [0/0], 01:40:53, direct
192.168.129.69/32, ubest/mbest: 1/0, attached
    *via 192.168.129.69, mgmt0, [0/0], 01:40:53, local

IP Route Table for VRF "egress-loadbalance-resolution-"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

```

</div></details>


今回で確認した内容は以上

## おわりに

CML で vEOS を使用した検証ができることを確認できた

簡易的なところまでしか見られてなく、動作試験などどこまでできるからはこれからになる

様々なメーカのネットワーク機器の接続試験も行えそう ([参考 CiscoDevNet  cml commnunity node-definitions](https://github.com/CiscoDevNet/cml-community/tree/master/node-definitions))

CML のノード定義設定などがまだ勉強不足で、config を CML の YAML に保存ができてないので、config 保存設定などもできれば試していきたい

## 参考

https://github.com/CiscoDevNet/cml-community/tree/master/node-definitions

https://docs.openstack.org/ja/image-guide/convert-images.html

https://arista.my.site.com/AristaCommunity/s/article/Validated-Configuration-Layer-3-Leaf-Spine-with-VXLAN-EVPN-Symmetric-IRB

https://changineer.info/network/arista_eos/arista_eos_mlag_basic.html