Terraform で NX-OS (Nexus 9K) に EVPN+VXLAN Overlay 追加・削除を試す

## はじめに

Cisco は NX-OS 向けに Terraform Module を提供している ([NX-OS Module](https://registry.terraform.io/providers/CiscoDevNet/nxos/latest/docs
))

Terraform は宣言的な管理が可能なため Kubernetes のような Code = インフラ状態が表現・管理しやすく、 より IaC として管理がしやすい

全体イメージ
![スクリーンショット 2025-03-02 16.14.37.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/30b66d0f-807e-4050-924c-b8edfbccdf10.png)

※ 今回は tfstate はローカルファイルとして実行するコードになっている (backend 設定はしてない)

今回 EVPN+VXLAN の既存環境に対して、オーバレイの追加作業および切り戻し作業の自動化として Terraform 実装を試したので、その内容を記載する

## 実施環境

自宅ラボにある CML での検証環境で試験している

### CML Version

CML-Personal 2.8.1 (2025.03.02 時点最新)

### サーバスペック

|    項目    |                         スペック                          |
| :--------: | :------------------------------------------------------: |
|   サーバ   |　                       NUC14RVHI7                        |
|    CPU     | Intel Core Ultra 7 155H <br> 16Core(P6/E8/LPE2)/22Thread (CML 割り当て 20vCPU)|
|   Memory   |        96GiB (DDR5-5600 48GiB x2) (CML 割り当て 92GiB)    |
|    Disk    |            1TB PCIe Gen4 (CML 割り当て 512GB)             |
|  拡張 NIC  |          USB NIC 1G x1 <br> (LUA5-U3-AGTE-NBK)           |
| ホスト OS |                     Ubuntu24.04 LTS                      |

### EVPN+VXLAN 設定

下記リンクの記事がベースとなっている (今回追加するところは事前に削除してから試験は始めている)

https://qiita.com/suzuyui/items/665507e992becf682f5a

![スクリーンショット 2025-03-02 22.29.26.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/bacafd3e-9ab6-4254-8a17-fd2d2d05363a.png)

※各機器の Mgmt0 は CML 外の NW に直接続することで、Terraform サーバと直接通信を可能にしている

### Terraform Version

Terraform : v1.10.5

nxos provider : [Version 0.5.8](https://registry.terraform.io/providers/CiscoDevNet/nxos/0.5.8/docs) (2025.03.02 時点最新)


## Terraform 実装

Code は長くなったので下記 GitHub に公開している

https://github.com/suzuyu/nxos-terraform/tree/v1.0.0

構成は複数に分けている

```sh
.
|-- README.md
|-- env
|   `-- cml
|       |-- README.md
|       |-- cml_yaml
|       |   `-- leaf_spine
|       |       |-- Leaf-Spine-Test3_VNI_After.yaml
|       |       `-- Leaf-Spine-Test3_VNI_Before.yaml
|       `-- vni
|           |-- README.md
|           |-- main.tf
|           |-- terraform.tfvars
|           |-- variables.tf
|           `-- vni.tf
`-- modules
    |-- nxos_l2vni
    |   |-- README.md
    |   |-- main.tf
    |   `-- variables.tf
    `-- nxos_l3vni
        |-- README.md
        |-- main.tf
        `-- variables.tf
```

- `modules`
  - 今回設定する L2VNI, L3VNI を設定するコードを共通化できるところを分離したコード
  - 変数を定義して `env/cml/vni/vni.tf` で呼び出して使用している
- `env/cml/cml_yaml/Leaf-Spine-Test3_VNI_Before.yaml`
  - 今回実施する事前構築状態の CML 設定をダウンロードした YAML
- `env/cml/vni`
  - 今回 terraform 操作を実施するフォルダ
  - `terraform.tfvars`
    - 今回設定するパラメータを定義して、今後オーバレイを追加する際は基本的にこの変数を変更するだけで追加削除ができる
  - `main.tf`
    - provider や機器の IP アドレスやパスワードを定義している
  - `vni.tf`
    - modules で定義している実行コードを、変数を代入して呼び出ししている

## Terraform 動作試験

実際に上記 Terraform コードでオーバレイを追加する動作試験を実施し、ログを記載する

### 概要

下記のような構成で、アンダーレイ・サーバ構築済みの構成([`env/cml/cml_yaml/Leaf-Spine-Test3_VNI_Before.yaml`](https://github.com/suzuyu/nxos-terraform/blob/v1.0.0/env/cml/cml_yaml/leaf_spine/Leaf-Spine-Test3_VNI_Before.yaml) の設定状態)で VNI の追加作業を実施する

![スクリーンショット 2025-03-02 16.21.15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/8969b88f-dddf-4c51-8a19-ee5243e5e941.png)

各機器の設定 Config は下記の通り

<details><summary>　lfsw01 before config　</summary><div>

```config
lfsw01# show run

!Command: show running-config
!Running configuration last done at: Sun Mar  2 20:59:58 2025
!Time: Sun Mar  2 21:00:28 2025

version 10.5(1) Bios:version  
hostname lfsw01
vdc lfsw01 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs ipv4 distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$PJAMHI$W4Ih7NfB87rKZatB0F3LTQtWPHkZK4uhH6Byb767Oj5  role network-admin
username cisco password 5 $5$FCPIIG$RRagIkHJeow6kxDTamd2Xsp4TUkeGCRQeVdz0pHwlD0  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
bfd startup-timer 0
snmp-server user admin network-admin auth md5 3328CD1117679702547B2F13039654E3AC6A priv aes-128 0153CA151511AC3C1C4A5A330C9253ACA93D localizedV2key
snmp-server user cisco network-admin auth md5 056E98403323A677350C5F675BC31EC5F735 priv aes-128 33199E4E4743BF3E6A11426643C317B4A667 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,100-101,1001,3001
vlan 100
  name tenant1-server-seg1
  vn-segment 10100
vlan 101
  name tenant1-vlan101
  vn-segment 10101
vlan 1001
  name tenant3-nw-vlan300
  vn-segment 30300
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vpc domain 1
  role priority 100
  peer-keepalive destination 192.168.129.52 source 192.168.129.51
  virtual peer-link destination 10.0.0.2 source 10.0.0.1 dscp 56


interface Vlan1

interface Vlan100
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan101
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface port-channel1
  description virtual-vpc-peer-link
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface port-channel11
  description tenant1-server01 bond0
  switchport mode trunk
  vpc 11

interface port-channel12
  description tenant2-server01 bond0
  switchport mode trunk
  vpc 12

interface port-channel13
  description tenant3-server01 bond0
  switchport mode trunk
  vpc 13

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf
  member vni 30300
    ingress-replication protocol bgp

interface Ethernet1/1
  description tenant1-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 11 mode active

interface Ethernet1/2
  description tenant2-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 12 mode active

interface Ethernet1/3
  description tenant3-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 13 mode active

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/1
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.4.0/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/1
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.3.0/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.51/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.1/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.1/32
  ip address 10.0.2.1/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  bfd
  router-id 10.0.0.1
router bgp 65001
  router-id 10.0.0.1
  address-family l2vpn evpn
    advertise-pip
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30300 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console

```
</div></details>


<details><summary>　lfsw02 before config　</summary><div>

```config:lfsw02
lfsw02# show run 

!Command: show running-config
!Running configuration last done at: Sun Mar  2 20:59:46 2025
!Time: Sun Mar  2 21:03:38 2025

version 10.5(1) Bios:version  
hostname lfsw02
vdc lfsw02 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs ipv4 distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$MIABHO$I4vvga9QpydmKdK1iLAnJtqJx/sFAUrWABoY6cEgBr.  role network-admin
username cisco password 5 $5$GMELDH$8v4a2WA5YuNub3XWE9/EKcgcIZpXPNPSGyubE/r0eLA  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 00514DE2E2F0FAE3138880548031669F521C priv aes-128 323C6DD1DD1B212C9C760EC557EEA547FEF8 localizedV2key
snmp-server user cisco network-admin auth md5 37427EC18B623A31C73A5FD91EE3AE1B83D7 priv aes-128 482F13A7F6505F4B96351E8D11EE8403C2D5 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,100-101,1001,3001
vlan 100
  name tenant1-server-seg1
  vn-segment 10100
vlan 101
  name tenant1-vlan101
  vn-segment 10101
vlan 1001
  name tenant3-nw-vlan300
  vn-segment 30300
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vpc domain 1
  role priority 200
  peer-keepalive destination 192.168.129.51 source 192.168.129.52
  virtual peer-link destination 10.0.0.1 source 10.0.0.2 dscp 56


interface Vlan1

interface Vlan100
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan101
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface port-channel1
  description virtual-vpc-peer-link
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface port-channel11
  description tenant1-server01 bond0
  switchport mode trunk
  vpc 11

interface port-channel12
  description tenant2-server01 bond0
  switchport mode trunk
  vpc 12

interface port-channel13
  description tenant3-server01 bond0
  switchport mode trunk
  vpc 13

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf
  member vni 30300
    ingress-replication protocol bgp

interface Ethernet1/1
  description tenant1-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 11 mode active

interface Ethernet1/2
  description tenant2-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 12 mode active

interface Ethernet1/3
  description tenant3-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 13 mode active

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/2
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.4.2/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/2
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.3.2/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.52/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.2/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.2/32
  ip address 10.0.2.1/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  router-id 10.0.0.2
router bgp 65001
  router-id 10.0.0.2
  address-family l2vpn evpn
    advertise-pip
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30300 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console


```

</div></details>


<details><summary>　lfsw03 before config　</summary><div>

```config:lfsw03
lfsw03# show run

!Command: show running-config
!Running configuration last done at: Sun Mar  2 20:59:51 2025
!Time: Sun Mar  2 21:04:33 2025

version 10.5(1) Bios:version  
hostname lfsw03
vdc lfsw03 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$DCALLD$97LM57TD1qBqvKxSPC.zrFWkwLb.mcI3oelSIY2ezK6  role network-admin
username cisco password 5 $5$PLGLCI$KPPOvBPkC9lKtmwVpmhHOPkkAMuMSqfWf2Lj7yVmBX0  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 4976440ADE75ACA80D512BEB0C28C2391689 priv aes-128 174C152CF52BB2BA0C0D5F90515ECB771395 localizedV2key
snmp-server user cisco network-admin auth md5 53150C1621F12906B9B08951D9C85FB8D557 priv aes-128 01462E3AB6CF0D1A9FEAC459969410F98A55 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,10-11,3001
vlan 10
  name tenant1-server-seg1
  vn-segment 10100
vlan 11
  name tenant1-server-seg2
  vn-segment 10101
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast


interface Vlan1

interface Vlan10
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan11
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf

interface Ethernet1/1
  description tenant1-server02 eth0
  switchport mode trunk

interface Ethernet1/2
  description tenant1-server03 eth0
  switchport mode trunk

interface Ethernet1/3
  description tenant2-server02 eth0
  switchport mode trunk

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/3
  no switchport
  mtu 9216
  ip address 10.0.4.4/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/3
  no switchport
  mtu 9216
  ip address 10.0.3.4/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.53/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.3/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.3/32
  ip address 10.0.2.2/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  router-id 10.0.0.3
router bgp 65001
  router-id 10.0.0.3
  address-family l2vpn evpn
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console


```

</div></details>

### 追加作業

下記パラメータを定義して実施する

- l3vni_map
  - L3VNI の設定を記載する
  - 今回は VNI 29001 のみだが、VNI ごとに複数記載可能
  - members に同様の設定をする leaf のホスト名をキーにして、ローカルの VLAN ID を記載する
- l2vni_map
  - L2VNI の設定を記載する
  - l3vni_map とほぼ同じ
  - ルーティングする場合(L3VNIと紐付け)`gateway_ip`も記載する(option)

```config:terraform.tfvars
l3vni_map = {
  vni_29001 = {
    vni = 29001
    vrf = "tenant2-vpc1"
    members = {
      lfsw01 = {
        vlan = "3002"
      }
      lfsw02 = {
        vlan = "3002"
      }
      lfsw03 = {
        vlan = "3002"
      }
    }
  }
}

l2vni_map = {
  vni_20200 = {
    vni = 20200
    vrf = "tenant2-vpc1"

    segment_name = "server-seg1"
    gateway_ip   = "172.17.0.254/24"
    members = {
      lfsw01 = {
        vlan = "200"
      }
      lfsw02 = {
        vlan = "200"
      }
      lfsw03 = {
        vlan = "20"
      }
    }
  }
}
```

上記パラメータ記載して、`terraform apply` を実施する

```sh
cd nxos-terraform/env/cml/vni/
terraform apply
```

上記で実施した動作を GIF にしたものが下記の通り

- CML にアクセス
- CML 上でサーバが Leaf を通して別のサーバに Ping が通らないことを確認する
- 各 Leaf で `show run diff` をして差分がないことを確認する (設定後に差分を見るため)
- VS Code でコードをざっとみて、`terraform apply` をして設定を投入
- 各機器で `show run diff` を実施して設定が投入されていることを確認する
- サーバで Ping 疎通ができるようになったことを確認する

![terraform_VNI追加_SD1.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/d7e571ea-ec6b-405d-8454-afa5f6fbeacb.gif)

上記の通り terraform で各機器に VLAN や VXLAN 設定が投入できたことが確認できた

各ログを下記に折りたたんで記載する

<details><summary>　terraform apply log　</summary><div>

注記： 最初に "nxos_save_config" が複数出ているのは実施後に `show run diff` を実施して差分を見れるように一時的にコメントアウトしているため

```sh
% terraform apply
nxos_save_config.main["lfsw03"]: Refreshing state...
nxos_save_config.main["lfsw02"]: Refreshing state...
nxos_save_config.main["lfsw01"]: Refreshing state...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  - destroy

Terraform will perform the following actions:

  # nxos_save_config.main["lfsw01"] will be destroyed
  # (because nxos_save_config.main is not in configuration)
  - resource "nxos_save_config" "main" {
      - device = "lfsw01" -> null
      - save   = true -> null
    }

  # nxos_save_config.main["lfsw02"] will be destroyed
  # (because nxos_save_config.main is not in configuration)
  - resource "nxos_save_config" "main" {
      - device = "lfsw02" -> null
      - save   = true -> null
    }

  # nxos_save_config.main["lfsw03"] will be destroyed
  # (because nxos_save_config.main is not in configuration)
  - resource "nxos_save_config" "main" {
      - device = "lfsw03" -> null
      - save   = true -> null
    }

  # module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw01"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-20200"
      + device       = "lfsw01"
      + fabric_encap = "vlan-200"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-server-seg1"
    }

  # module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw02"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-20200"
      + device       = "lfsw02"
      + fabric_encap = "vlan-200"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-server-seg1"
    }

  # module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw03"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-20200"
      + device       = "lfsw03"
      + fabric_encap = "vlan-20"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-server-seg1"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw01"] will be created
  + resource "nxos_evpn_vni" "main" {
      + device              = "lfsw01"
      + encap               = "vxlan-20200"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw02"] will be created
  + resource "nxos_evpn_vni" "main" {
      + device              = "lfsw02"
      + encap               = "vxlan-20200"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw03"] will be created
  + resource "nxos_evpn_vni" "main" {
      + device              = "lfsw03"
      + encap               = "vxlan-20200"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw01"] will be created
  + resource "nxos_evpn_vni_route_target" "export" {
      + device       = "lfsw01"
      + direction    = "export"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw02"] will be created
  + resource "nxos_evpn_vni_route_target" "export" {
      + device       = "lfsw02"
      + direction    = "export"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw03"] will be created
  + resource "nxos_evpn_vni_route_target" "export" {
      + device       = "lfsw03"
      + direction    = "export"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw01"] will be created
  + resource "nxos_evpn_vni_route_target" "import" {
      + device       = "lfsw01"
      + direction    = "import"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw02"] will be created
  + resource "nxos_evpn_vni_route_target" "import" {
      + device       = "lfsw02"
      + direction    = "import"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw03"] will be created
  + resource "nxos_evpn_vni_route_target" "import" {
      + device       = "lfsw03"
      + direction    = "import"
      + encap        = "vxlan-20200"
      + id           = (known after apply)
      + route_target = "route-target:unknown:0:0"
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw01"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "export" {
      + device    = "lfsw01"
      + direction = "export"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw02"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "export" {
      + device    = "lfsw02"
      + direction = "export"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw03"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "export" {
      + device    = "lfsw03"
      + direction = "export"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw01"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "import" {
      + device    = "lfsw01"
      + direction = "import"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw02"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "import" {
      + device    = "lfsw02"
      + direction = "import"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw03"] will be created
  + resource "nxos_evpn_vni_route_target_direction" "import" {
      + device    = "lfsw03"
      + direction = "import"
      + encap     = "vxlan-20200"
      + id        = (known after apply)
    }

  # module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw01"] will be created
  + resource "nxos_hmm_interface" "main" {
      + admin_state  = "enabled"
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + mode         = "anycastGW"
    }

  # module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw02"] will be created
  + resource "nxos_hmm_interface" "main" {
      + admin_state  = "enabled"
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + mode         = "anycastGW"
    }

  # module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw03"] will be created
  + resource "nxos_hmm_interface" "main" {
      + admin_state  = "enabled"
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan20"
      + mode         = "anycastGW"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw01"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw01"
      + drop_glean   = "disabled"
      + forward      = "disabled"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw02"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw02"
      + drop_glean   = "disabled"
      + forward      = "disabled"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw03"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw03"
      + drop_glean   = "disabled"
      + forward      = "disabled"
      + id           = (known after apply)
      + interface_id = "vlan20"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw01"] will be created
  + resource "nxos_ipv4_interface_address" "main" {
      + address      = "172.17.0.254/24"
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + tag          = 0
      + type         = "primary"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw02"] will be created
  + resource "nxos_ipv4_interface_address" "main" {
      + address      = "172.17.0.254/24"
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + tag          = 0
      + type         = "primary"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw03"] will be created
  + resource "nxos_ipv4_interface_address" "main" {
      + address      = "172.17.0.254/24"
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan20"
      + tag          = 0
      + type         = "primary"
      + vrf          = "tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw01"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = false
      + device                        = "lfsw01"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 20200
    }

  # module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw02"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = false
      + device                        = "lfsw02"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 20200
    }

  # module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw03"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = false
      + device                        = "lfsw03"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 20200
    }

  # module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw01"] will be created
  + resource "nxos_nve_vni_ingress_replication" "main" {
      + device   = "lfsw01"
      + id       = (known after apply)
      + protocol = "bgp"
      + vni      = 20200
    }

  # module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw02"] will be created
  + resource "nxos_nve_vni_ingress_replication" "main" {
      + device   = "lfsw02"
      + id       = (known after apply)
      + protocol = "bgp"
      + vni      = 20200
    }

  # module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw03"] will be created
  + resource "nxos_nve_vni_ingress_replication" "main" {
      + device   = "lfsw03"
      + id       = (known after apply)
      + protocol = "bgp"
      + vni      = 20200
    }

  # module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw01"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1-server-seg1"
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw02"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1-server-seg1"
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw03"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1-server-seg1"
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan20"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw01"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw02"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan200"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw03"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan20"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw01"] will be created
  + resource "nxos_bgp_address_family" "main" {
      + address_family                         = "ipv4-ucast"
      + advertise_l2vpn_evpn                   = "disabled"
      + advertise_only_active_routes           = "disabled"
      + advertise_physical_ip_for_type5_routes = "disabled"
      + asn                                    = "65001"
      + critical_nexthop_timeout               = "crit"
      + default_information_originate          = "disabled"
      + device                                 = "lfsw01"
      + id                                     = (known after apply)
      + max_ecmp_paths                         = 1
      + max_external_ecmp_paths                = 1
      + max_external_internal_ecmp_paths       = 1
      + max_local_ecmp_paths                   = 1
      + max_mixed_ecmp_paths                   = 1
      + non_critical_nexthop_timeout           = "noncrit"
      + prefix_priority                        = "none"
      + retain_rt_all                          = "disabled"
      + vni_ethernet_tag                       = "disabled"
      + vrf                                    = "tenant2-vpc1"
      + wait_igp_converged                     = "disabled"
    }

  # module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw02"] will be created
  + resource "nxos_bgp_address_family" "main" {
      + address_family                         = "ipv4-ucast"
      + advertise_l2vpn_evpn                   = "disabled"
      + advertise_only_active_routes           = "disabled"
      + advertise_physical_ip_for_type5_routes = "disabled"
      + asn                                    = "65001"
      + critical_nexthop_timeout               = "crit"
      + default_information_originate          = "disabled"
      + device                                 = "lfsw02"
      + id                                     = (known after apply)
      + max_ecmp_paths                         = 1
      + max_external_ecmp_paths                = 1
      + max_external_internal_ecmp_paths       = 1
      + max_local_ecmp_paths                   = 1
      + max_mixed_ecmp_paths                   = 1
      + non_critical_nexthop_timeout           = "noncrit"
      + prefix_priority                        = "none"
      + retain_rt_all                          = "disabled"
      + vni_ethernet_tag                       = "disabled"
      + vrf                                    = "tenant2-vpc1"
      + wait_igp_converged                     = "disabled"
    }

  # module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw03"] will be created
  + resource "nxos_bgp_address_family" "main" {
      + address_family                         = "ipv4-ucast"
      + advertise_l2vpn_evpn                   = "disabled"
      + advertise_only_active_routes           = "disabled"
      + advertise_physical_ip_for_type5_routes = "disabled"
      + asn                                    = "65001"
      + critical_nexthop_timeout               = "crit"
      + default_information_originate          = "disabled"
      + device                                 = "lfsw03"
      + id                                     = (known after apply)
      + max_ecmp_paths                         = 1
      + max_external_ecmp_paths                = 1
      + max_external_internal_ecmp_paths       = 1
      + max_local_ecmp_paths                   = 1
      + max_mixed_ecmp_paths                   = 1
      + non_critical_nexthop_timeout           = "noncrit"
      + prefix_priority                        = "none"
      + retain_rt_all                          = "disabled"
      + vni_ethernet_tag                       = "disabled"
      + vrf                                    = "tenant2-vpc1"
      + wait_igp_converged                     = "disabled"
    }

  # module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw01"] will be created
  + resource "nxos_bgp_route_redistribution" "main" {
      + address_family    = "ipv4-ucast"
      + asn               = "65001"
      + device            = "lfsw01"
      + id                = (known after apply)
      + protocol          = "direct"
      + protocol_instance = "none"
      + route_map         = "permit-all-v4"
      + scope             = "inter"
      + srv6_prefix_type  = "unspecified"
      + vrf               = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw02"] will be created
  + resource "nxos_bgp_route_redistribution" "main" {
      + address_family    = "ipv4-ucast"
      + asn               = "65001"
      + device            = "lfsw02"
      + id                = (known after apply)
      + protocol          = "direct"
      + protocol_instance = "none"
      + route_map         = "permit-all-v4"
      + scope             = "inter"
      + srv6_prefix_type  = "unspecified"
      + vrf               = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw03"] will be created
  + resource "nxos_bgp_route_redistribution" "main" {
      + address_family    = "ipv4-ucast"
      + asn               = "65001"
      + device            = "lfsw03"
      + id                = (known after apply)
      + protocol          = "direct"
      + protocol_instance = "none"
      + route_map         = "permit-all-v4"
      + scope             = "inter"
      + srv6_prefix_type  = "unspecified"
      + vrf               = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw01"] will be created
  + resource "nxos_bgp_vrf" "main" {
      + asn    = "65001"
      + device = "lfsw01"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw02"] will be created
  + resource "nxos_bgp_vrf" "main" {
      + asn    = "65001"
      + device = "lfsw02"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw03"] will be created
  + resource "nxos_bgp_vrf" "main" {
      + asn    = "65001"
      + device = "lfsw03"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw01"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-29001"
      + device       = "lfsw01"
      + fabric_encap = "vlan-3002"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-l3vni"
    }

  # module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw02"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-29001"
      + device       = "lfsw02"
      + fabric_encap = "vlan-3002"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-l3vni"
    }

  # module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw03"] will be created
  + resource "nxos_bridge_domain" "main" {
      + access_encap = "vxlan-29001"
      + device       = "lfsw03"
      + fabric_encap = "vlan-3002"
      + id           = (known after apply)
      + name         = "tenant2-vpc1-l3vni"
    }

  # module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw01"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw01"
      + drop_glean   = "disabled"
      + forward      = "enabled"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw02"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw02"
      + drop_glean   = "disabled"
      + forward      = "enabled"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw03"] will be created
  + resource "nxos_ipv4_interface" "main" {
      + device       = "lfsw03"
      + drop_glean   = "disabled"
      + forward      = "enabled"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + unnumbered   = "unspecified"
      + urpf         = "disabled"
      + vrf          = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw01"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = true
      + device                        = "lfsw01"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 29001
    }

  # module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw02"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = true
      + device                        = "lfsw02"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 29001
    }

  # module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw03"] will be created
  + resource "nxos_nve_vni" "main" {
      + associate_vrf                 = true
      + device                        = "lfsw03"
      + id                            = (known after apply)
      + multicast_group               = "0.0.0.0"
      + multisite_ingress_replication = "disable"
      + suppress_arp                  = "off"
      + vni                           = 29001
    }

  # module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw01"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1"
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw02"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1"
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw03"] will be created
  + resource "nxos_svi_interface" "main" {
      + admin_state  = "up"
      + bandwidth    = 1000000
      + delay        = 1
      + description  = "tenant2-vpc1"
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + medium       = "bcast"
      + mtu          = 1500
    }

  # module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw01"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw01"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw02"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw02"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw03"] will be created
  + resource "nxos_svi_interface_vrf" "main" {
      + device       = "lfsw03"
      + id           = (known after apply)
      + interface_id = "vlan3002"
      + vrf_dn       = "sys/inst-tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf.main["lfsw01"] will be created
  + resource "nxos_vrf" "main" {
      + device = "lfsw01"
      + encap  = "vxlan-29001"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf.main["lfsw02"] will be created
  + resource "nxos_vrf" "main" {
      + device = "lfsw02"
      + encap  = "vxlan-29001"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf.main["lfsw03"] will be created
  + resource "nxos_vrf" "main" {
      + device = "lfsw03"
      + encap  = "vxlan-29001"
      + id     = (known after apply)
      + name   = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw01"] will be created
  + resource "nxos_vrf_address_family" "main" {
      + address_family = "ipv4-ucast"
      + device         = "lfsw01"
      + id             = (known after apply)
      + vrf            = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw02"] will be created
  + resource "nxos_vrf_address_family" "main" {
      + address_family = "ipv4-ucast"
      + device         = "lfsw02"
      + id             = (known after apply)
      + vrf            = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw03"] will be created
  + resource "nxos_vrf_address_family" "main" {
      + address_family = "ipv4-ucast"
      + device         = "lfsw03"
      + id             = (known after apply)
      + vrf            = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw01"] will be created
  + resource "nxos_vrf_route_target" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw02"] will be created
  + resource "nxos_vrf_route_target" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw03"] will be created
  + resource "nxos_vrf_route_target" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw01"] will be created
  + resource "nxos_vrf_route_target" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw02"] will be created
  + resource "nxos_vrf_route_target" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw03"] will be created
  + resource "nxos_vrf_route_target" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw01"] will be created
  + resource "nxos_vrf_route_target" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw02"] will be created
  + resource "nxos_vrf_route_target" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw03"] will be created
  + resource "nxos_vrf_route_target" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw01"] will be created
  + resource "nxos_vrf_route_target" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw02"] will be created
  + resource "nxos_vrf_route_target" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw03"] will be created
  + resource "nxos_vrf_route_target" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target                = "route-target:unknown:0:0"
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw01"] will be created
  + resource "nxos_vrf_route_target_address_family" "main" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw02"] will be created
  + resource "nxos_vrf_route_target_address_family" "main" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw03"] will be created
  + resource "nxos_vrf_route_target_address_family" "main" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw01"] will be created
  + resource "nxos_vrf_route_target_address_family" "main_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw02"] will be created
  + resource "nxos_vrf_route_target_address_family" "main_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw03"] will be created
  + resource "nxos_vrf_route_target_address_family" "main_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw01"] will be created
  + resource "nxos_vrf_route_target_direction" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw02"] will be created
  + resource "nxos_vrf_route_target_direction" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw03"] will be created
  + resource "nxos_vrf_route_target_direction" "export" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw01"] will be created
  + resource "nxos_vrf_route_target_direction" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw02"] will be created
  + resource "nxos_vrf_route_target_direction" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw03"] will be created
  + resource "nxos_vrf_route_target_direction" "export_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "export"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw01"] will be created
  + resource "nxos_vrf_route_target_direction" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw02"] will be created
  + resource "nxos_vrf_route_target_direction" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw03"] will be created
  + resource "nxos_vrf_route_target_direction" "import" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "ipv4-ucast"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw01"] will be created
  + resource "nxos_vrf_route_target_direction" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw01"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw02"] will be created
  + resource "nxos_vrf_route_target_direction" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw02"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw03"] will be created
  + resource "nxos_vrf_route_target_direction" "import_evpn" {
      + address_family              = "ipv4-ucast"
      + device                      = "lfsw03"
      + direction                   = "import"
      + id                          = (known after apply)
      + route_target_address_family = "l2vpn-evpn"
      + vrf                         = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw01"] will be created
  + resource "nxos_vrf_routing" "main" {
      + device              = "lfsw01"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
      + vrf                 = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw02"] will be created
  + resource "nxos_vrf_routing" "main" {
      + device              = "lfsw02"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
      + vrf                 = "tenant2-vpc1"
    }

  # module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw03"] will be created
  + resource "nxos_vrf_routing" "main" {
      + device              = "lfsw03"
      + id                  = (known after apply)
      + route_distinguisher = "rd:unknown:0:0"
      + vrf                 = "tenant2-vpc1"
    }

Plan: 102 to add, 0 to change, 3 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

nxos_save_config.main["lfsw02"]: Destroying...
nxos_save_config.main["lfsw03"]: Destroying...
nxos_save_config.main["lfsw01"]: Destroying...
nxos_save_config.main["lfsw03"]: Destruction complete after 0s
nxos_save_config.main["lfsw02"]: Destruction complete after 0s
nxos_save_config.main["lfsw01"]: Destruction complete after 0s
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw01"]: Creation complete after 0s [id=sys/bd/bd-[vlan-3002]]
module.l3vni["vni_29001"].nxos_vrf.main["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_vrf.main["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw03"]: Creation complete after 0s [id=sys/bd/bd-[vlan-3002]]
module.l3vni["vni_29001"].nxos_bridge_domain.main["lfsw02"]: Creation complete after 0s [id=sys/bd/bd-[vlan-3002]]
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf.main["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw01"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[29001]]
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw02"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]]
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw03"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]]
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw02"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[29001]]
module.l3vni["vni_29001"].nxos_svi_interface.main["lfsw01"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]]
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_nve_vni.main["lfsw03"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[29001]]
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw02"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw02"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan3002]]
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw02"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]/rtvrfMbr]
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw01"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_vrf_routing.main["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw03"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]/rtvrfMbr]
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_bgp_vrf.main["lfsw03"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]]
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw03"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan3002]]
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_ipv4_interface.main["lfsw01"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan3002]]
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_svi_interface_vrf.main["lfsw01"]: Creation complete after 0s [id=sys/intf/svi-[vlan3002]/rtvrfMbr]
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw02"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw03"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_bgp_address_family.main["lfsw01"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_vrf_address_family.main["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw02"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]/interleak-[direct]-interleak-[none]]
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]]
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw03"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]/interleak-[direct]-interleak-[none]]
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_bgp_route_redistribution.main["lfsw01"]: Creation complete after 0s [id=sys/bgp/inst/dom-[tenant2-vpc1]/af-[ipv4-ucast]/interleak-[direct]-interleak-[none]]
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]]
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main_evpn["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_address_family.main["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export_evpn["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import_evpn["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.import["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target_direction.export["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw03"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw01"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw02"]: Creating...
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw01"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw03"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.export_evpn["lfsw02"]: Creation complete after 0s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import_evpn["lfsw02"]: Creation complete after 1s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[l2vpn-evpn]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.import["lfsw02"]: Creation complete after 1s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l3vni["vni_29001"].nxos_vrf_route_target.export["lfsw02"]: Creation complete after 1s [id=sys/inst-[tenant2-vpc1]/dom-[tenant2-vpc1]/af-[ipv4-ucast]/ctrl-[ipv4-ucast]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw02"]: Creation complete after 0s [id=sys/bd/bd-[vlan-200]]
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw01"]: Creation complete after 0s [id=sys/bd/bd-[vlan-200]]
module.l2vni["vni_20200"].nxos_bridge_domain.main["lfsw03"]: Creation complete after 0s [id=sys/bd/bd-[vlan-20]]
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw01"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]]
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw01"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]]
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw03"]: Creation complete after 0s [id=sys/intf/svi-[vlan20]]
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw01"]: Creation complete after 0s [id=sys/intf/svi-[vlan200]]
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw03"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]]
module.l2vni["vni_20200"].nxos_evpn_vni.main["lfsw02"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]]
module.l2vni["vni_20200"].nxos_svi_interface.main["lfsw02"]: Creation complete after 0s [id=sys/intf/svi-[vlan200]]
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw02"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]]
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni.main["lfsw03"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw02"]: Creation complete after 0s [id=sys/hmm/fwdinst/if-[vlan200]]
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw01"]: Creation complete after 0s [id=sys/hmm/fwdinst/if-[vlan200]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw02"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan200]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw02"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]]
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw01"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan200]]
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw02"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw01"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]]
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_ipv4_interface.main["lfsw03"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan20]]
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw03"]: Creation complete after 0s [id=sys/intf/svi-[vlan20]/rtvrfMbr]
module.l2vni["vni_20200"].nxos_hmm_interface.main["lfsw03"]: Creation complete after 0s [id=sys/hmm/fwdinst/if-[vlan20]]
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw01"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]]
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw01"]: Creation complete after 0s [id=sys/intf/svi-[vlan200]/rtvrfMbr]
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw01"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]/IngRepl]
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw01"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan200]/addr-[172.17.0.254/24]]
module.l2vni["vni_20200"].nxos_svi_interface_vrf.main["lfsw02"]: Creation complete after 0s [id=sys/intf/svi-[vlan200]/rtvrfMbr]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.import["lfsw03"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target_direction.export["lfsw03"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw03"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw01"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw02"]: Creating...
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw01"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw02"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]/IngRepl]
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw02"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan200]/addr-[172.17.0.254/24]]
module.l2vni["vni_20200"].nxos_nve_vni_ingress_replication.main["lfsw03"]: Creation complete after 0s [id=sys/eps/epId-[1]/nws/vni-[20200]/IngRepl]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw01"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_ipv4_interface_address.main["lfsw03"]: Creation complete after 0s [id=sys/ipv4/inst/dom-[tenant2-vpc1]/if-[vlan20]/addr-[172.17.0.254/24]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw02"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw02"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.export["lfsw03"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[export]/ent-[route-target:unknown:0:0]]
module.l2vni["vni_20200"].nxos_evpn_vni_route_target.import["lfsw03"]: Creation complete after 0s [id=sys/evpn/bdevi-[vxlan-20200]/rttp-[import]/ent-[route-target:unknown:0:0]]

Apply complete! Resources: 102 added, 0 changed, 3 destroyed.
```

</div></details>

<details><summary>　lfsw01 diff　</summary><div>


```diff:lfsw01
lfsw01# show run diff
*** Startup-config
--- Running-config
***************
*** 42,64 ****
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,100-101,1001,3001
  vlan 100
    name tenant1-server-seg1
    vn-segment 10100
  vlan 101
    name tenant1-vlan101
    vn-segment 10101
  vlan 1001
    name tenant3-nw-vlan300
    vn-segment 30300
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
  
  ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
  route-map permit-all-v4 permit 100
    match ip address prefix-list all-v4 
  vrf context management
--- 41,69 ----
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,100-101,200,1001,3001-3002
  vlan 100
    name tenant1-server-seg1
    vn-segment 10100
  vlan 101
    name tenant1-vlan101
    vn-segment 10101
+ vlan 200
+   name tenant2-vpc1-server-seg1
+   vn-segment 20200
  vlan 1001
    name tenant3-nw-vlan300
    vn-segment 30300
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
+ vlan 3002
+   name tenant2-vpc1-l3vni
+   vn-segment 29001
  
  ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
  route-map permit-all-v4 permit 100
    match ip address prefix-list all-v4 
  vrf context management
***************
*** 68,77 ****
--- 73,88 ----
    vni 19001
    rd auto
    address-family ipv4 unicast
      route-target both auto
      route-target both auto evpn
+ vrf context tenant2-vpc1
+   vni 29001
+   rd auto
+   address-family ipv4 unicast
+     route-target both auto
+     route-target both auto evpn
  vpc domain 1
    role priority 100
    peer-keepalive destination 192.168.129.52 source 192.168.129.51
    virtual peer-link destination 10.0.0.2 source 10.0.0.1 dscp 56
  
***************
*** 90,104 ****
--- 101,127 ----
    no shutdown
    vrf member tenant1-vpc1
    ip address 172.16.1.254/24
    fabric forwarding mode anycast-gateway
  
+ interface Vlan200
+   description tenant2-vpc1-server-seg1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip address 172.17.0.254/24
+ 
  interface Vlan3001
    no shutdown
    vrf member tenant1-vpc1
    ip forward
  
+ interface Vlan3002
+   description tenant2-vpc1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip forward
+ 
  interface port-channel1
    description virtual-vpc-peer-link
    switchport mode trunk
    spanning-tree port type network
    vpc peer-link
***************
*** 126,135 ****
--- 149,161 ----
    member vni 10100
      ingress-replication protocol bgp
    member vni 10101
      ingress-replication protocol bgp
    member vni 19001 associate-vrf
+   member vni 20200
+     ingress-replication protocol bgp
+   member vni 29001 associate-vrf
    member vni 30300
      ingress-replication protocol bgp
  
  interface Ethernet1/1
    description tenant1-server01 ens2
***************
*** 371,389 ****
--- 397,422 ----
        send-community
        send-community extended
    vrf tenant1-vpc1
      address-family ipv4 unicast
        redistribute direct route-map permit-all-v4
+   vrf tenant2-vpc1
+     address-family ipv4 unicast
+       redistribute direct route-map permit-all-v4
  evpn
    vni 10100 l2
      rd auto
      route-target import auto
      route-target export auto
    vni 10101 l2
      rd auto
      route-target import auto
      route-target export auto
+   vni 20200 l2
+     rd auto
+     route-target import auto
+     route-target export auto
    vni 30300 l2
      rd auto
      route-target import auto
      route-target export auto
  
lfsw01# show run

!Command: show running-config
!Running configuration last done at: Sun Mar  2 21:07:45 2025
!Time: Sun Mar  2 21:17:53 2025

version 10.5(1) Bios:version  
hostname lfsw01
vdc lfsw01 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs ipv4 distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$PJAMHI$W4Ih7NfB87rKZatB0F3LTQtWPHkZK4uhH6Byb767Oj5  role network-admin
username cisco password 5 $5$FCPIIG$RRagIkHJeow6kxDTamd2Xsp4TUkeGCRQeVdz0pHwlD0  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
bfd startup-timer 0
snmp-server user admin network-admin auth md5 3328CD1117679702547B2F13039654E3AC6A priv aes-128 0153CA151511AC3C1C4A5A330C9253ACA93D localizedV2key
snmp-server user cisco network-admin auth md5 056E98403323A677350C5F675BC31EC5F735 priv aes-128 33199E4E4743BF3E6A11426643C317B4A667 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,100-101,200,1001,3001-3002
vlan 100
  name tenant1-server-seg1
  vn-segment 10100
vlan 101
  name tenant1-vlan101
  vn-segment 10101
vlan 200
  name tenant2-vpc1-server-seg1
  vn-segment 20200
vlan 1001
  name tenant3-nw-vlan300
  vn-segment 30300
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001
vlan 3002
  name tenant2-vpc1-l3vni
  vn-segment 29001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context tenant2-vpc1
  vni 29001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vpc domain 1
  role priority 100
  peer-keepalive destination 192.168.129.52 source 192.168.129.51
  virtual peer-link destination 10.0.0.2 source 10.0.0.1 dscp 56


interface Vlan1

interface Vlan100
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan101
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  description tenant2-vpc1-server-seg1
  no shutdown
  vrf member tenant2-vpc1
  ip address 172.17.0.254/24

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface Vlan3002
  description tenant2-vpc1
  no shutdown
  vrf member tenant2-vpc1
  ip forward

interface port-channel1
  description virtual-vpc-peer-link
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface port-channel11
  description tenant1-server01 bond0
  switchport mode trunk
  vpc 11

interface port-channel12
  description tenant2-server01 bond0
  switchport mode trunk
  vpc 12

interface port-channel13
  description tenant3-server01 bond0
  switchport mode trunk
  vpc 13

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf
  member vni 20200
    ingress-replication protocol bgp
  member vni 29001 associate-vrf
  member vni 30300
    ingress-replication protocol bgp

interface Ethernet1/1
  description tenant1-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 11 mode active

interface Ethernet1/2
  description tenant2-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 12 mode active

interface Ethernet1/3
  description tenant3-server01 ens2
  lacp rate fast
  switchport mode trunk
  channel-group 13 mode active

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/1
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.4.0/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/1
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.3.0/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.51/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.1/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.1/32
  ip address 10.0.2.1/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  bfd
  router-id 10.0.0.1
router bgp 65001
  router-id 10.0.0.1
  address-family l2vpn evpn
    advertise-pip
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
  vrf tenant2-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20200 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30300 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console


```

</div></details>

<details><summary>　lfsw02 diff　</summary><div>


```diff:lfsw02_diff
lfsw02# show run diff
*** Startup-config
--- Running-config
***************
*** 41,63 ****
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,100-101,1001,3001
  vlan 100
    name tenant1-server-seg1
    vn-segment 10100
  vlan 101
    name tenant1-vlan101
    vn-segment 10101
  vlan 1001
    name tenant3-nw-vlan300
    vn-segment 30300
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
  
  ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
  route-map permit-all-v4 permit 100
    match ip address prefix-list all-v4 
  vrf context management
--- 40,68 ----
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,100-101,200,1001,3001-3002
  vlan 100
    name tenant1-server-seg1
    vn-segment 10100
  vlan 101
    name tenant1-vlan101
    vn-segment 10101
+ vlan 200
+   name tenant2-vpc1-server-seg1
+   vn-segment 20200
  vlan 1001
    name tenant3-nw-vlan300
    vn-segment 30300
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
+ vlan 3002
+   name tenant2-vpc1-l3vni
+   vn-segment 29001
  
  ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
  route-map permit-all-v4 permit 100
    match ip address prefix-list all-v4 
  vrf context management
***************
*** 67,76 ****
--- 72,87 ----
    vni 19001
    rd auto
    address-family ipv4 unicast
      route-target both auto
      route-target both auto evpn
+ vrf context tenant2-vpc1
+   vni 29001
+   rd auto
+   address-family ipv4 unicast
+     route-target both auto
+     route-target both auto evpn
  vpc domain 1
    role priority 200
    peer-keepalive destination 192.168.129.51 source 192.168.129.52
    virtual peer-link destination 10.0.0.1 source 10.0.0.2 dscp 56
  
***************
*** 89,103 ****
--- 100,126 ----
    no shutdown
    vrf member tenant1-vpc1
    ip address 172.16.1.254/24
    fabric forwarding mode anycast-gateway
  
+ interface Vlan200
+   description tenant2-vpc1-server-seg1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip address 172.17.0.254/24
+ 
  interface Vlan3001
    no shutdown
    vrf member tenant1-vpc1
    ip forward
  
+ interface Vlan3002
+   description tenant2-vpc1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip forward
+ 
  interface port-channel1
    description virtual-vpc-peer-link
    switchport mode trunk
    spanning-tree port type network
    vpc peer-link
***************
*** 125,134 ****
--- 148,160 ----
    member vni 10100
      ingress-replication protocol bgp
    member vni 10101
      ingress-replication protocol bgp
    member vni 19001 associate-vrf
+   member vni 20200
+     ingress-replication protocol bgp
+   member vni 29001 associate-vrf
    member vni 30300
      ingress-replication protocol bgp
  
  interface Ethernet1/1
    description tenant1-server01 ens3
***************
*** 369,387 ****
--- 395,420 ----
        send-community
        send-community extended
    vrf tenant1-vpc1
      address-family ipv4 unicast
        redistribute direct route-map permit-all-v4
+   vrf tenant2-vpc1
+     address-family ipv4 unicast
+       redistribute direct route-map permit-all-v4
  evpn
    vni 10100 l2
      rd auto
      route-target import auto
      route-target export auto
    vni 10101 l2
      rd auto
      route-target import auto
      route-target export auto
+   vni 20200 l2
+     rd auto
+     route-target import auto
+     route-target export auto
    vni 30300 l2
      rd auto
      route-target import auto
      route-target export auto
  
lfsw02# show run

!Command: show running-config
!Running configuration last done at: Sun Mar  2 21:07:35 2025
!Time: Sun Mar  2 21:20:37 2025

version 10.5(1) Bios:version  
hostname lfsw02
vdc lfsw02 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs ipv4 distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$MIABHO$I4vvga9QpydmKdK1iLAnJtqJx/sFAUrWABoY6cEgBr.  role network-admin
username cisco password 5 $5$GMELDH$8v4a2WA5YuNub3XWE9/EKcgcIZpXPNPSGyubE/r0eLA  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 00514DE2E2F0FAE3138880548031669F521C priv aes-128 323C6DD1DD1B212C9C760EC557EEA547FEF8 localizedV2key
snmp-server user cisco network-admin auth md5 37427EC18B623A31C73A5FD91EE3AE1B83D7 priv aes-128 482F13A7F6505F4B96351E8D11EE8403C2D5 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,100-101,200,1001,3001-3002
vlan 100
  name tenant1-server-seg1
  vn-segment 10100
vlan 101
  name tenant1-vlan101
  vn-segment 10101
vlan 200
  name tenant2-vpc1-server-seg1
  vn-segment 20200
vlan 1001
  name tenant3-nw-vlan300
  vn-segment 30300
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001
vlan 3002
  name tenant2-vpc1-l3vni
  vn-segment 29001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context tenant2-vpc1
  vni 29001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vpc domain 1
  role priority 200
  peer-keepalive destination 192.168.129.51 source 192.168.129.52
  virtual peer-link destination 10.0.0.1 source 10.0.0.2 dscp 56


interface Vlan1

interface Vlan100
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan101
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  description tenant2-vpc1-server-seg1
  no shutdown
  vrf member tenant2-vpc1
  ip address 172.17.0.254/24

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface Vlan3002
  description tenant2-vpc1
  no shutdown
  vrf member tenant2-vpc1
  ip forward

interface port-channel1
  description virtual-vpc-peer-link
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface port-channel11
  description tenant1-server01 bond0
  switchport mode trunk
  vpc 11

interface port-channel12
  description tenant2-server01 bond0
  switchport mode trunk
  vpc 12

interface port-channel13
  description tenant3-server01 bond0
  switchport mode trunk
  vpc 13

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf
  member vni 20200
    ingress-replication protocol bgp
  member vni 29001 associate-vrf
  member vni 30300
    ingress-replication protocol bgp

interface Ethernet1/1
  description tenant1-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 11 mode active

interface Ethernet1/2
  description tenant2-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 12 mode active

interface Ethernet1/3
  description tenant3-server01 ens3
  lacp rate fast
  switchport mode trunk
  channel-group 13 mode active

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/2
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.4.2/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/2
  no switchport
  mtu 9216
  port-type fabric
  ip address 10.0.3.2/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.52/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.2/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.2/32
  ip address 10.0.2.1/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  router-id 10.0.0.2
router bgp 65001
  router-id 10.0.0.2
  address-family l2vpn evpn
    advertise-pip
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
  vrf tenant2-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20200 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30300 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console


```

</div></details>

<details><summary>　lfsw03 diff　</summary><div>

```diff:lfsw03_diff
lfsw03# show run diff
*** Startup-config
--- Running-config
***************
*** 40,56 ****
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,10-11,3001-3002
  vlan 10
    name tenant1-server-seg1
    vn-segment 10100
  vlan 11
    name tenant1-server-seg2
    vn-segment 10101
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
  vlan 3002
    name tenant2-vpc1-l3vni
--- 39,58 ----
  rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
  rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
  rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
  
  fabric forwarding anycast-gateway-mac 2020.0000.00aa
! vlan 1,10-11,20,3001-3002
  vlan 10
    name tenant1-server-seg1
    vn-segment 10100
  vlan 11
    name tenant1-server-seg2
    vn-segment 10101
+ vlan 20
+   name tenant2-vpc1-server-seg1
+   vn-segment 20200
  vlan 3001
    name tenant1-vpc1-l3vni
    vn-segment 19001
  vlan 3002
    name tenant2-vpc1-l3vni
***************
*** 66,75 ****
--- 68,81 ----
    vni 19001
    rd auto
    address-family ipv4 unicast
  vrf context tenant2-vpc1
    vni 29001
+   rd auto
+   address-family ipv4 unicast
+     route-target both auto
+     route-target both auto evpn
  
  
  interface Vlan1
  
  interface Vlan10
***************
*** 84,108 ****
--- 90,130 ----
    no shutdown
    vrf member tenant1-vpc1
    ip address 172.16.1.254/24
    fabric forwarding mode anycast-gateway
  
+ interface Vlan20
+   description tenant2-vpc1-server-seg1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip address 172.17.0.254/24
+   fabric forwarding mode anycast-gateway
+ 
  interface Vlan3001
    no shutdown
    vrf member tenant1-vpc1
    ip forward
  
+ interface Vlan3002
+   description tenant2-vpc1
+   no shutdown
+   vrf member tenant2-vpc1
+   ip forward
+ 
  interface nve1
    no shutdown
    host-reachability protocol bgp
    advertise virtual-rmac
    source-interface loopback1
    member vni 10100
      ingress-replication protocol bgp
    member vni 10101
      ingress-replication protocol bgp
    member vni 19001 associate-vrf
+   member vni 20200
+     ingress-replication protocol bgp
+   member vni 29001 associate-vrf
  
  interface Ethernet1/1
    description tenant1-server02 eth0
    switchport mode trunk
  
***************
*** 332,349 ****
--- 354,378 ----
        send-community
        send-community extended
    vrf tenant1-vpc1
      address-family ipv4 unicast
        redistribute direct route-map permit-all-v4
+   vrf tenant2-vpc1
+     address-family ipv4 unicast
+       redistribute direct route-map permit-all-v4
  evpn
    vni 10100 l2
      rd auto
      route-target import auto
      route-target export auto
    vni 10101 l2
      rd auto
      route-target import auto
      route-target export auto
+   vni 20200 l2
+     rd auto
+     route-target import auto
+     route-target export auto
  
  no logging console
  
  
lfsw03# show run

!Command: show running-config
!Running configuration last done at: Sun Mar  2 21:07:40 2025
!Time: Sun Mar  2 21:21:35 2025

version 10.5(1) Bios:version  
hostname lfsw03
vdc lfsw03 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature nxapi
cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature lldp
feature bfd
clock timezone JST 9 0
feature nv overlay

no password strength-check
username admin password 5 $5$DCALLD$97LM57TD1qBqvKxSPC.zrFWkwLb.mcI3oelSIY2ezK6  role network-admin
username cisco password 5 $5$PLGLCI$KPPOvBPkC9lKtmwVpmhHOPkkAMuMSqfWf2Lj7yVmBX0  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
ssh key rsa 2048 
no ip domain-lookup
copp profile strict
snmp-server user admin network-admin auth md5 4976440ADE75ACA80D512BEB0C28C2391689 priv aes-128 174C152CF52BB2BA0C0D5F90515ECB771395 localizedV2key
snmp-server user cisco network-admin auth md5 53150C1621F12906B9B08951D9C85FB8D557 priv aes-128 01462E3AB6CF0D1A9FEAC459969410F98A55 localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac 2020.0000.00aa
vlan 1,10-11,20,3001-3002
vlan 10
  name tenant1-server-seg1
  vn-segment 10100
vlan 11
  name tenant1-server-seg2
  vn-segment 10101
vlan 20
  name tenant2-vpc1-server-seg1
  vn-segment 20200
vlan 3001
  name tenant1-vpc1-l3vni
  vn-segment 19001
vlan 3002
  name tenant2-vpc1-l3vni
  vn-segment 29001

ip prefix-list all-v4 seq 10 permit 0.0.0.0/0 ge 1 le 32 
route-map permit-all-v4 permit 100
  match ip address prefix-list all-v4 
vrf context management
  ip name-server 192.168.129.254
  ip route 0.0.0.0/0 192.168.129.254
vrf context tenant1-vpc1
  vni 19001
  rd auto
  address-family ipv4 unicast
vrf context tenant2-vpc1
  vni 29001
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn


interface Vlan1

interface Vlan10
  description tenant1-server-seg1
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan11
  description tenant1-server-seg2
  no shutdown
  vrf member tenant1-vpc1
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan20
  description tenant2-vpc1-server-seg1
  no shutdown
  vrf member tenant2-vpc1
  ip address 172.17.0.254/24
  fabric forwarding mode anycast-gateway

interface Vlan3001
  no shutdown
  vrf member tenant1-vpc1
  ip forward

interface Vlan3002
  description tenant2-vpc1
  no shutdown
  vrf member tenant2-vpc1
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  advertise virtual-rmac
  source-interface loopback1
  member vni 10100
    ingress-replication protocol bgp
  member vni 10101
    ingress-replication protocol bgp
  member vni 19001 associate-vrf
  member vni 20200
    ingress-replication protocol bgp
  member vni 29001 associate-vrf

interface Ethernet1/1
  description tenant1-server02 eth0
  switchport mode trunk

interface Ethernet1/2
  description tenant1-server03 eth0
  switchport mode trunk

interface Ethernet1/3
  description tenant2-server02 eth0
  switchport mode trunk

interface Ethernet1/4
  shutdown

interface Ethernet1/5
  shutdown

interface Ethernet1/6
  shutdown

interface Ethernet1/7
  shutdown

interface Ethernet1/8
  shutdown

interface Ethernet1/9
  shutdown

interface Ethernet1/10
  shutdown

interface Ethernet1/11
  shutdown

interface Ethernet1/12
  shutdown

interface Ethernet1/13
  shutdown

interface Ethernet1/14
  shutdown

interface Ethernet1/15
  shutdown

interface Ethernet1/16
  shutdown

interface Ethernet1/17
  shutdown

interface Ethernet1/18
  shutdown

interface Ethernet1/19
  shutdown

interface Ethernet1/20
  shutdown

interface Ethernet1/21
  shutdown

interface Ethernet1/22
  shutdown

interface Ethernet1/23
  shutdown

interface Ethernet1/24
  shutdown

interface Ethernet1/25
  shutdown

interface Ethernet1/26
  shutdown

interface Ethernet1/27
  shutdown

interface Ethernet1/28
  shutdown

interface Ethernet1/29
  shutdown

interface Ethernet1/30
  shutdown

interface Ethernet1/31
  shutdown

interface Ethernet1/32
  shutdown

interface Ethernet1/33
  shutdown

interface Ethernet1/34
  shutdown

interface Ethernet1/35
  shutdown

interface Ethernet1/36
  shutdown

interface Ethernet1/37
  shutdown

interface Ethernet1/38
  shutdown

interface Ethernet1/39
  shutdown

interface Ethernet1/40
  shutdown

interface Ethernet1/41
  shutdown

interface Ethernet1/42
  shutdown

interface Ethernet1/43
  shutdown

interface Ethernet1/44
  shutdown

interface Ethernet1/45
  shutdown

interface Ethernet1/46
  shutdown

interface Ethernet1/47
  description spsw02 eth1/3
  no switchport
  mtu 9216
  ip address 10.0.4.4/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/48
  description spsw01 eth1/3
  no switchport
  mtu 9216
  ip address 10.0.3.4/31
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface Ethernet1/55

interface Ethernet1/56

interface Ethernet1/57

interface Ethernet1/58

interface Ethernet1/59

interface Ethernet1/60

interface Ethernet1/61

interface Ethernet1/62

interface Ethernet1/63

interface Ethernet1/64

interface mgmt0
  vrf member management
  ip address 192.168.129.53/24

interface loopback0
  description RouterID Loopback
  ip address 10.0.0.3/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  description VTEP Loopback
  ip address 10.0.1.3/32
  ip address 10.0.2.2/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
  exec-timeout 0
line vty
  exec-timeout 0
boot nxos bootflash:/nxos64-cs.10.5.1.F.bin 
router ospf 1
  router-id 10.0.0.3
router bgp 65001
  router-id 10.0.0.3
  address-family l2vpn evpn
  neighbor 10.0.0.253
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.0.0.254
    remote-as internal
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf tenant1-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
  vrf tenant2-vpc1
    address-family ipv4 unicast
      redistribute direct route-map permit-all-v4
evpn
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10101 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20200 l2
    rd auto
    route-target import auto
    route-target export auto

no logging console


```

</div></details>

### 切り戻し

追加パラメータをコメントアウトして再度 `terraform apply` するのみで切り戻しが可能

```config:切り戻し
l3vni_map = {
  # vni_29001 = {
  #   vni = 29001
  #   vrf = "tenant2-vpc1"
  #   members = {
  #     lfsw01 = {
  #       vlan = "3002"
  #     }
  #     lfsw02 = {
  #       vlan = "3002"
  #     }
  #     lfsw03 = {
  #       vlan = "3002"
  #     }
  #   }
  # }
}

l2vni_map = {
  # vni_20200 = {
  #   vni = 20200
  #   vrf = "tenant2-vpc1"

  #   segment_name = "server-seg1"
  #   gateway_ip   = "172.17.0.254/24"
  #   members = {
  #     lfsw01 = {
  #       vlan = "200"
  #     }
  #     lfsw02 = {
  #       vlan = "200"
  #     }
  #     lfsw03 = {
  #       vlan = "20"
  #     }
  #   }
  # }
}

```

上記切り戻しをしたいところをコメントアウト(もしくは削除)して再度 `terraform apply` を実行する

```sh:切り戻しも実行時と同じコマンド
terraform apply
```

下記が実行した際の画面 GIF 。`show run diff` をして設定前と差分がないことを確認できる

![terraform_VNI削除_SD1.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/6d41a78f-f5a8-4287-931e-984046e0beb3.gif)

対象機器のみ対象にパラメータをコメントアウトするとそこだけ削除も可能。下記は `lfsw03` のみをコメントアウトして実行した場合の例

```config

l3vni_map = {
  vni_29001 = {
    vni = 29001
    vrf = "tenant2-vpc1"
    members = {
      lfsw01 = {
        vlan = "3002"
      }
      lfsw02 = {
        vlan = "3002"
      }
      # lfsw03 = {
      #   vlan = "3002"
      # }
    }
  }
}

l2vni_map = {
  vni_20200 = {
    vni = 20200
    vrf = "tenant2-vpc1"

    segment_name = "server-seg1"
    gateway_ip   = "172.17.0.254/24"
    members = {
      lfsw01 = {
        vlan = "200"
      }
      lfsw02 = {
        vlan = "200"
      }
      # lfsw03 = {
      #   vlan = "20"
      # }
    }
  }
}

```

上記実行画面の GIF が下記の通り

![terraform_VNI削除_lfsw03only_SD.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/326183/f1c1ef7c-d545-45b8-9115-9fed81be49f0.gif)

## おわりに

Terraform で L3VNI/L2VNI の追加削除が可能であることを確認できた

最終的に全て Terraform で管理するか、初期設定は別管理にして、管理範囲を分けるかなど検討が必要

## 参考

https://registry.terraform.io/providers/CiscoDevNet/nxos/latest/docs

## 追記

### DualStack 対応

IPv6 を実施する Overlay 環境に対応するために DualStack 対応を別途実施した

対応内容は下記 GitHub 内に記載・公開している

https://github.com/suzuyu/nxos-terraform/tree/v1.2.0/env/cml/vni_dualstack

terraform の nxos provider の対応状況は v0.5.8 時点では Interface 向けの IPv6 設定が見当たらないため、[REST モジュール](https://registry.terraform.io/providers/CiscoDevNet/nxos/latest/docs/resources/rest) を使用して対応した

REST API 内容は [NX-API Sandbox](https://www.cisco.com/c/ja_jp/td/docs/dcn/nx-os/nexus9000/102x/programmability/cisco-nexus-9000-series-nx-os-programmability-guide-release-102x/m-n9k-nx-api-developer-sandbox-101x.html) を利用して作成した