SONiCを VirtualBox + Vagrant で動かす


SONiCを試してみようとすると，KVM用のファイルしか無いようです．個人的にVirtualBox + Vagrantのほうが使いやすいので，VirtualBox + Vagrantで動かす方法をまとめてみたという話です．ただし，ネットワーク実験ができれば良いということを目的にしているので，それ以外の部分は，雑に設定していますがご了承下さい．

## 実行環境
OS: Windows 10
Virtualbox: 6.1.34
Vagrant: 2.2.19
SONiC: SONiC.master.102109-392899682


## SONiCをVirtualBox + Vagrantで動くようにする
まず，SONiCをダウンロードして，VirtualBoxで動かせるようにします．そしてVirtualboxのVMからVagrant Boxを作成します．

### SONiCのダウンロード
[https://sonic.software/](https://sonic.software/)から`sonic-vs.img.gz`をダウンロードしてください．注意事項として，一番上のリンクが一番古いやつです．一番下が最新のものなので，注意してください．
(Branch 202012のファイルだと，bgpのサービスが起動後に落ちてしまいました;;)


### imgファイルをdviファイルに変換

`sonic-vs.img.gz`ファイルの解凍して，`vdi`ファイルに変換します．変換には`qemu-img.exe`を使用します．これは，[QEMU](https://www.qemu.org/download/)をダウンロードする必要があります．

```powershell
 "C:\Program Files\qemu\qemu-img.exe" convert -O vdi sonic-vs.img sonic-vs.vdi
```
imgファイルをdviファイルに変換する方法として，VBoxManageもありますが，私の環境だとうまくいきませんでした．

変換後VirtualBoxで新規のVMを立てて，仮想ハードディスクとして作成したdviファイルを指定します．

私の環境の設定例を以下に示します．
![screenshot000194.JPG](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2661127/772100c4-c736-dc12-5bd5-b50632881134.jpeg)

VMを起動して，動くかを確かめます．ユーザ名は`admin`でパスワードは`YourPaSsWoRd`です．

起動後`show version`でバージョン確認．
```
$ show version
SONiC Software Version: SONiC.master.102109-392899682
Distribution: Debian 11.3
Kernel: 5.10.0-12-2-amd64
Build commit: 392899682
Build date: Mon May 23 19:34:41 UTC 2022
Built by: AzDevOps@sonic-build-workers-001J3P

Platform: x86_64-kvm_x86_64-r0
HwSKU: Force10-S6000
ASIC: vs
ASIC Count: 1
Serial Number: N/A
Model Number: N/A
Hardware Revision: N/A
Uptime: 03:30:40 up 3 min,  1 user,  load average: 0.49, 0.75, 0.36
Date: Sun 29 May 2022 03:30:40
```

### VirtualBox上のVMをVagrant Boxに変換
Vagrant Boxを作成する前にVirtaulBoxのGuestAdditionsをインストール．インストールするバージョンは，自分の環境にあったものに変えてください．

```bash
sudo apt update -y
sudo apt install -y build-essential dkms wget
wget http://download.virtualbox.org/virtualbox/6.1.34/VBoxGuestAdditions_6.1.34.iso
sudo mkdir /media/VBoxGuestAdditions
sudo mount -o loop,ro VBoxGuestAdditions_6.1.34.iso /media/VBoxGuestAdditions
sudo sh /media/VBoxGuestAdditions/VBoxLinuxAdditions.run
rm VBoxGuestAdditions_6.1.34.iso
```

GuestAddtionsをインストール後，VMを終了します．その後，Vagrant Boxを作成します．
作成するコマンド以下です．`<VirtaulBoxName>`は作ったVirtaulBoxのVMの名前です．
```shell
vagrant package --base "<VirtualBoxName>" --output sonic-vs.box
vagrant box add sonic-vs.box --name sonic-vs
```

Vagrant Boxが追加されたのかを確認します．
```
$ vagrant box list
CumulusCommunity/cumulus-vx (virtualbox, 5.1.0)
bento/ubuntu-18.04          (virtualbox, 202112.19.0)
bento/ubuntu-20.04          (virtualbox, 202112.19.0)
bento/ubuntu-20.10          (virtualbox, 202107.28.0)
bento/ubuntu-21.10          (virtualbox, 202112.19.0)
jpiel/ubuntu16.04-desktop   (virtualbox, 2019.05.01.0)
sonic-vs                    (virtualbox, 0)
ubuntu/jammy64              (virtualbox, 20220513.0.0)
```

ここでは，vagrantのユーザやSSHの設定は行いません．理由としては，しなくてもネットワークの実験はできるからです．vagrantのユーザなどの設定をするなら，古いですが[VIRTUALBOX から VAGARNT の BOX 作成](https://qiita.com/Burning-Chai/items/8299ff70f44b3b54cf64)が参考になると思います．

## 動作確認
動作確認として，下のようなネットワークを構築します．
それぞれ図のようにAS番号を割り当て，eBGPでルートの交換を行います．BGPの設定はUnnumberedで行います．

![test_topo.drawio.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/2661127/2aef6f06-fe8d-5c9c-02cb-62a5d6c1096a.png)


### /etc/sonic/config_db.jsonの設定
SONiCの`config_db.json`の内容を書き換えていきます．具体的には，初期設定でBGPの設定などが書かれていますが，今回はvtyshで設定するので，BGPの設定などを削除します．
```json:/etc/sonic/config_db.json
{
  "DEVICE_METADATA": {
      "localhost": {
          "hwsku": "Force10-S6000",
          "platform": "x86_64-kvm_x86_64-r0",
          "mac": "08:00:27:f6:18:74",
          "hostname": "sonic",
          "type": "LeafRouter"
      }
  },
  "PORT": {
      "Ethernet0": {
          "lanes": "25,26,27,28",
          "alias": "fortyGigE0/0",
          "index": "0",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet4": {
          "lanes": "29,30,31,32",
          "alias": "fortyGigE0/4",
          "index": "1",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet8": {
          "lanes": "33,34,35,36",
          "alias": "fortyGigE0/8",
          "index": "2",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet12": {
          "lanes": "37,38,39,40",
          "alias": "fortyGigE0/12",
          "index": "3",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet16": {
          "lanes": "45,46,47,48",
          "alias": "fortyGigE0/16",
          "index": "4",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet20": {
          "lanes": "41,42,43,44",
          "alias": "fortyGigE0/20",
          "index": "5",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet24": {
          "lanes": "1,2,3,4",
          "alias": "fortyGigE0/24",
          "index": "6",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet28": {
          "lanes": "5,6,7,8",
          "alias": "fortyGigE0/28",
          "index": "7",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet32": {
          "lanes": "13,14,15,16",
          "alias": "fortyGigE0/32",
          "index": "8",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet36": {
          "lanes": "9,10,11,12",
          "alias": "fortyGigE0/36",
          "index": "9",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet40": {
          "lanes": "17,18,19,20",
          "alias": "fortyGigE0/40",
          "index": "10",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet44": {
          "lanes": "21,22,23,24",
          "alias": "fortyGigE0/44",
          "index": "11",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet48": {
          "lanes": "53,54,55,56",
          "alias": "fortyGigE0/48",
          "index": "12",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet52": {
          "lanes": "49,50,51,52",
          "alias": "fortyGigE0/52",
          "index": "13",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet56": {
          "lanes": "57,58,59,60",
          "alias": "fortyGigE0/56",
          "index": "14",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet60": {
          "lanes": "61,62,63,64",
          "alias": "fortyGigE0/60",
          "index": "15",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet64": {
          "lanes": "69,70,71,72",
          "alias": "fortyGigE0/64",
          "index": "16",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet68": {
          "lanes": "65,66,67,68",
          "alias": "fortyGigE0/68",
          "index": "17",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet72": {
          "lanes": "73,74,75,76",
          "alias": "fortyGigE0/72",
          "index": "18",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet76": {
          "lanes": "77,78,79,80",
          "alias": "fortyGigE0/76",
          "index": "19",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet80": {
          "lanes": "109,110,111,112",
          "alias": "fortyGigE0/80",
          "index": "20",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet84": {
          "lanes": "105,106,107,108",
          "alias": "fortyGigE0/84",
          "index": "21",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet88": {
          "lanes": "113,114,115,116",
          "alias": "fortyGigE0/88",
          "index": "22",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet92": {
          "lanes": "117,118,119,120",
          "alias": "fortyGigE0/92",
          "index": "23",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet96": {
          "lanes": "125,126,127,128",
          "alias": "fortyGigE0/96",
          "index": "24",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet100": {
          "lanes": "121,122,123,124",
          "alias": "fortyGigE0/100",
          "index": "25",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet104": {
          "lanes": "81,82,83,84",
          "alias": "fortyGigE0/104",
          "index": "26",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet108": {
          "lanes": "85,86,87,88",
          "alias": "fortyGigE0/108",
          "index": "27",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet112": {
          "lanes": "93,94,95,96",
          "alias": "fortyGigE0/112",
          "index": "28",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet116": {
          "lanes": "89,90,91,92",
          "alias": "fortyGigE0/116",
          "index": "29",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet120": {
          "lanes": "101,102,103,104",
          "alias": "fortyGigE0/120",
          "index": "30",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet124": {
          "lanes": "97,98,99,100",
          "alias": "fortyGigE0/124",
          "index": "31",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      }
  },
  "FLEX_COUNTER_TABLE": {
      "ACL": {
          "FLEX_COUNTER_STATUS": "disable",
          "FLEX_COUNTER_DELAY_STATUS": "true",
          "POLL_INTERVAL": "10000"
      }
  },
  "LOOPBACK_INTERFACE": {
      "Loopback0|10.1.0.1/32": {}
  },
  "INTERFACE": {
    "Ethernet0": {},
    "Ethernet4": {},
    "Ethernet8": {},
    "Ethernet12": {},
    "Ethernet16": {},
    "Ethernet20": {},
    "Ethernet24": {},
    "Ethernet28": {},
    "Ethernet32": {},
    "Ethernet36": {},
    "Ethernet40": {},
    "Ethernet44": {},
    "Ethernet48": {},
    "Ethernet52": {},
    "Ethernet56": {},
    "Ethernet60": {},
    "Ethernet64": {},
    "Ethernet68": {},
    "Ethernet72": {},
    "Ethernet76": {},
    "Ethernet80": {},
    "Ethernet84": {},
    "Ethernet88": {},
    "Ethernet92": {},
    "Ethernet96": {},
    "Ethernet100": {},
    "Ethernet104": {},
    "Ethernet108": {},
    "Ethernet112": {},
    "Ethernet116": {},
    "Ethernet120": {},
    "Ethernet124": {}
  }
}
```

ファイルを書き換えた後は，リロードします．
```bash
sudo config reload -y
```

### vtyshの設定
Spine1の設定を以下に示します．基本的には，BGP Unnumberedの設定を行います．
```
enable
configure terminal
interface lo
  ip address 10.0.0.1/32
interface Ethernet0
  ip address 10.1.0.1/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
interface Ethernet4
  ip address 10.2.0.1/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
router bgp 65200
  bgp router-id 10.0.0.1
  no bgp ebgp-requires-policy
  bgp bestpath as-path multipath-relax
  neighbor FABRIC peer-group
  neighbor FABRIC remote-as external
  neighbor FABRIC capability extended-nexthop
  neighbor Ethernet0 interface peer-group FABRIC
  neighbor Ethernet4 interface peer-group FABRIC
  address-family ipv4 unicast
    network 10.0.0.1/32
  exit-address-family
```


### Vagrantfile

VagrantBoxの設定では，sshの設定に注意してください．
sshは，ユーザとパスワードで行うようにします．また，共有フォルダはエラーをはくので，disabledにします．
```ruby
switch.ssh.username = 'admin'
switch.ssh.password = 'YourPaSsWoRd'
switch.ssh.insert_key = false
switch.vm.synced_folder '.', '/vagrant', disabled: true
```

VagrantBoxの設定全文を以下に示します．長いです．
configのリロードの前後にサービスの起動待ちの待機時間を設定してます．長めに設定しているので，環境によっては短めに設定できると思います．
```ruby
# enable gui
$gui = true

# ------------------------------------------------------------
# Description
# ------------------------------------------------------------
$description = <<"EOS"
# Cumulus and SONiC
## Cumulus
user: vagrant
password: vagrant
## SONiC
user: admin
password: YourPaSsWoRd
EOS

# ------------------------------------------------------------
# FRRouting Config
# ------------------------------------------------------------
$reset_sonic = <<-'SCRIPT'
sudo cat <<'EOF' | sudo tee /etc/sonic/config_db.json
{
  "DEVICE_METADATA": {
      "localhost": {
          "hwsku": "Force10-S6000",
          "platform": "x86_64-kvm_x86_64-r0",
          "mac": "08:00:27:f6:18:74",
          "hostname": "%{hostname}",
          "type": "LeafRouter"
      }
  },
  "PORT": {
      "Ethernet0": {
          "lanes": "25,26,27,28",
          "alias": "fortyGigE0/0",
          "index": "0",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet4": {
          "lanes": "29,30,31,32",
          "alias": "fortyGigE0/4",
          "index": "1",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet8": {
          "lanes": "33,34,35,36",
          "alias": "fortyGigE0/8",
          "index": "2",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet12": {
          "lanes": "37,38,39,40",
          "alias": "fortyGigE0/12",
          "index": "3",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet16": {
          "lanes": "45,46,47,48",
          "alias": "fortyGigE0/16",
          "index": "4",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet20": {
          "lanes": "41,42,43,44",
          "alias": "fortyGigE0/20",
          "index": "5",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet24": {
          "lanes": "1,2,3,4",
          "alias": "fortyGigE0/24",
          "index": "6",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet28": {
          "lanes": "5,6,7,8",
          "alias": "fortyGigE0/28",
          "index": "7",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet32": {
          "lanes": "13,14,15,16",
          "alias": "fortyGigE0/32",
          "index": "8",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet36": {
          "lanes": "9,10,11,12",
          "alias": "fortyGigE0/36",
          "index": "9",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet40": {
          "lanes": "17,18,19,20",
          "alias": "fortyGigE0/40",
          "index": "10",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet44": {
          "lanes": "21,22,23,24",
          "alias": "fortyGigE0/44",
          "index": "11",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet48": {
          "lanes": "53,54,55,56",
          "alias": "fortyGigE0/48",
          "index": "12",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet52": {
          "lanes": "49,50,51,52",
          "alias": "fortyGigE0/52",
          "index": "13",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet56": {
          "lanes": "57,58,59,60",
          "alias": "fortyGigE0/56",
          "index": "14",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet60": {
          "lanes": "61,62,63,64",
          "alias": "fortyGigE0/60",
          "index": "15",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet64": {
          "lanes": "69,70,71,72",
          "alias": "fortyGigE0/64",
          "index": "16",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet68": {
          "lanes": "65,66,67,68",
          "alias": "fortyGigE0/68",
          "index": "17",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet72": {
          "lanes": "73,74,75,76",
          "alias": "fortyGigE0/72",
          "index": "18",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet76": {
          "lanes": "77,78,79,80",
          "alias": "fortyGigE0/76",
          "index": "19",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet80": {
          "lanes": "109,110,111,112",
          "alias": "fortyGigE0/80",
          "index": "20",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet84": {
          "lanes": "105,106,107,108",
          "alias": "fortyGigE0/84",
          "index": "21",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet88": {
          "lanes": "113,114,115,116",
          "alias": "fortyGigE0/88",
          "index": "22",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet92": {
          "lanes": "117,118,119,120",
          "alias": "fortyGigE0/92",
          "index": "23",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet96": {
          "lanes": "125,126,127,128",
          "alias": "fortyGigE0/96",
          "index": "24",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet100": {
          "lanes": "121,122,123,124",
          "alias": "fortyGigE0/100",
          "index": "25",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet104": {
          "lanes": "81,82,83,84",
          "alias": "fortyGigE0/104",
          "index": "26",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet108": {
          "lanes": "85,86,87,88",
          "alias": "fortyGigE0/108",
          "index": "27",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet112": {
          "lanes": "93,94,95,96",
          "alias": "fortyGigE0/112",
          "index": "28",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet116": {
          "lanes": "89,90,91,92",
          "alias": "fortyGigE0/116",
          "index": "29",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet120": {
          "lanes": "101,102,103,104",
          "alias": "fortyGigE0/120",
          "index": "30",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      },
      "Ethernet124": {
          "lanes": "97,98,99,100",
          "alias": "fortyGigE0/124",
          "index": "31",
          "speed": "40000",
          "admin_status": "up",
          "mtu": "9100"
      }
  },
  "FLEX_COUNTER_TABLE": {
      "ACL": {
          "FLEX_COUNTER_STATUS": "disable",
          "FLEX_COUNTER_DELAY_STATUS": "true",
          "POLL_INTERVAL": "10000"
      }
  },
  "LOOPBACK_INTERFACE": {
      "Loopback0|10.1.0.1/32": {}
  },
  "INTERFACE": {
    "Ethernet0": {},
    "Ethernet4": {},
    "Ethernet8": {},
    "Ethernet12": {},
    "Ethernet16": {},
    "Ethernet20": {},
    "Ethernet24": {},
    "Ethernet28": {},
    "Ethernet32": {},
    "Ethernet36": {},
    "Ethernet40": {},
    "Ethernet44": {},
    "Ethernet48": {},
    "Ethernet52": {},
    "Ethernet56": {},
    "Ethernet60": {},
    "Ethernet64": {},
    "Ethernet68": {},
    "Ethernet72": {},
    "Ethernet76": {},
    "Ethernet80": {},
    "Ethernet84": {},
    "Ethernet88": {},
    "Ethernet92": {},
    "Ethernet96": {},
    "Ethernet100": {},
    "Ethernet104": {},
    "Ethernet108": {},
    "Ethernet112": {},
    "Ethernet116": {},
    "Ethernet120": {},
    "Ethernet124": {}
  }
}
EOF
# 起動待ち
sleep 4m
sudo config reload -y
# 起動待ち
sleep 2m
SCRIPT

$daemons = <<-'SCRIPT'
cat << 'EOF' | tee /etc/frr/daemons
zebra=yes
bgpd=yes
ospfd=no
ospf6d=no
ripd=no
ripngd=no
isisd=no
fabricd=no
pimd=no
ldpd=no
nhrpd=no
eigrpd=no
babeld=no
sharpd=no
pbrd=no
fabricd=no
vrrpd=no
vtysh_enable=yes
zebra_options="  -M cumulus_mlag -M snmp -A 127.0.0.1 -s 90000000"
bgpd_options="   -M snmp -A 127.0.0.1"
ospfd_options="  -M snmp -A 127.0.0.1"
ospf6d_options=" -M snmp -A ::1"
ripd_options="   -A 127.0.0.1"
ripngd_options=" -A ::1"
isisd_options="  -A 127.0.0.1"
pimd_options="   -A 127.0.0.1"
ldpd_options="   -A 127.0.0.1"
nhrpd_options="  -A 127.0.0.1"
eigrpd_options=" -A 127.0.0.1"
babeld_options=" -A 127.0.0.1"
sharpd_options=" -A 127.0.0.1"
pbrd_options="   -A 127.0.0.1"
staticd_options="-A 127.0.0.1"
fabricd_options="-A 127.0.0.1"
vrrpd_options="  -A 127.0.0.1"
EOF
systemctl restart frr
SCRIPT

$spine1_conf = <<-'CONFIG'
enable
configure terminal
interface lo
  ip address 10.0.0.1/32
interface Ethernet0
  ip address 10.1.0.1/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
interface Ethernet4
  ip address 10.2.0.1/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
router bgp 65200
  bgp router-id 10.0.0.1
  no bgp ebgp-requires-policy
  bgp bestpath as-path multipath-relax
  neighbor FABRIC peer-group
  neighbor FABRIC remote-as external
  neighbor FABRIC capability extended-nexthop
  neighbor Ethernet0 interface peer-group FABRIC
  neighbor Ethernet4 interface peer-group FABRIC
  address-family ipv4 unicast
    network 10.0.0.1/32
  exit-address-family
end
write mem
CONFIG

$leaf1_conf = <<-'CONFIG'
enable
configure terminal
interface lo
  ip address 10.0.1.1/32
interface swp1
  ip address 10.1.0.2/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
interface swp2
  ip address 192.168.1.1/24
  no shut
router bgp 65100
  bgp router-id 10.0.1.1
  no bgp ebgp-requires-policy
  bgp bestpath as-path multipath-relax
  neighbor FABRIC peer-group
  neighbor FABRIC remote-as external
  neighbor FABRIC capability extended-nexthop
  neighbor swp1 interface peer-group FABRIC
  neighbor swp2 interface peer-group FABRIC
  address-family ipv4 unicast
    network 10.0.1.1/32
    network 192.168.1.1/24
  exit-address-family
end
write mem
CONFIG

$leaf2_conf = <<-'CONFIG'
enable
configure terminal
interface lo
  ip address 10.0.1.2/32
interface swp1
  ip address 10.2.0.2/24
  ipv6 nd ra-interval 10
  no ipv6 nd suppress-ra
  no shut
interface swp2
  ip address 192.168.2.1/24
  no shut
router bgp 65101
  bgp router-id 10.0.1.2
  no bgp ebgp-requires-policy
  bgp bestpath as-path multipath-relax
  neighbor FABRIC peer-group
  neighbor FABRIC remote-as external
  neighbor FABRIC capability extended-nexthop
  neighbor swp1 interface peer-group FABRIC
  neighbor swp2 interface peer-group FABRIC
  address-family ipv4 unicast
    network 10.0.1.2/32
    network 192.168.2.1/24
  exit-address-family
end
write mem
CONFIG


def to_vtysh_cmd(s)
    "vtysh" + s.split("\n").map{|s| " -c \"" + s + "\""}.join()
end

# ------------------------------------------------------------
# Host Config
# ------------------------------------------------------------
$host1_conf = <<-'SCRIPT'
ip link set eth1 up
ip address add 192.168.1.2/24 dev eth1
ip route add 192.168.2.0/24 via 192.168.1.1 dev eth1
SCRIPT

$host2_conf = <<-'SCRIPT'
ip link set eth1 up
ip address add 192.168.2.2/24 dev eth1
ip route add 192.168.1.0/24 via 192.168.2.1 dev eth1
SCRIPT


# ------------------------------------------------------------
# vagrant configure version 2
# ------------------------------------------------------------
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

    config.vbguest.auto_update = false

    # 
    # SONiC
    # ------------------------------------------------------------
    spine1 = "clos-spine1"
    config.vm.define spine1 do |switch|
        switch.vm.box = "sonic-vs"

        switch.ssh.username = 'admin'
        switch.ssh.password = 'YourPaSsWoRd'
        switch.ssh.insert_key = false

        switch.vm.boot_timeout = 18000
        switch.vm.synced_folder '.', '/vagrant', disabled: true

        # private network config
        switch.vm.network "private_network", virtualbox__intnet: "net1", auto_config: false
        switch.vm.network "private_network", virtualbox__intnet: "net2", auto_config: false

        # # set network config
        switch.vm.provision "shell", inline: $reset_sonic % {hostname: spine1}
        switch.vm.provision "shell", inline: to_vtysh_cmd($spine1_conf)

        # virtual box config
        switch.vm.provider "virtualbox" do |vb|
            vb.name = spine1
            vb.gui = $gui
            vb.cpus = 1
            vb.memory = 2048
            vb.customize [
                "modifyvm", :id,
                "--vram", "16", 
                "--clipboard", "bidirectional",
                "--draganddrop", "bidirectional",
                "--ioapic", "off",
                "--graphicscontroller", "vmsvga",
                "--accelerate3d", "off",
                "--hwvirtex", "on",
                "--nestedpaging", "on",
                "--largepages", "on",
                "--pae", "on",
                "--audio", "none",
                "--chipset", "piix3",
                "--uartmode1", "disconnected",
                "--description", $description
            ]

            (1..switch.vm.networks.count {|nw| !nw.include? :forwarded_port}).each do |i|
                vb.customize ["modifyvm", :id, "--nicpromisc#{i+1}", "allow-all"]
            end
        end
      end

    # 
    # Cumulus Linux
    # ------------------------------------------------------------
    leaf1 = "clos-leaf1"
    config.vm.define leaf1 do |switch|
        switch.vm.hostname = leaf1
        switch.vm.box = "CumulusCommunity/cumulus-vx"

        # private network config
        switch.vm.network "private_network", virtualbox__intnet: "net1", auto_config: false
        switch.vm.network "private_network", virtualbox__intnet: "net3", auto_config: false

        # set network config
        switch.vm.provision "shell", inline: $daemons
        switch.vm.provision "shell", inline: to_vtysh_cmd($leaf1_conf)

        # virtual box config
        switch.vm.provider "virtualbox" do |vb|
            vb.name = leaf1
            vb.gui = $gui
            vb.cpus = 1
            vb.memory = 1024
            vb.customize [
                "modifyvm", :id,
                "--vram", "16", 
                "--clipboard", "bidirectional",
                "--draganddrop", "bidirectional",
                "--ioapic", "on",
                "--graphicscontroller", "vmsvga",
                "--accelerate3d", "off",
                "--hwvirtex", "on",
                "--nestedpaging", "on",
                "--largepages", "on",
                "--pae", "on",
                "--audio", "none",
                "--chipset", "ich9",
                "--description", $description
            ]

            (1..switch.vm.networks.count {|nw| !nw.include? :forwarded_port}).each do |i|
                vb.customize ["modifyvm", :id, "--nicpromisc#{i+1}", "allow-all"]
            end
        end
    end

    leaf2 = "clos-leaf2"
    config.vm.define leaf2 do |switch|
        switch.vm.hostname = leaf2
        switch.vm.box = "CumulusCommunity/cumulus-vx"

        # private network config
        switch.vm.network "private_network", virtualbox__intnet: "net2", auto_config: false
        switch.vm.network "private_network", virtualbox__intnet: "net4", auto_config: false

        # set network config
        switch.vm.provision "shell", inline: $daemons
        switch.vm.provision "shell", inline: to_vtysh_cmd($leaf2_conf)

        # virtual box config
        switch.vm.provider "virtualbox" do |vb|
            vb.name = leaf2
            vb.gui = $gui
            vb.cpus = 1
            vb.memory = 1024
            vb.customize [
                "modifyvm", :id,
                "--vram", "16", 
                "--clipboard", "bidirectional",
                "--draganddrop", "bidirectional",
                "--ioapic", "on",
                "--graphicscontroller", "vmsvga",
                "--accelerate3d", "off",
                "--hwvirtex", "on",
                "--nestedpaging", "on",
                "--largepages", "on",
                "--pae", "on",
                "--audio", "none",
                "--chipset", "ich9",
                "--description", $description
            ]

            (1..switch.vm.networks.count {|nw| !nw.include? :forwarded_port}).each do |i|
                vb.customize ["modifyvm", :id, "--nicpromisc#{i+1}", "allow-all"]
            end
        end
    end

    # 
    # Ubntu
    # ------------------------------------------------------------
    host1 = "clos-host1"
    config.vm.define host1 do |host|
        host.vm.hostname = host1
        host.vm.box = 'bento/ubuntu-20.04'

        # private network config
        host.vm.network "private_network", virtualbox__intnet: "net3", auto_config: false

        host.vm.provision "shell", inline: $host1_conf

        # virtual box config
        host.vm.provider "virtualbox" do |vb|
            vb.name = host1
            vb.gui = $gui
            vb.cpus = 1
            vb.memory = 1024
            vb.customize [
                "modifyvm", :id,
                "--vram", "16", 
                "--clipboard", "bidirectional",
                "--draganddrop", "bidirectional",
                "--ioapic", "on",
                "--graphicscontroller", "vmsvga",
                "--accelerate3d", "off",
                "--hwvirtex", "on",
                "--nestedpaging", "on",
                "--largepages", "on",
                "--pae", "on",
                "--audio", "none",
                "--chipset", "ich9",
                "--description", $description
            ]

            (1..host.vm.networks.count {|nw| !nw.include? :forwarded_port}).each do |i|
                vb.customize ["modifyvm", :id, "--nicpromisc#{i+1}", "allow-all"]
            end
        end
    end

    host2 = "clos-host2"
    config.vm.define host2 do |host|
        host.vm.hostname = host2
        host.vm.box = 'bento/ubuntu-20.04'

        # private network config
        host.vm.network "private_network", virtualbox__intnet: "net4", auto_config: false

        host.vm.provision "shell", inline: $host2_conf

        # virtual box config
        host.vm.provider "virtualbox" do |vb|
            vb.name = host2
            vb.gui = $gui
            vb.cpus = 1
            vb.memory = 1024
            vb.customize [
                "modifyvm", :id,
                "--vram", "16", 
                "--clipboard", "bidirectional",
                "--draganddrop", "bidirectional",
                "--ioapic", "on",
                "--graphicscontroller", "vmsvga",
                "--accelerate3d", "off",
                "--hwvirtex", "on",
                "--nestedpaging", "on",
                "--largepages", "on",
                "--pae", "on",
                "--audio", "none",
                "--chipset", "ich9",
                "--description", $description
            ]

            (1..host.vm.networks.count {|nw| !nw.include? :forwarded_port}).each do |i|
                vb.customize ["modifyvm", :id, "--nicpromisc#{i+1}", "allow-all"]
            end
        end
    end
end
```

## 実行
では，`vagrant up`で実行します．

### spine1 (SONiC)
起動後，`vagrant ssh clos-spine1`でSONiCにログインします．繰り返しですが，パスワードは`YourPaSsWoRd`です．

```bash
$ vagrant ssh clos-spine1
==> clos-spine1: The machine you're attempting to SSH into is configured to use
==> clos-spine1: password-based authentication. Vagrant can't script entering the
==> clos-spine1: password for you. If you're prompted for a password, please enter
==> clos-spine1: the same password you have configured in the Vagrantfile.
admin@127.0.0.1's password: 
Linux clos-spine1 5.10.0-12-2-amd64 #1 SMP Debian 5.10.103-1 (2022-03-07) x86_64
You are on
  ____   ___  _   _ _  ____
 / ___| / _ \| \ | (_)/ ___|
 \___ \| | | |  \| | | |
  ___) | |_| | |\  | | |___
 |____/ \___/|_| \_|_|\____|

-- Software for Open Networking in the Cloud --

Unauthorized access and/or use are prohibited.
All access and/or use are subject to monitoring.

Help:    http://azure.github.io/SONiC/

Last login: Tue May 31 08:19:37 2022 from 10.0.2.2
admin@clos-spine1:~$ 
```

`vtysh`でBGPの確認とルーティングテーブルの確認をします．
```bash
clos-spine1# show bgp summary 

IPv4 Unicast Summary:
BGP router identifier 10.0.0.1, local AS number 65200 vrf-id 0
BGP table version 13
RIB entries 9, using 1728 bytes of memory
Peers 2, using 43 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
Ethernet0       4      65100       152       151        0    0    0 00:06:45            2        5
Ethernet4       4      65101       126       126        0    0    0 00:05:24            2        5

Total number of neighbors 2
clos-spine1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

K>* 0.0.0.0/0 [0/0] via 10.0.2.2, eth0, 00:09:31
C>* 10.0.0.1/32 is directly connected, lo, 00:07:56
B>* 10.0.1.1/32 [20/0] via fe80::a00:27ff:fe8f:18b1, Ethernet0, weight 1, 00:06:50
B>* 10.0.1.2/32 [20/0] via fe80::a00:27ff:fecc:5d9, Ethernet4, weight 1, 00:05:28
C>* 10.0.2.0/24 is directly connected, eth0, 00:09:31
C>* 10.1.0.0/24 is directly connected, Ethernet0, 00:07:56
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:09:01
C>* 10.2.0.0/24 is directly connected, Ethernet4, 00:07:56
B>* 192.168.1.0/24 [20/0] via fe80::a00:27ff:fe8f:18b1, Ethernet0, weight 1, 00:06:50
B>* 192.168.2.0/24 [20/0] via fe80::a00:27ff:fecc:5d9, Ethernet4, weight 1, 00:05:28
clos-spine1# 
```

### leaf1 (Cumulus Linux)
leaf1も同様に，ログインして，BGPとルーティングを確認します．Cumulusは，ちゃんとした(？)Vagrant Boxなので，パスワードなどはvagrantです．
```bash
clos-leaf1# show bgp su

IPv4 Unicast Summary:
BGP router identifier 10.0.1.1, local AS number 65100 vrf-id 0
BGP table version 9
RIB entries 9, using 1800 bytes of memory
Peers 2, using 46 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor          V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
clos-spine1(swp1) 4      65200       201       201        0    0    0 00:09:41            3        5
swp2              4          0         0         0        0    0    0    never         Idle        0

Total number of neighbors 2
clos-leaf1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, Z - FRR,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
B>* 10.0.0.1/32 [20/0] via fe80::a00:27ff:fef6:1874, swp1, weight 1, 00:09:53
C>* 10.0.1.1/32 is directly connected, lo, 00:09:57
B>* 10.0.1.2/32 [20/0] via fe80::a00:27ff:fef6:1874, swp1, weight 1, 00:08:31
C>* 10.1.0.0/24 is directly connected, swp1, 00:09:57
C>* 192.168.1.0/24 is directly connected, swp2, 00:09:57
B>* 192.168.2.0/24 [20/0] via fe80::a00:27ff:fef6:1874, swp1, weight 1, 00:08:31
clos-leaf1# 
```

### 疎通確認
最後に，host1からhost2へPingが通るかを確かめます．
```bash
vagrant@clos-host1:~$ ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_seq=1 ttl=61 time=63.8 ms
64 bytes from 192.168.2.2: icmp_seq=2 ttl=61 time=44.8 ms
64 bytes from 192.168.2.2: icmp_seq=3 ttl=61 time=49.8 ms
64 bytes from 192.168.2.2: icmp_seq=4 ttl=61 time=13.5 ms
64 bytes from 192.168.2.2: icmp_seq=5 ttl=61 time=26.7 ms
^C
--- 192.168.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4014ms
rtt min/avg/max/mdev = 13.539/39.726/63.798/17.684 ms
```
通りました．

## おわりに

今回は，SONiCをVirtaulBox + Vagrant で動かしてみました．参考になれば幸いです．

## 参考文献
- [SONiCを少しかじってみた](https://debslink.hatenadiary.jp/entry/20210131/1612091391)
- [SONiCとCumulus LinuxでBGP unnumbered](https://debslink.hatenadiary.jp/entry/20210822/1629600666)
- [Edgecore SONiC BGP Unnumbered](https://support.edge-core.com/hc/en-us/articles/900002377366--Edgecore-SONiC-BGP-Unnumbered)
- [Vagrant, Creating a Base Box](https://www.vagrantup.com/docs/providers/virtualbox/boxes)
- [VIRTUALBOX から VAGRANT の BOX を作成する](http://vistylee.com/virtualbox-vagrant-box/)