Repository: sonic-net/sonic-utilities

Issue:
[GCU] bgp speaker rollback failure 
<!--
    If you are reporting a new issue, make sure that we do not have any duplicates
    already open. You can ensure this by searching the issue list for this
    repository. If there is a duplicate, please close your issue and add a comment
    to the existing issue instead.

    If you suspect your issue is a bug, please edit your issue description to
    include the BUG REPORT INFORMATION shown below. If you fail to provide this
    information within 7 days, we cannot debug your issue and will close it. We
    will, however, reopen it if you later provide the information.

    For more information about reporting issues, see
    https://github.com/Azure/SONiC/wiki#report-issues

    ---------------------------------------------------
    GENERAL SUPPORT INFORMATION
    ---------------------------------------------------

    The GitHub issue tracker is for bug reports and feature requests.
    General support can be found at the following locations:

    - SONiC Support Forums - https://groups.google.com/forum/#!forum/sonicproject

    ---------------------------------------------------
    BUG REPORT INFORMATION
    ---------------------------------------------------
    Use the commands below to provide key information from your environment:
    You do NOT have to include this information if this is a FEATURE REQUEST
-->

#### Description
SONiC doesn't allow two bgp speaker have the same listen range. But it might happen during the GCU.
Let's say `SPEAKER_CONFIG_A` has `IP_RANGE_A`. We want to replace it with `SPEAKER_CONFIG_B` with `IP_RANGE_A`.
If the GCU sorting algorithm give out this patch order: 
`{op_1: add SPEAKER_CONFIG_B; op_2: remove SPEAKER_CONFIG_A}` 
Then after the op_1, there are two config share the same ip range, which result in the failure.
<!--
     Briefly describe the problem you are having in a few paragraphs.
-->

## Steps to reproduce the issue
1. Env: kvm-t0.
original speaker config:
```
admin@vlab-01:~/speaker/test$ sudo config checkpoint init

{
"BGP_PEER_RANGE": {
        "BGPSLBPassive": {
            "ip_range": [
                "10.255.0.0/25"
            ],
            "name": "BGPSLBPassive",
            "src_address": "10.1.0.32"
        },
        "BGPVac": {
            "ip_range": [
                "192.168.0.0/21"
            ],
            "name": "BGPVac",
            "src_address": "10.1.0.32"
        }
    }
}
```
2. Apply test config to speaker
```

admin@vlab-01:~/speaker/test$ cat tc_add.json
[
 {
  "op": "add",
  "path": "/BGP_PEER_RANGE",
  "value": {
   "BGPSLBPassive": {
    "ip_range": [
     "192.168.0.0/21"
    ],
    "name": "BGPSLBPassive",
    "src_address": "10.1.0.32"
   },
   "BGPSLBPassiveV6": {
    "ip_range": [
     "fc02:1000::/64"
    ],
    "name": "BGPSLBPassiveV6",
    "src_address": "fc00:1::32"
   }
  }
 }
]
admin@vlab-01:~/speaker/test$ sudo config apply-patch tc_add.json
...
Patch Applier: Applying 4 changes in order:
Patch Applier:   * [{"op": "remove", "path": "/BGP_PEER_RANGE/BGPVac"}]
Patch Applier:   * [{"op": "add", "path": "/BGP_PEER_RANGE/BGPSLBPassiveV6", "value": {"ip_range": ["fc02:1000::/64"], "name": "BGPSLBPassiveV6", "src_address": "fc00:1::32"}}]
Patch Applier:   * [{"op": "remove", "path": "/BGP_PEER_RANGE/BGPSLBPassive"}]
Patch Applier:   * [{"op": "add", "path": "/BGP_PEER_RANGE/BGPSLBPassive", "value": {"ip_range": ["192.168.0.0/21"], "name": "BGPSLBPassive", "src_address": "10.1.0.32"}}]
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
```
Config after the patch:
```
{
  "BGP_PEER_RANGE": {
        "BGPSLBPassive": {
            "ip_range": [
                "192.168.0.0/21"
            ],
            "name": "BGPSLBPassive",
            "src_address": "10.1.0.32"
        },
        "BGPSLBPassiveV6": {
            "ip_range": [
                "fc02:1000::/64"
            ],
            "name": "BGPSLBPassiveV6",
            "src_address": "fc00:1::32"
        }
    }
}
```

3. rollback to init config and check syslog err
```
admin@vlab-01:~/speaker/test$ sudo config rollback init
Patch Applier: Applying 4 changes in order:
Patch Applier:   * [{"op": "remove", "path": "/BGP_PEER_RANGE/BGPSLBPassiveV6"}]

========Below "add" is ahead of "remove" operation results in two same ip range occur at the same time, which is incorrect======

Patch Applier:   * [{"op": "add", "path": "/BGP_PEER_RANGE/BGPVac", "value": {"ip_range": ["192.168.0.0/21"], "name": "BGPVac", "src_address": "10.1.0.32"}}]
Patch Applier:   * [{"op": "remove", "path": "/BGP_PEER_RANGE/BGPSLBPassive"}]
Patch Applier:   * [{"op": "add", "path": "/BGP_PEER_RANGE/BGPSLBPassive", "value": {"ip_range": ["10.255.0.0/25"], "name": "BGPSLBPassive", "src_address": "10.1.0.32"}}]
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Rollbacker: Config rollbacking completed.
Config rolled back successfully.

admin@vlab-01:~$ show logging -f | grep ERR
...
Mar 28 07:45:54.686979 vlab-01 ERR bgp#bgpcfgd: command execution returned 13. Command: '['vtysh', '-f', '/tmp/tmp5d_n7k5w']', stdout: '% Same listen range is attached to peer-group BGPSLBPassive#012', stderr: 'line 37: Failure to communicate[13] to bgpd, line:   bgp listen range 192.168.0.0/21 peer-group BGPVac#012#012'
Mar 28 07:45:54.686979 vlab-01 ERR bgp#bgpcfgd: ConfigMgr::commit(): can't push configuration from file='/tmp/tmp5d_n7k5w', rc='13', stdout='% Same listen range is attached to peer-group BGPSLBPassive#012', stderr='line 37: Failure to communicate[13] to bgpd, line:   bgp listen range 192.168.0.0/21 peer-group BGPVac#012#012'

```

#### Describe the results you received


#### Describe the results you expected


#### Additional information you deem important (e.g. issue happens only occasionally)


#### Output of `show version`

```
(paste your output here)
```

<!--
     Also attach debug file produced by `sudo generate_dump`
-->




Discussion:
This is a YANG validation issue, it is not correctly validating the ip_ranges are different. We need to create an issue to fix the yang. If the YANG issue will take a long time to fix, we can hard-code the fix in GCU similar to what we did for the lanes validator, and remove the hard-coding once yang model validation isupdated.
Link yang issue here.
https://github.com/Azure/sonic-buildimage/issues/10376

Resolving patch:
diff --git a/generic_config_updater/gu_common.py b/generic_config_updater/gu_common.py
--- a/generic_config_updater/gu_common.py
+++ b/generic_config_updater/gu_common.py
@@ -118,10 +118,33 @@ def validate_config_db_config(self, config_db_as_json):
             sy.loadData(tmp_config_db_as_json)
 
             sy.validate_data_tree()
-            return True, None
+
+            # TODO: modularize custom validations better or move directly to sonic-yang module
+            return self.validate_bgp_peer_group(config_db_as_json)
         except sonic_yang.SonicYangException as ex:
             return False, ex
 
+    def validate_bgp_peer_group(self, config_db):
+        if "BGP_PEER_RANGE" not in config_db:
+            return True, None
+
+        visited = {}
+        table = config_db["BGP_PEER_RANGE"]
+        for peer_group_name in table:
+            peer_group = table[peer_group_name]
+            if "ip_range" not in peer_group:
+                continue
+
+            # TODO: convert string to IpAddress object for better handling of IPs
+            # TODO: validate range intersection
+            ip_range = peer_group["ip_range"]
+            for ip in ip_range:
+                if ip in visited:
+                    return False, f"{ip} is duplicated in BGP_PEER_RANGE: {set([peer_group_name, visited[ip]])}"
+                visited[ip] = peer_group_name
+
+        return True, None
+
     def crop_tables_without_yang(self, config_db_as_json):
         sy = self.create_sonic_yang_with_loaded_models()
 


Test patch:
diff --git a/tests/generic_config_updater/gu_common_test.py b/tests/generic_config_updater/gu_common_test.py
--- a/tests/generic_config_updater/gu_common_test.py
+++ b/tests/generic_config_updater/gu_common_test.py
@@ -186,6 +186,80 @@ def test_validate_config_db_config__invalid_config__returns_false(self):
         self.assertEqual(expected, actual)
         self.assertIsNotNone(error)
 
+    def test_validate_bgp_peer_group__valid_non_intersecting_ip_ranges__returns_true(self):
+        # Arrange
+        config_wrapper = gu_common.ConfigWrapper()
+        config = {
+            "BGP_PEER_RANGE":
+            {
+                "BGPSLBPassive": {
+                    "ip_range": ["1.1.1.1/31", "10.10.10.10/16", "100.100.100.100/24"]
+                },
+                "BgpVac": {
+                    "ip_range": ["2.2.2.2/31", "20.20.20.20/16", "200.200.200.200/24"]
+                }
+            }
+        }
+
+        # Act
+        actual, error = config_wrapper.validate_bgp_peer_group(config)
+
+        # Assert
+        self.assertTrue(actual)
+        self.assertIsNone(error)
+
+    def test_validate_bgp_peer_group__same_ip_prefix__return_false(self):
+        # duplicate v4 within same ip_range
+        self.check_validate_bgp_peer_group(
+            ["1.1.1.1/16", "1.1.1.1/16"],
+            duplicated_ip="1.1.1.1/16")
+        # duplicate v4 within different ip_ranges
+        self.check_validate_bgp_peer_group(
+            ["1.1.1.1/16"],
+            ["1.1.1.1/16"],
+            duplicated_ip="1.1.1.1/16")
+        # duplicate v4 within different ip_ranges, but many ips
+        self.check_validate_bgp_peer_group(
+            ["1.1.1.1/16", "1.1.1.1/31", "10.10.10.10/16", "100.100.100.100/24"],
+            ["2.2.2.2/31", "20.20.20.20/16", "200.200.200.200/24", "1.1.1.1/16"],
+            duplicated_ip="1.1.1.1/16")
+        # duplicate v6 within same ip_range
+        self.check_validate_bgp_peer_group(
+            ["fc00:1::32/16", "fc00:1::32/16"],
+            duplicated_ip="fc00:1::32/16")
+        # duplicate v6 within different ip_ranges
+        self.check_validate_bgp_peer_group(
+            ["fc00:1::32/16"],
+            ["fc00:1::32/16"],
+            duplicated_ip="fc00:1::32/16")
+        # duplicate v6 within different ip_ranges, but many ips
+        self.check_validate_bgp_peer_group(
+            ["fc00:1::32/16", "fc00:1::32/31", "10:1::1/16", "100:1::1/24"],
+            ["2:1::1/31", "20:1::1/16", "200:1::1/24", "fc00:1::32/16"],
+            duplicated_ip="fc00:1::32/16")
+
+    def check_validate_bgp_peer_group(self, ip_range, other_ip_range=[], duplicated_ip=None):
+        # Arrange
+        config_wrapper = gu_common.ConfigWrapper()
+        config = {
+            "BGP_PEER_RANGE":
+            {
+                "BGPSLBPassive": {
+                    "ip_range": ip_range
+                },
+                "BgpVac": {
+                    "ip_range": other_ip_range
+                },
+            }
+        }
+
+        # Act
+        actual, error = config_wrapper.validate_bgp_peer_group(config)
+
+        # Assert
+        self.assertFalse(actual)
+        self.assertTrue(duplicated_ip in error)
+
     def test_crop_tables_without_yang__returns_cropped_config_db_as_json(self):
         # Arrange
         config_wrapper = gu_common.ConfigWrapper()