Repository: CrowdStrike/caracara

Issue:
[ BUG ] 409 Rule group version mismatch error when updating rule group
# Bug Report Template

Rule group version mismatch error when updating IOA rule group rules.

## Describe the bug

I receive a 409 'rule group version mismatch, reload data and try again' when attempting to update existing rules for an IOA rule group.

It appears to be due to the client locally incrementing the version when submitting `custom_ioa_api.update_rules()`.

## To Reproduce

```python
import logging
import os

import caracara

logging.basicConfig(level=logging.INFO)

client = caracara.Client(
    client_id=os.getenv('FALCON_CLIENT_ID'),
    client_secret=os.getenv('FALCON_CLIENT_SECRET'),
)

rule_group = client.custom_ioas.describe_rule_groups()["effa63e3d7ce4a95aa6abdb128225ecc"]

client.custom_ioas.update_rule_group(rule_group)
```

```
INFO:caracara.client:Setting up the Caracara client and configuring authentication
INFO:caracara.client:Client ID: XXXXX; Cloud: auto; Member CID: None
INFO:caracara.client:Base URL: US1
INFO:caracara.client:Requesting API token
INFO:caracara.client:Resolved Base URL: https://api.us-2.crowdstrike.com
INFO:caracara.client:Caracara client configured
INFO:caracara.modules.CustomIoaApiModule:Describing all Falcon IOA Rule Groups matching filter: None
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Pagination Style 1: Grabbing all pages from the partial handle_errors function (limit: 100)
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Grabbing first batch of items 1 to up to 100
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Retrieved a batch of 3 items
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Total number of resources: 3
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Pagination Style 1: Grabbing all pages from the handle_errors function (limit: 100)
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Grabbing first batch of items 1 to up to 100
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Retrieved a batch of 9 items
INFO:caracara.modules.CustomIoaApiModule.caracara.common.pagination:Total number of resources: 9
INFO:caracara.common.batching:Batch data retrieval for handle_errors (9 items)
INFO:caracara.common.batching:Divided the item IDs into 1 batches
INFO:caracara.common.batching:ThreadPoolExecutor-0_0 | Batch worker started with a list of 9 items. Function: handle_errors
INFO:caracara.common.batching:ThreadPoolExecutor-0_0 | Retrieved 9 resources
Traceback (most recent call last):
  File "/home/zaci/PycharmProjects/Caracara-Bug/scripts/reproduce.py", line 15, in <module>
    client.custom_ioas.update_rule_group(rule_group)
  File "/home/zaci/PycharmProjects/Caracara-Bug/.venv/lib/python3.10/site-packages/caracara/modules/custom_ioa/custom_ioa.py", line 134, in update_rule_group
    new_group = self._create_update_delete_rules(new_group, comment=comment)
  File "/home/zaci/PycharmProjects/Caracara-Bug/.venv/lib/python3.10/site-packages/caracara/modules/custom_ioa/custom_ioa.py", line 183, in _create_update_delete_rules
    response = instr(self.custom_ioa_api.update_rules)(body={
  File "/home/zaci/PycharmProjects/Caracara-Bug/.venv/lib/python3.10/site-packages/caracara/modules/custom_ioa/custom_ioa.py", line 29, in handle_errors
    raise ValueError(errors)
ValueError: [{'code': 409, 'message': 'rule group version mismatch, reload data and try again'}]
```

![image](https://github.com/user-attachments/assets/175ef143-b9d0-475a-a0d8-5a715ae25c2e)

## Expected behavior

The version id should not be incremented when there are no changes to the rule group.

## Environment

### Operating System Version

Linux arch 6.11.2-arch1-1 #\1 SMP PREEMPT_DYNAMIC Fri, 04 Oct 2024 21:51:11 +0000 x86_64 GNU/Linux

### Python Version

Python 3.10.14

### Poetry Version

1.8.3

### Python Package Versions

```
caracara==0.8.0
caracara-filters==0.2.0
crowdstrike-falconpy==1.4.5
```

## Additional context

I am able to stop the error by removing the increment in custom_ioa.py.

https://github.com/CrowdStrike/caracara/blob/507b3780d67a6830b1df3dee58a6863723b8b8dd/caracara/modules/custom_ioa/custom_ioa.py#L186

```diff
        # Update the existing rules, if there are any
        if len(existing_rules) > 0:
            response = instr(self.custom_ioa_api.update_rules)(body={
                "comment": comment,
                "rule_updates": [rule.dump_update() for rule in existing_rules],
-                "rulegroup_version": group.version + 1,
+                "rulegroup_version": group.version,
                "rulegroup_id": group.id_,
            })
            raw_group = response["body"]["resources"][0]
```

Discussion:
@ChristopherHammond13: Thank you so much for raising this! You're right that we do not handle the case where a rule is submitted but has not been changed in this library. Assigning myself because this is a case that we should handle.

Solution patch:
diff --git a/caracara/modules/custom_ioa/custom_ioa.py b/caracara/modules/custom_ioa/custom_ioa.py
index 6e59240..42860c2 100644
--- a/caracara/modules/custom_ioa/custom_ioa.py
+++ b/caracara/modules/custom_ioa/custom_ioa.py
@@ -1,6 +1,7 @@
 """Caracara Indicator of Attack (IOA) API module."""
 
 from functools import partial
+from itertools import chain
 from time import monotonic
 from typing import Dict, List, Union
 
@@ -96,7 +97,7 @@ class CustomIoaApiModule(FalconApiModule):
             rule.group = new_group
 
         # Update the rules
-        new_group = self._create_update_delete_rules(new_group, comment=comment)
+        new_group = self._update_create_delete_rules(new_group, comment=comment)
 
         return new_group
 
@@ -138,7 +139,7 @@ class CustomIoaApiModule(FalconApiModule):
         new_group.rules_to_delete = group.rules_to_delete
 
         # Update the rules
-        new_group = self._create_update_delete_rules(new_group, comment=comment)
+        new_group = self._update_create_delete_rules(new_group, comment=comment)
 
         return new_group
 
@@ -167,7 +168,7 @@ class CustomIoaApiModule(FalconApiModule):
                 ids_to_delete.append(rule_group)
         instr(self.custom_ioa_api.delete_rule_groups)(ids=ids_to_delete, comment=comment)
 
-    def _create_update_delete_rules(self, group: IoaRuleGroup, comment: str) -> IoaRuleGroup:
+    def _update_create_delete_rules(self, group: IoaRuleGroup, comment: str) -> IoaRuleGroup:
         existing_rules = []
         to_be_created = []
         for rule in group.rules:
@@ -176,25 +177,13 @@ class CustomIoaApiModule(FalconApiModule):
             else:
                 to_be_created.append(rule)
 
-        # Create the new rules
-        new_rules = []
-        for rule in to_be_created:
-            resp = instr(self.custom_ioa_api.create_rule)(body=rule.dump_create(comment=comment))
-            raw_rule = resp["body"]["resources"][0]
-            new_rule = CustomIoaRule.from_data_dict(
-                raw_rule,
-                rule_type=self._get_rule_types_cached()[raw_rule["ruletype_id"]],
-            )
-            new_rule.rulegroup_id = group.id_
-            new_rules.append(new_rule)
-
         # Update the existing rules, if there are any
         if len(existing_rules) > 0:
             response = instr(self.custom_ioa_api.update_rules)(
                 body={
                     "comment": comment,
                     "rule_updates": [rule.dump_update() for rule in existing_rules],
-                    "rulegroup_version": group.version + 1,
+                    "rulegroup_version": group.version,
                     "rulegroup_id": group.id_,
                 }
             )
@@ -204,9 +193,28 @@ class CustomIoaApiModule(FalconApiModule):
             rule_types = self._get_rule_types_cached()
             new_group = IoaRuleGroup.from_data_dict(data_dict=raw_group, rule_type_map=rule_types)
         else:
-            group.rules = new_rules
             new_group = group
 
+        # Create the new rules
+        new_rules = []
+        for rule in to_be_created:
+            resp = instr(self.custom_ioa_api.create_rule)(body=rule.dump_create(comment=comment))
+            raw_rule = resp["body"]["resources"][0]
+            new_rule = CustomIoaRule.from_data_dict(
+                raw_rule,
+                rule_type=self._get_rule_types_cached()[raw_rule["ruletype_id"]],
+            )
+            new_rule.rulegroup_id = group.id_
+            new_rules.append(new_rule)
+            new_group.version += 1
+
+        new_group.rules = list(
+            chain(
+                (rule for rule in group.rules if rule.exists_in_cloud()),
+                (new_rule for rule in new_rules),
+            )
+        )
+
         # Delete rules queued for deletion, if any
         if len(group.rules_to_delete) > 0:
             ids_to_delete = [rule.instance_id for rule in group.rules_to_delete]
@@ -215,6 +223,8 @@ class CustomIoaApiModule(FalconApiModule):
             )
             # If successful (i.e. no exceptions raised), clear the deletion queue
             group.rules_to_delete = []
+            new_group.version += 1
+
         return new_group
 
     @filter_string
diff --git a/caracara/modules/custom_ioa/rules.py b/caracara/modules/custom_ioa/rules.py
index 5314e48..881636a 100644
--- a/caracara/modules/custom_ioa/rules.py
+++ b/caracara/modules/custom_ioa/rules.py
@@ -262,7 +262,7 @@ class IoaRuleGroup:
         return {
             "comment": comment,
             "rule_updates": [rule.dump_update(group=self) for rule in self.rules],
-            "rulegroup_version": self.version + 1,
+            "rulegroup_version": self.version,
             "rulegroup_id": self.id_,
         }
 


Test patch:
diff --git a/tests/unit_tests/test_custom_ioas.py b/tests/unit_tests/test_custom_ioas.py
index 647496c..b94361a 100644
--- a/tests/unit_tests/test_custom_ioas.py
+++ b/tests/unit_tests/test_custom_ioas.py
@@ -575,14 +575,15 @@ def test_update_rule_groups_with_rule_changes(
         raw_group["description"] = body["description"]
         raw_group["enabled"] = body["enabled"]
         raw_group["comment"] = body["comment"]
+        raw_group["version"] += 1
         return {"body": {"resources": [raw_group]}}
 
     custom_ioa_api.update_rule_group.side_effect = mock_update_rule_group
 
     def mock_update_rules(body):
         assert body["rulegroup_id"] == raw_group["id"]
-        assert body["rulegroup_version"] == raw_group["version"] + 1
-        raw_group["version"] = body["rulegroup_version"]
+        assert body["rulegroup_version"] == raw_group["version"]
+        raw_group["version"] += 1
         for raw_rule_update in body["rule_updates"]:
             matching_rules = [
                 i
@@ -604,6 +605,7 @@ def test_update_rule_groups_with_rule_changes(
 
     def mock_create_rule(body):
         assert raw_group["id"] == body["rulegroup_id"]
+        raw_group["version"] += 1
         new_rule = {
             "customer_id": "test_customer",
             "instance_id": "test_rule_03",
@@ -632,6 +634,15 @@ def test_update_rule_groups_with_rule_changes(
         raw_group["rules"].append(new_rule)
         return {"body": {"resources": [new_rule]}}
 
+    def mock_delete_rule(rule_group_id, ids, comment):
+        assert raw_group["id"] == rule_group_id
+        assert ids
+        assert comment
+        raw_group["version"] += 1
+        return {"body": {}}
+
+    custom_ioa_api.delete_rules.side_effect = mock_delete_rule
+
     custom_ioa_api.create_rule.side_effect = mock_create_rule
 
     custom_ioa_api.query_rule_types.side_effect = create_mock_query_resources(
@@ -647,8 +658,8 @@ def test_update_rule_groups_with_rule_changes(
     # Assert falconpy called correctly
     # This consists of
     # - A rule group update
-    # - A rule deletion
     # - A rule update
+    # - A rule deletion
     # - A rule creation
     custom_ioa_api.update_rule_group.assert_called_once_with(
         body={
@@ -660,11 +671,6 @@ def test_update_rule_groups_with_rule_changes(
             "comment": "test update comment",
         }
     )
-    custom_ioa_api.delete_rules.assert_called_once_with(
-        rule_group_id="test_group_01",
-        ids=["test_rule_01"],
-        comment="test update comment",
-    )
     custom_ioa_api.update_rules.assert_called_once_with(
         body={
             "rulegroup_id": "test_group_01",
@@ -695,5 +701,10 @@ def test_update_rule_groups_with_rule_changes(
             "comment": "test update comment",
         }
     )
+    custom_ioa_api.delete_rules.assert_called_once_with(
+        rule_group_id="test_group_01",
+        ids=["test_rule_01"],
+        comment="test update comment",
+    )
     # Assert new group is as expected
-    assert new_group.version == group.version + 1
+    assert new_group.version == group.version + 4