Repository: nautobot/nautobot

Issue:
Dynamic Group issues after object modification
<!--
    NOTE: IF YOUR ISSUE DOES NOT FOLLOW THIS TEMPLATE, IT WILL BE CLOSED.

    This form is only for reporting reproducible bugs. If you need assistance
    with Nautobot installation, or if you have a general question, please start a
    discussion instead: https://github.com/nautobot/nautobot/discussions

    Please describe the environment in which you are running Nautobot. Be sure
    that you are running an unmodified instance of the latest stable release
    before submitting a bug report, and that any plugins have been disabled.
-->
### Environment
* Nautobot version (Docker tag too if applicable): 1.5.16
* Python version:
* Database platform, version:
* Middleware(s):

<!--
    Describe in detail the exact steps that someone else can take to reproduce
    this bug using the current stable release of Nautobot. Begin with the
    creation of any necessary database objects and call out every operation
    being performed explicitly. If reporting a bug in the REST API, be sure to
    reconstruct the raw HTTP request(s) being made: Don't rely on a client
    library such as pynautobot.
-->
### Steps to Reproduce
1. Create two `Platform` objects:

```
- name: juniper_junos
  slug: juniper_junos
- name: cisco_ios
  slug: cisco_ios
```

2. Create dynamic group for `Device` object , in a `Platform` field select two created platforms. 

<img width="654" alt="Screenshot 2023-06-19 at 19 32 06" src="https://github.com/nautobot/nautobot/assets/16935122/3d50a8cc-ee2b-4e11-bc75-157d1d1dd075">

3. Modify `Platform` slug and name: change `cisco_ios` to `cisco-ios` (dash instead of underscore)

4. Attempt to modify dynamic group

5. After modifying Platform, `cisco-ios` is now not visible in a form. 

<img width="683" alt="Screenshot 2023-06-19 at 19 33 23" src="https://github.com/nautobot/nautobot/assets/16935122/f61e5067-c3a6-4021-b6f0-c5b99fe291bc">

<!-- What did you expect to happen? -->

### Expected Behavior

Clicking "Update" button results in modifying the dynamic group.

<!-- What happened instead? -->
### Observed Behavior

Clicking "Update" button and saving dynamic group results with an error:

<img width="752" alt="Screenshot 2023-06-19 at 19 34 12" src="https://github.com/nautobot/nautobot/assets/16935122/37b4fcef-efe2-49dd-b07e-f5865e0bbff2">



Discussion:
Hey team! Please [add your planning poker estimate](https://app.zenhub.com/workspaces/Nautobot-Core-Product-64a8783b7a3f4b1225973a0c/issues/nautobot/nautobot/3949?planning-poker) with Zenhub @bryanculver @glennmatthews @gsnider2195 @HanlinMiao @jathanism @timizuoebideri1
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/forms/models.py", line 413, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 1251, in full_clean
    raise ValidationError(errors)

During handling of the above exception ({'platform': ['Select a valid choice. cisco-ios is not one of the available choices.']}), another exception occurred:
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/source/nautobot/core/views/generic.py", line 419, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/source/nautobot/utilities/views.py", line 114, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/source/nautobot/extras/views.py", line 616, in post
    if form.is_valid():
  File "/usr/local/lib/python3.10/site-packages/django/forms/forms.py", line 175, in is_valid
    return self.is_bound and not self.errors
  File "/usr/local/lib/python3.10/site-packages/django/forms/forms.py", line 170, in errors
    self.full_clean()
  File "/usr/local/lib/python3.10/site-packages/django/forms/forms.py", line 374, in full_clean
    self._post_clean()
  File "/usr/local/lib/python3.10/site-packages/django/forms/models.py", line 415, in _post_clean
    self._update_errors(e)
  File "/usr/local/lib/python3.10/site-packages/django/forms/models.py", line 389, in _update_errors
    self.add_error(None, errors)
  File "/usr/local/lib/python3.10/site-packages/django/forms/forms.py", line 343, in add_error
    raise ValueError(

Exception Type: ValueError at /extras/dynamic-groups/test/edit/
Exception Value: 'DynamicGroupForm' has no field named 'platform'.
```
For context, DynamicGroup updates via the UI are done through multiple forms rather than a single unified form class - `DynamicGroupForm` only understands/validates/updates the base class attribute (`name`, `content_type`, etc.) while the `filter` is updated via a form dynamically constructed by `instance.generate_filter_form()`. The issue here is that because the underlying data changed, the existing DynamicGroup instance's `filter` data is no longer valid (in this specific case it references a `slug` that no longer exists, but the same issue would happen for e.g. a UUID-based filter that referenced a now-deleted object), and so when the `DynamicGroupForm.is_valid()` is called, calling the `instance.full_clean()` fails *before the generated "filter_form" has a chance to update the instance to a new (valid) filter value*.

~Most likely the required fix here will be to change the form definitions and/or code flow in `DynamicGroupEditView.post()` so that there's only a single form class/object that handles the updating/validation of the entire DynamicGroup instance data in a single pass. This will be pretty invasive code-wise but I think it's necessary.~

> Changing the model `clean`/`clean_fields`/`full_clean` logic turned out to be a cleaner way to accomplish this specific fix.

Resolving patch:
diff --git a/nautobot/extras/models/groups.py b/nautobot/extras/models/groups.py
--- a/nautobot/extras/models/groups.py
+++ b/nautobot/extras/models/groups.py
@@ -520,6 +520,15 @@ def delete(self, *args, **kwargs):
             )
         return super().delete(*args, **kwargs)
 
+    def clean_fields(self, exclude=None):
+        if exclude is None:
+            exclude = []
+
+        if "filter" not in exclude:
+            self.clean_filter()
+
+        super().clean_fields(exclude=exclude)
+
     def clean(self):
         super().clean()
 
@@ -530,9 +539,6 @@ def clean(self):
             if self.content_type != database_object.content_type:
                 raise ValidationError({"content_type": "ContentType cannot be changed once created"})
 
-        # Validate `filter` dict
-        self.clean_filter()
-
     def generate_query_for_filter(self, filter_field, value):
         """
         Return a `Q` object generated from a `filter_field` and `value`.


Test patch:
diff --git a/nautobot/extras/tests/test_dynamicgroups.py b/nautobot/extras/tests/test_dynamicgroups.py
--- a/nautobot/extras/tests/test_dynamicgroups.py
+++ b/nautobot/extras/tests/test_dynamicgroups.py
@@ -213,7 +213,7 @@ def test_content_type_is_immutable(self):
             instance.content_type = self.dynamicgroup_ct
             instance.validated_save()
 
-    def test_clean_filter_not_dict(self):
+    def test_full_clean_filter_not_dict(self):
         """Test that invalid filter types raise errors."""
         instance = self.groups[0]
         with self.assertRaises(ValidationError):
@@ -228,14 +228,26 @@ def test_clean_filter_not_dict(self):
             instance.filter = "site=ams01"
             instance.validated_save()
 
-    def test_clean_filter_not_valid(self):
+    def test_full_clean_filter_not_valid(self):
         """Test that an invalid filter dict raises an error."""
         instance = self.groups[0]
         with self.assertRaises(ValidationError):
             instance.filter = {"site": -42}
             instance.validated_save()
 
-    def test_clean_valid(self):
+    def test_clean_fields_exclude_filter(self):
+        """Test that filter validation is skipped when an appropriate `exclude` parameter is provided."""
+        instance = self.groups[0]
+        instance.filter = {"platform": -42}
+        instance.clean_fields(exclude=["filter"])
+        instance.full_clean(exclude=["filter"])
+
+        with self.assertRaises(ValidationError):
+            instance.clean_fields()
+        with self.assertRaises(ValidationError):
+            instance.full_clean()
+
+    def test_full_clean_valid(self):
         """Test a clean validation."""
         group = self.groups[0]
         group.refresh_from_db()