Repository: nautobot/nautobot

Issue:
Prefix with tags is created in 2 steps
<!--
    NOTE: IF YOUR ISSUE DOES NOT FOLLOW THIS TEMPLATE, IT WILL BE CLOSED.

    This form is only for reporting reproducible bugs. If you need assistance
    with Nautobot installation, or if you have a general question, please start a
    discussion instead: https://github.com/nautobot/nautobot/discussions

    Please describe the environment in which you are running Nautobot. Be sure
    that you are running an unmodified instance of the latest stable release
    before submitting a bug report, and that any plugins have been disabled.
-->
### Environment
* Python version: 3.6.8
* Nautobot version: v1.0.0b2

<!--
    Describe in detail the exact steps that someone else can take to reproduce
    this bug using the current stable release of Nautobot. Begin with the
    creation of any necessary database objects and call out every operation
    being performed explicitly. If reporting a bug in the REST API, be sure to
    reconstruct the raw HTTP request(s) being made: Don't rely on a client
    library such as pynautobot.
-->

When creating a new prefix while specifying any tags, it is not created with it right away but first without tags and than it is updated with them. The result is correct, having prefix with tags created. But the process is wrong, because it triggers 2 webhooks (`type create` and `type update`), but there should only be single for `type update`, as this is creation.

### Steps to Reproduce
1. create new prefix with tags
![image](https://user-images.githubusercontent.com/26114847/113024418-02e35f00-9187-11eb-9b79-47fb9f545e39.png)


2. check changelog
![image](https://user-images.githubusercontent.com/26114847/113024187-c1eb4a80-9186-11eb-826e-ca945805b0e0.png)
![image](https://user-images.githubusercontent.com/26114847/113024374-f52dd980-9186-11eb-97d6-89aef6e63a71.png)

<!-- What did you expect to happen? -->
### Expected Behavior
Single change logging creation of the prefix

<!-- What happened instead? -->
### Observed Behavior
Change logging creation of the prefix and another one logging update (adding tags)



Discussion:
Thanks for the report @Thetacz. This is working as intended because the tags are generic objects that can be applied to many object types, and therefore are added using a many-to-many relationship, which is secondary to the initial object creation.

Tags can't be associated with an object until after the object is created and has its primary key populated (UUID). The primary key is what is used to associate the tags.

I agree that this isn't necessarily intuitive at first glance, but it's a fundamental issue with the implementation of change logging, and therefore webhooks, because of the nature of the events that are fired (create prefix, update prefix) in order to attach the tags.
Thanks for the response @jathanism. I see the issue here - through a design choice there is no way to link tag to object that is getting created. It needs to happen in a new step, which is object update. Since not triggering webhook on this kind of change is not really a good idea, there is probably nothing to be done about it.

Note I just tested updating existing object and there are also 2 changes being done. One for tags and one for the rest. If the change is only tags, the 1st change is empty.
![image](https://user-images.githubusercontent.com/26114847/113110520-75941f00-9207-11eb-9600-399f9feb0c31.png)
![image](https://user-images.githubusercontent.com/26114847/113110567-85136800-9207-11eb-99d3-d9bbd784dc6f.png)
Can you do at least something about this one? For example is it necessary to log zero change?
Hmm... Yes. This latter case is a known issue that we inherited from NetBox (See: https://github.com/netbox-community/netbox/issues/5583). This has come up recently in some other discussions.

Let's give it the old scoping effort here and see what to do about it.

This is a great idea, but not something we can prioritize right now. Help is welcome and wanted! 
Also related: https://github.com/netbox-community/netbox/pull/6516

Resolving patch:
diff --git a/nautobot/extras/signals.py b/nautobot/extras/signals.py
--- a/nautobot/extras/signals.py
+++ b/nautobot/extras/signals.py
@@ -45,23 +45,33 @@ def _handle_changed_object(request, sender, instance, **kwargs):
     """
     Fires when an object is created or updated.
     """
-    # Queue the object for processing once the request completes
+    m2m_changed = False
+
+    # Determine the type of change being made
     if kwargs.get("created"):
         action = ObjectChangeActionChoices.ACTION_CREATE
     elif "created" in kwargs:
         action = ObjectChangeActionChoices.ACTION_UPDATE
     elif kwargs.get("action") in ["post_add", "post_remove"] and kwargs["pk_set"]:
         # m2m_changed with objects added or removed
+        m2m_changed = True
         action = ObjectChangeActionChoices.ACTION_UPDATE
     else:
         return
 
     # Record an ObjectChange if applicable
     if hasattr(instance, "to_objectchange"):
-        objectchange = instance.to_objectchange(action)
-        objectchange.user = _get_user_if_authenticated(request, objectchange)
-        objectchange.request_id = request.id
-        objectchange.save()
+        if m2m_changed:
+            ObjectChange.objects.filter(
+                changed_object_type=ContentType.objects.get_for_model(instance),
+                changed_object_id=instance.pk,
+                request_id=request.id,
+            ).update(object_data=instance.to_objectchange(action).object_data)
+        else:
+            objectchange = instance.to_objectchange(action)
+            objectchange.user = _get_user_if_authenticated(request, objectchange)
+            objectchange.request_id = request.id
+            objectchange.save()
 
     # Enqueue webhooks
     enqueue_webhooks(instance, request.user, request.id, action)


Test patch:
diff --git a/nautobot/extras/tests/test_changelog.py b/nautobot/extras/tests/test_changelog.py
--- a/nautobot/extras/tests/test_changelog.py
+++ b/nautobot/extras/tests/test_changelog.py
@@ -53,24 +53,18 @@ def test_create_object(self):
         response = self.client.post(**request)
         self.assertHttpStatus(response, 302)
 
+        # Verify the creation of a new ObjectChange record
         site = Site.objects.get(name="Test Site 1")
         # First OC is the creation; second is the tags update
-        oc_list = ObjectChange.objects.filter(
+        oc = ObjectChange.objects.get(
             changed_object_type=ContentType.objects.get_for_model(Site),
             changed_object_id=site.pk,
-        ).order_by("time")
-        self.assertEqual(oc_list[0].changed_object, site)
-        self.assertEqual(oc_list[0].action, ObjectChangeActionChoices.ACTION_CREATE)
-        self.assertEqual(
-            oc_list[0].object_data["custom_fields"]["my_field"],
-            form_data["cf_my_field"],
         )
-        self.assertEqual(
-            oc_list[0].object_data["custom_fields"]["my_field_select"],
-            form_data["cf_my_field_select"],
-        )
-        self.assertEqual(oc_list[1].action, ObjectChangeActionChoices.ACTION_UPDATE)
-        self.assertEqual(oc_list[1].object_data["tags"], ["Tag 1", "Tag 2"])
+        self.assertEqual(oc.changed_object, site)
+        self.assertEqual(oc.action, ObjectChangeActionChoices.ACTION_CREATE)
+        self.assertEqual(oc.object_data["custom_fields"]["my_field"], form_data["cf_my_field"])
+        self.assertEqual(oc.object_data["custom_fields"]["my_field_select"], form_data["cf_my_field_select"])
+        self.assertEqual(oc.object_data["tags"], ["Tag 1", "Tag 2"])
 
     def test_update_object(self):
         site = Site(
@@ -99,8 +93,8 @@ def test_update_object(self):
         response = self.client.post(**request)
         self.assertHttpStatus(response, 302)
 
+        # Verify the creation of a new ObjectChange record
         site.refresh_from_db()
-        # Get only the most recent OC
         oc = ObjectChange.objects.filter(
             changed_object_type=ContentType.objects.get_for_model(Site),
             changed_object_id=site.pk,
@@ -192,16 +186,14 @@ def test_create_object(self):
         self.assertHttpStatus(response, status.HTTP_201_CREATED)
 
         site = Site.objects.get(pk=response.data["id"])
-        # First OC is the creation; second is the tags update
-        oc_list = ObjectChange.objects.filter(
+        oc = ObjectChange.objects.get(
             changed_object_type=ContentType.objects.get_for_model(Site),
             changed_object_id=site.pk,
-        ).order_by("time")
-        self.assertEqual(oc_list[0].changed_object, site)
-        self.assertEqual(oc_list[0].action, ObjectChangeActionChoices.ACTION_CREATE)
-        self.assertEqual(oc_list[0].object_data["custom_fields"], data["custom_fields"])
-        self.assertEqual(oc_list[1].action, ObjectChangeActionChoices.ACTION_UPDATE)
-        self.assertEqual(oc_list[1].object_data["tags"], ["Tag 1", "Tag 2"])
+        )
+        self.assertEqual(oc.changed_object, site)
+        self.assertEqual(oc.action, ObjectChangeActionChoices.ACTION_CREATE)
+        self.assertEqual(oc.object_data["custom_fields"], data["custom_fields"])
+        self.assertEqual(oc.object_data["tags"], ["Tag 1", "Tag 2"])
 
     def test_update_object(self):
         """Test PUT with changelogs."""
@@ -229,11 +221,10 @@ def test_update_object(self):
         self.assertHttpStatus(response, status.HTTP_200_OK)
 
         site = Site.objects.get(pk=response.data["id"])
-        # Get only the most recent OC
-        oc = ObjectChange.objects.filter(
+        oc = ObjectChange.objects.get(
             changed_object_type=ContentType.objects.get_for_model(Site),
             changed_object_id=site.pk,
-        ).first()
+        )
         self.assertEqual(oc.changed_object, site)
         self.assertEqual(oc.action, ObjectChangeActionChoices.ACTION_UPDATE)
         self.assertEqual(oc.object_data["custom_fields"], data["custom_fields"])
diff --git a/nautobot/utilities/testing/api.py b/nautobot/utilities/testing/api.py
--- a/nautobot/utilities/testing/api.py
+++ b/nautobot/utilities/testing/api.py
@@ -8,6 +8,8 @@
 from rest_framework.test import APIClient, APITransactionTestCase as _APITransactionTestCase
 
 from nautobot.users.models import ObjectPermission, Token
+from nautobot.extras.choices import ObjectChangeActionChoices
+from nautobot.extras.models import ObjectChange
 from .utils import disable_warnings
 from .views import ModelTestCase
 
@@ -314,8 +316,9 @@ def test_create_object(self):
                 response = self.client.post(self._get_list_url(), create_data, format="json", **self.header)
                 self.assertHttpStatus(response, status.HTTP_201_CREATED)
                 self.assertEqual(self._get_queryset().count(), initial_count + i + 1)
+                instance = self._get_queryset().get(pk=response.data["id"])
                 self.assertInstanceEqual(
-                    self._get_queryset().get(pk=response.data["id"]),
+                    instance,
                     create_data,
                     exclude=self.validation_excluded_fields,
                     api=True,
@@ -326,6 +329,14 @@ def test_create_object(self):
                     expected_slug = slugify(getattr(object, self.slug_source))
                     self.assertEqual(object.slug, expected_slug)
 
+                # Verify ObjectChange creation
+                if hasattr(self.model, "to_objectchange"):
+                    objectchanges = ObjectChange.objects.filter(
+                        changed_object_type=ContentType.objects.get_for_model(instance), changed_object_id=instance.pk
+                    )
+                    self.assertEqual(len(objectchanges), 1)
+                    self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_CREATE)
+
         def test_bulk_create_objects(self):
             """
             POST a set of objects in a single request.
@@ -397,6 +408,14 @@ def test_update_object(self):
             instance.refresh_from_db()
             self.assertInstanceEqual(instance, update_data, exclude=self.validation_excluded_fields, api=True)
 
+            # Verify ObjectChange creation
+            if hasattr(self.model, "to_objectchange"):
+                objectchanges = ObjectChange.objects.filter(
+                    changed_object_type=ContentType.objects.get_for_model(instance), changed_object_id=instance.pk
+                )
+                self.assertEqual(len(objectchanges), 1)
+                self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_UPDATE)
+
         def test_bulk_update_objects(self):
             """
             PATCH a set of objects in a single request.
@@ -460,6 +479,14 @@ def test_delete_object(self):
             self.assertHttpStatus(response, status.HTTP_204_NO_CONTENT)
             self.assertFalse(self._get_queryset().filter(pk=instance.pk).exists())
 
+            # Verify ObjectChange creation
+            if hasattr(self.model, "to_objectchange"):
+                objectchanges = ObjectChange.objects.filter(
+                    changed_object_type=ContentType.objects.get_for_model(instance), changed_object_id=instance.pk
+                )
+                self.assertEqual(len(objectchanges), 1)
+                self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_DELETE)
+
         def test_bulk_delete_objects(self):
             """
             DELETE a set of objects in a single request.
diff --git a/nautobot/utilities/testing/views.py b/nautobot/utilities/testing/views.py
--- a/nautobot/utilities/testing/views.py
+++ b/nautobot/utilities/testing/views.py
@@ -13,8 +13,8 @@
 from netaddr import IPNetwork
 from taggit.managers import TaggableManager
 
-from nautobot.extras.choices import CustomFieldTypeChoices, RelationshipSideChoices
-from nautobot.extras.models import ChangeLoggedModel, Tag
+from nautobot.extras.choices import CustomFieldTypeChoices, RelationshipSideChoices, ObjectChangeActionChoices
+from nautobot.extras.models import ChangeLoggedModel, ObjectChange, Tag
 from nautobot.users.models import ObjectPermission
 from nautobot.utilities.permissions import resolve_permission_ct
 from nautobot.utilities.fields import JSONArrayField
@@ -425,9 +425,20 @@ def test_create_object_with_permission(self):
             self.assertHttpStatus(self.client.post(**request), 302)
             self.assertEqual(initial_count + 1, self._get_queryset().count())
             if hasattr(self.model, "last_updated"):
-                self.assertInstanceEqual(self._get_queryset().order_by("last_updated").last(), self.form_data)
+                instance = self._get_queryset().order_by("last_updated").last()
+                self.assertInstanceEqual(instance, self.form_data)
             else:
-                self.assertInstanceEqual(self._get_queryset().last(), self.form_data)
+                instance = self._get_queryset().last()
+                self.assertInstanceEqual(instance, self.form_data)
+
+            if hasattr(self.model, "to_objectchange"):
+                # Verify ObjectChange creation
+                objectchanges = ObjectChange.objects.filter(
+                    changed_object_type=ContentType.objects.get_for_model(instance),
+                    changed_object_id=instance.pk,
+                )
+                self.assertEqual(len(objectchanges), 1)
+                self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_CREATE)
 
         @override_settings(EXEMPT_VIEW_PERMISSIONS=["*"])
         def test_create_object_with_constrained_permission(self):
@@ -537,6 +548,15 @@ def test_edit_object_with_permission(self):
             self.assertHttpStatus(self.client.post(**request), 302)
             self.assertInstanceEqual(self._get_queryset().get(pk=instance.pk), self.form_data)
 
+            if hasattr(self.model, "to_objectchange"):
+                # Verify ObjectChange creation
+                objectchanges = ObjectChange.objects.filter(
+                    changed_object_type=ContentType.objects.get_for_model(instance),
+                    changed_object_id=instance.pk,
+                )
+                self.assertEqual(len(objectchanges), 1)
+                self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_UPDATE)
+
         @override_settings(EXEMPT_VIEW_PERMISSIONS=["*"])
         def test_edit_object_with_constrained_permission(self):
             instance1, instance2 = self._get_queryset().all()[:2]
@@ -614,6 +634,15 @@ def test_delete_object_with_permission(self):
             with self.assertRaises(ObjectDoesNotExist):
                 self._get_queryset().get(pk=instance.pk)
 
+            if hasattr(self.model, "to_objectchange"):
+                # Verify ObjectChange creation
+                objectchanges = ObjectChange.objects.filter(
+                    changed_object_type=ContentType.objects.get_for_model(instance),
+                    changed_object_id=instance.pk,
+                )
+                self.assertEqual(len(objectchanges), 1)
+                self.assertEqual(objectchanges[0].action, ObjectChangeActionChoices.ACTION_DELETE)
+
         @override_settings(EXEMPT_VIEW_PERMISSIONS=["*"])
         def test_delete_object_with_constrained_permission(self):
             instance1, instance2 = self._get_queryset().all()[:2]