Repository: netbox-community/netbox

Issue:
Omit internal fields from change logs
### Proposed Changes

Omit private/internal model fields from change logs. The current list of private fields:

* `_name`: used for ordering
* (`dcim.Interface`) `_connected_circuittermination`, `_connected_interface`: unsure if they're meant to be visible in the change logs
* (`dcim.Region`) `level`, `lft`, `rght`, `tree_id`: the MPTT fields
* (`dcim.Cable`) `_abs_length`, `_termination_a_device`, `_termination_b_device`: normalization/caching


### Justification

The fields do not serve any purpose outside of NetBox. By removing them from the change logs, they will look more readable to a user. This idea surfaced in https://github.com/netbox-community/netbox/issues/4183#issuecomment-587141060.


Discussion:
Here's an example of how a region's change log would benefit from this.

#### Before
```json
{
    "created": "2020-02-10",
    "last_updated": "2020-02-10T13:19:37.319Z",
    "level": 0,
    "lft": 1,
    "name": "'murica",
    "parent": null,
    "rght": 2,
    "slug": "murica",
    "tree_id": 2
}
```


#### After
```json
{
    "created": "2020-02-10",
    "last_updated": "2020-02-10T13:19:37.319Z",
    "name": "'murica",
    "parent": null,
    "slug": "murica"
}
```
That make sense. I accept this
We need to devise a robust mechanism for controlling whether a field appears in the serialized representation. I would prefer to avoid making any static declarations (like what we do for CSV headers) as they're tedious to maintain. Simply ignoring fields prefaced with an underscore probably takes care of 80% of this. We can implement/tweak `to_objectchange()` on the remaining models to suppress any unwanted fields.
> We need to devise a robust mechanism for controlling whether a field appears in the serialized representation. I would prefer to avoid making any static declarations (like what we do for CSV headers) as they're tedious to maintain.

What about a hybrid, allows us to handle the MPTT case as that isn't captured by the underscore case.
I think it's reasonable to tackle that using a custom `to_objectchange()` method on the Region model.
I see this was accepted. Anyone working on this or shall I?

I was gonna follow what Jeremy said and pop any fields prefixed with `_` in `ChangeLoggedModel.to_objectchange` and override `Region.to_objectchange` to additionally remove the MPTT fields.
@hSaria feel free to take this one if you'd like.

Resolving patch:
diff --git a/netbox/dcim/models/__init__.py b/netbox/dcim/models/__init__.py
--- a/netbox/dcim/models/__init__.py
+++ b/netbox/dcim/models/__init__.py
@@ -20,10 +20,10 @@
 from dcim.constants import *
 from dcim.fields import ASNField
 from dcim.elevations import RackElevationSVG
-from extras.models import ConfigContextModel, CustomFieldModel, TaggedItem
+from extras.models import ConfigContextModel, CustomFieldModel, ObjectChange, TaggedItem
 from utilities.fields import ColorField, NaturalOrderingField
 from utilities.models import ChangeLoggedModel
-from utilities.utils import to_meters
+from utilities.utils import serialize_object, to_meters
 from .device_component_templates import (
     ConsolePortTemplate, ConsoleServerPortTemplate, DeviceBayTemplate, FrontPortTemplate, InterfaceTemplate,
     PowerOutletTemplate, PowerPortTemplate, RearPortTemplate,
@@ -118,6 +118,15 @@ def get_site_count(self):
             Q(region__in=self.get_descendants())
         ).count()
 
+    def to_objectchange(self, action):
+        # Remove MPTT-internal fields
+        return ObjectChange(
+            changed_object=self,
+            object_repr=str(self),
+            action=action,
+            object_data=serialize_object(self, exclude=['level', 'lft', 'rght', 'tree_id'])
+        )
+
 
 #
 # Sites
diff --git a/netbox/utilities/utils.py b/netbox/utilities/utils.py
--- a/netbox/utilities/utils.py
+++ b/netbox/utilities/utils.py
@@ -81,10 +81,12 @@ def get_subquery(model, field):
     return subquery
 
 
-def serialize_object(obj, extra=None):
+def serialize_object(obj, extra=None, exclude=None):
     """
     Return a generic JSON representation of an object using Django's built-in serializer. (This is used for things like
-    change logging, not the REST API.) Optionally include a dictionary to supplement the object data.
+    change logging, not the REST API.) Optionally include a dictionary to supplement the object data. A list of keys
+    can be provided to exclude them from the returned dictionary. Private fields (prefaced with an underscore) are
+    implicitly excluded.
     """
     json_str = serialize('json', [obj])
     data = json.loads(json_str)[0]['fields']
@@ -103,6 +105,16 @@ def serialize_object(obj, extra=None):
     if extra is not None:
         data.update(extra)
 
+    # Copy keys to list to avoid 'dictionary changed size during iteration' exception
+    for key in list(data):
+        # Private fields shouldn't be logged in the object change
+        if isinstance(key, str) and key.startswith('_'):
+            data.pop(key)
+
+        # Explicitly excluded keys
+        if isinstance(exclude, (list, tuple)) and key in exclude:
+            data.pop(key)
+
     return data
 
 


Test patch: