Repository: netbox-community/netbox

Issue:
Interface.objects.order_naturally() enhancements
<!--
    Before opening a new issue, please search through the existing issues to
    see if your topic has already been addressed. Note that you may need to
    remove the "is:open" filter from the search bar to include closed issues.

    Check the appropriate type for your issue below by placing an x between the
    brackets. If none of the below apply, please raise your issue for
    discussion on our mailing list:

        https://groups.google.com/forum/#!forum/netbox-discuss

    Please note that issues which do not fall under any of the below categories
    will be closed.
--->
### Issue type
[X] Feature request <!-- Requesting the implementation of a new feature -->
[X] Bug report      <!-- Reporting unexpected or erroneous behavior -->
[ ] Documentation   <!-- Proposing a modification to the documentation -->

<!--
    Please describe the environment in which you are running NetBox. (Be sure
    to verify that you are running the latest stable release of NetBox before
    submitting a bug report.)
-->
### Environment
* Python version:  <!-- Example: 3.5.4 -->
* NetBox version:  <!-- Example: 2.1.3 -->

<!--
    BUG REPORTS must include:
        * A list of the steps needed to reproduce the bug
        * A description of the expected behavior
        * Any relevant error messages (screenshots may also help)

    FEATURE REQUESTS must include:
        * A detailed description of the proposed functionality
        * A use case for the new feature
        * A rough description of any necessary changes to the database schema
        * Any relevant third-party libraries which would be needed
-->
### Description
I have discovered a few issues/limitations in the `order_naturally()` method which I have had to address in a local project, and would be happy to submit a PR for if desired.

The first is a bug in the actual ordering. Given interfaces with the following names:
* Ethernet1/3/1
* Ethernet1/4
* Ethernet1/5/1

I would expect `order_naturally()` to return them in that same order. However, it is instead returning them in the order:
* Ethernet1/3/1
* Ethernet1/5/1
* Ethernet1/4

This is a side effect of the regular expressions used in the method being anchored to the end of the string.

Second, I have added support for a third slash in the name (`Ethernet1/2/3/4`).

And finally, because [it is highly recommended you do not use the `.extra()` method](https://docs.djangoproject.com/en/1.11/ref/models/querysets/#extra), I have updated the method to use `.annotate()` and `RawSQL()` instead.

Again, I can have a pull request ready for this ASAP if desired, along with appropriate tests.


Discussion:

Resolving patch:
diff --git a/netbox/dcim/models.py b/netbox/dcim/models.py
--- a/netbox/dcim/models.py
+++ b/netbox/dcim/models.py
@@ -13,6 +13,7 @@
 from django.core.validators import MaxValueValidator, MinValueValidator
 from django.db import models
 from django.db.models import Count, Q, ObjectDoesNotExist
+from django.db.models.expressions import RawSQL
 from django.urls import reverse
 from django.utils.encoding import python_2_unicode_compatible
 
@@ -642,15 +643,16 @@ def order_naturally(self, method=IFACE_ORDERING_POSITION):
         To order interfaces naturally, the `name` field is split into six distinct components: leading text (type),
         slot, subslot, position, channel, and virtual circuit:
 
-            {type}{slot}/{subslot}/{position}:{channel}.{vc}
+            {type}{slot}/{subslot}/{position}/{subposition}:{channel}.{vc}
 
-        Components absent from the interface name are ignored. For example, an interface named GigabitEthernet0/1 would
-        be parsed as follows:
+        Components absent from the interface name are ignored. For example, an interface named GigabitEthernet1/2/3
+        would be parsed as follows:
 
             name = 'GigabitEthernet'
-            slot =  None
-            subslot = 0
-            position = 1
+            slot =  1
+            subslot = 2
+            position = 3
+            subposition = 0
             channel = None
             vc = 0
 
@@ -659,17 +661,31 @@ def order_naturally(self, method=IFACE_ORDERING_POSITION):
         """
         sql_col = '{}.name'.format(self.model._meta.db_table)
         ordering = {
-            IFACE_ORDERING_POSITION: ('_slot', '_subslot', '_position', '_channel', '_vc', '_type', 'name'),
-            IFACE_ORDERING_NAME: ('_type', '_slot', '_subslot', '_position', '_channel', '_vc', 'name'),
+            IFACE_ORDERING_POSITION: ('_slot', '_subslot', '_position', '_subposition', '_channel', '_vc', '_type', 'name'),
+            IFACE_ORDERING_NAME: ('_type', '_slot', '_subslot', '_position', '_subposition', '_channel', '_vc', 'name'),
         }[method]
-        return self.extra(select={
-            '_type': "SUBSTRING({} FROM '^([^0-9]+)')".format(sql_col),
-            '_slot': "CAST(SUBSTRING({} FROM '([0-9]+)\/[0-9]+\/[0-9]+(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col),
-            '_subslot': "CAST(SUBSTRING({} FROM '([0-9]+)\/[0-9]+(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col),
-            '_position': "CAST(SUBSTRING({} FROM '([0-9]+)(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col),
-            '_channel': "COALESCE(CAST(SUBSTRING({} FROM ':([0-9]+)(\.[0-9]+)?$') AS integer), 0)".format(sql_col),
-            '_vc': "COALESCE(CAST(SUBSTRING({} FROM '\.([0-9]+)$') AS integer), 0)".format(sql_col),
-        }).order_by(*ordering)
+
+        fields = {
+            '_type': RawSQL(r"SUBSTRING({} FROM '^([^0-9]+)')".format(sql_col), []),
+            '_slot': RawSQL(r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)([0-9]+)') AS integer)".format(sql_col), []),
+            '_subslot': RawSQL(
+                r"COALESCE(CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+)\/([0-9]+)') AS integer), 0)".format(
+                    sql_col), []
+            ),
+            '_position': RawSQL(
+                r"COALESCE(CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+\/){{2}}([0-9]+)') AS integer), 0)".format(
+                    sql_col), []
+            ),
+            '_subposition': RawSQL(
+                r"COALESCE(CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+\/){{3}}([0-9]+)') AS integer), 0)".format(
+                    sql_col), []
+            ),
+            '_channel': RawSQL(
+                r"COALESCE(CAST(SUBSTRING({} FROM ':([0-9]+)(\.[0-9]+)?$') AS integer), 0)".format(sql_col), []),
+            '_vc': RawSQL(r"COALESCE(CAST(SUBSTRING({} FROM '\.([0-9]+)$') AS integer), 0)".format(sql_col), []),
+        }
+
+        return self.annotate(**fields).order_by(*ordering)
 
     def connectable(self):
         """


Test patch:
diff --git a/netbox/dcim/tests/test_models.py b/netbox/dcim/tests/test_models.py
--- a/netbox/dcim/tests/test_models.py
+++ b/netbox/dcim/tests/test_models.py
@@ -98,3 +98,108 @@ def test_mount_zero_ru(self):
             face=None,
         )
         self.assertTrue(pdu)
+
+
+class InterfaceTestCase(TestCase):
+
+    def setUp(self):
+
+        self.site = Site.objects.create(
+            name='TestSite1',
+            slug='my-test-site'
+        )
+        self.rack = Rack.objects.create(
+            name='TestRack1',
+            facility_id='A101',
+            site=self.site,
+            u_height=42
+        )
+        self.manufacturer = Manufacturer.objects.create(
+            name='Acme',
+            slug='acme'
+        )
+
+        self.device_type = DeviceType.objects.create(
+            manufacturer=self.manufacturer,
+            model='FrameForwarder 2048',
+            slug='ff2048'
+        )
+        self.role = DeviceRole.objects.create(
+            name='Switch',
+            slug='switch',
+        )
+
+    def test_interface_order_natural(self):
+        device1 = Device.objects.create(
+            name='TestSwitch1',
+            device_type=self.device_type,
+            device_role=self.role,
+            site=self.site,
+            rack=self.rack,
+            position=10,
+            face=RACK_FACE_REAR,
+        )
+        interface1 = Interface.objects.create(
+            device=device1,
+            name='Ethernet1/3/1'
+        )
+        interface2 = Interface.objects.create(
+            device=device1,
+            name='Ethernet1/5/1'
+        )
+        interface3 = Interface.objects.create(
+            device=device1,
+            name='Ethernet1/4'
+        )
+        interface4 = Interface.objects.create(
+            device=device1,
+            name='Ethernet1/3/2/4'
+        )
+        interface5 = Interface.objects.create(
+            device=device1,
+            name='Ethernet1/3/2/1'
+        )
+
+        self.assertEqual(
+            list(Interface.objects.all().order_naturally()),
+            [interface1, interface5, interface4, interface3, interface2]
+        )
+
+    def test_interface_order_natural_subinterfaces(self):
+        device1 = Device.objects.create(
+            name='TestSwitch1',
+            device_type=self.device_type,
+            device_role=self.role,
+            site=self.site,
+            rack=self.rack,
+            position=10,
+            face=RACK_FACE_REAR,
+        )
+        interface1 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0/0/3'
+        )
+        interface2 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0/0/2.2'
+        )
+        interface3 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0/0/0.120'
+        )
+        interface4 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0/0/0'
+        )
+        interface5 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0/0/1.117'
+        )
+        interface6 = Interface.objects.create(
+            device=device1,
+            name='GigabitEthernet0'
+        )
+        self.assertEqual(
+            list(Interface.objects.all().order_naturally()),
+            [interface6, interface4, interface3, interface5, interface2, interface1]
+        )