Repository: netbox-community/netbox

Issue:
Inconsistent /api/virtualization/interfaces/ results
<!--
    Before opening a new issue, please search through the existing issues to
    see if your topic has already been addressed. Note that you may need to
    remove the "is:open" filter from the search bar to include closed issues.

    Check the appropriate type for your issue below by placing an x between the
    brackets. For assistance with installation issues, or for any other issues
    other than those listed below, please raise your topic for discussion on
    our mailing list:

        https://groups.google.com/forum/#!forum/netbox-discuss

    Please note that issues which do not fall under any of the below categories
    will be closed. Due to an excessive backlog of feature requests, we are
    not currently accepting any proposals which extend NetBox's feature scope.

    Do not prepend any sort of tag to your issue's title. An administrator will
    review your issue and assign labels as appropriate.
--->
### Issue type
[ ] Feature request <!-- An enhancement of existing functionality -->
[ x  ] Bug report      <!-- Unexpected or erroneous behavior -->
[ ] Documentation   <!-- A modification to the documentation -->

<!--
    Please describe the environment in which you are running NetBox. (Be sure
    to verify that you are running the latest stable release of NetBox before
    submitting a bug report.) If you are submitting a bug report and have made
    any changes to the code base, please first validate that your bug can be
    recreated while running an official release.
-->
### Environment
* Python version:  3.6.5
* NetBox version:  2.3.4

<!--
    BUG REPORTS must include:
        * A list of the steps needed for someone else to reproduce the bug
        * A description of the expected and observed behavior
        * Any relevant error messages (screenshots may also help)

    FEATURE REQUESTS must include:
        * A detailed description of the proposed functionality
        * A use case for the new feature
        * A rough description of any necessary changes to the database schema
        * Any relevant third-party libraries which would be needed
-->
### Description

Querying all virtualized interfaces returns inconsistents results : the count is OK, but some interfaces are missing and some are duplicated.

The underlying query is ordering by an empty column ("dcim_device"."name") which seems to fall under the non predictable results case : https://www.postgresql.org/docs/current/static/queries-limit.html

```sql
... WHERE "dcim_interface"."virtual_machine_id" IS NOT NULL 
ORDER BY "dcim_device"."name" ASC, "dcim_interface"."name" ASC LIMIT 1000 OFFSET 50
```


Discussion:
Thank you for taking the time to report a bug. We'll need more information to investigate this issue. Per the [contributing guidelines](https://github.com/digitalocean/netbox/blob/develop/CONTRIBUTING.md) please ensure that you have included all of the following:

* The environment in which NetBox is running
* **The exact steps that can be taken to reproduce the issue**
* Any error messages generated
* Screenshots (if applicable)

Please update your issue to include all of the information listed above. If no response is received within a week, this issue will be closed. Thanks!
```python
import pynetbox

for vif in nb.virtualization.interfaces.all():
     print(vif.id)
```
Duplicate ids are printed. 

I guess it requires you to have enough interfaces to get this bug triggered consistently.

I've "fixed" it by changing the ordering in the Interface model (```ordering = ['id', 'name']``` instead of ```ordering = ['device', 'name']```).

Still can't reproduce this. The queryset which retrieves the interfaces is:

```
Interface.objects.filter(virtual_machine__isnull=False).select_related('virtual_machine')
```

Not sure how that could result in duplicate objects being returned. I'll need sample data which triggers the bug to dig any further into this.
Can I send you a dump of my database by email ?
Sorry, you'll need to derive a sample set from your data which can be used to replicate the problem and post it here.
Closing due to lack of activity.
Hello Jeremy,

Finally got some time to work on this.

The key is to a have a mixed set of device and vm interfaces and to have a count above the paging size (>1000).

On a fresh 2.5.6 install with the initial data and after loading some dummy devices and vms with the attached script, you should witness the inconsistency :

```python
vifs = [iface.id for iface in nb.virtualization.interfaces.all()]

len(vifs)
600

len(set(vifs))
567
```


[reproduce_2207.py.txt](https://github.com/digitalocean/netbox/files/2886832/reproduce_2207.py.txt)
@dsanader Unless you can recreate this using bare HTTP calls NetBox REST API, you'll need to open this as a bug against the pynetbox library.
You're hard on me...

Anyway, same outcome with curl :
```
curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/" > first.json
curl -H "authorization: Token d34db33f" "https://netbox-test.example.com/api/virtualization/interfaces/?limit=600&offset=50"> second.json
jq '.results[] | .id' first.json second.json | sort | uniq -c | wc -l
567
```

The issues goes away (600 unique ids) as soon as I :

```diff --git a/netbox/virtualization/api/views.py b/netbox/virtualization/api/views.py
index 3b0c02b2..f9502071 100644
--- a/netbox/virtualization/api/views.py
+++ b/netbox/virtualization/api/views.py
@@ -65,7 +65,7 @@ class VirtualMachineViewSet(CustomFieldModelViewSet):
 class InterfaceViewSet(ModelViewSet):
     queryset = Interface.objects.filter(
         virtual_machine__isnull=False
-    ).select_related('virtual_machine').prefetch_related('tags')
+    ).select_related('virtual_machine').prefetch_related('tags').order_by('id')
     serializer_class = serializers.InterfaceSerializer
     filterset_class = filters.InterfaceFilter
```


Resolving patch:
diff --git a/netbox/dcim/managers.py b/netbox/dcim/managers.py
--- a/netbox/dcim/managers.py
+++ b/netbox/dcim/managers.py
@@ -64,11 +64,15 @@ def get_queryset(self):
 
         The original `name` field is considered in its entirety to serve as a fallback in the event interfaces do not
         match any of the prescribed fields.
+
+        The `id` field is included to enforce deterministic ordering of interfaces in similar vein of other device
+        components.
         """
 
         sql_col = '{}.name'.format(self.model._meta.db_table)
         ordering = [
-            '_slot', '_subslot', '_position', '_subposition', '_type', '_id', '_channel', '_vc', 'name',
+            '_slot', '_subslot', '_position', '_subposition', '_type', '_id', '_channel', '_vc', 'name', 'pk'
+
         ]
 
         fields = {


Test patch: