Repository: netbox-community/netbox

Issue:
After upgrade from 3.5 to 3.7, instantiating a DeviceType which contains nested InventoryItems throws "InventoryItem matching query does not exist"
### Deployment Type

Self-hosted

### NetBox Version

v3.7.4

### Python Version

3.11

### Steps to Reproduce

1. Upgrade from netbox-docker v3.5-2.6.1 to v3.7-2.8.0 via git release branch
2. Upgrade from Postgres 15 to Postgres 16 via sql dump/restore
3. Observe migrations complete successfully
4. Create a new DeviceType
5. Add InventoryItem A to new DeviceType
6. Add InventoryItem B to new DeviceType with InventoryItem A as Parent
7. Create a new Device from this DeviceType

### Expected Behavior

A new Device is created with nested InventoryItems from DeviceType.

### Observed Behavior

Instantiating a DeviceType which contains nested InventoryItems throws:
```
DoesNotExist at /dcim/devices/add/

InventoryItem matching query does not exist.

Request Method: 	POST
Request URL: 	http://netbox.mydomain.tld/dcim/devices/add/
Django Version: 	4.2.11
Exception Type: 	DoesNotExist
Exception Value: 	

InventoryItem matching query does not exist.

Exception Location: 	/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py, line 637, in get
Raised during: 	dcim.views.DeviceEditView
Python Executable: 	/opt/netbox/venv/bin/python
Python Version: 	3.11.4
Python Path: 	

['/opt/netbox/netbox/',
 '/usr/lib/python311.zip',
 '/usr/lib/python3.11',
 '/usr/lib/python3.11/lib-dynload',
 '/opt/netbox/venv/lib/python3.11/site-packages']

Server time: 	Tue, 02 Apr 2024 00:05:31 +0000

Environment:


Request Method: POST
Request URL: http://netbox.mydomain.tld/dcim/devices/add/

Django Version: 4.2.11
Python Version: 3.11.4
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django.forms',
 'corsheaders',
 'debug_toolbar',
 'graphiql_debug_toolbar',
 'django_filters',
 'django_tables2',
 'django_prometheus',
 'graphene_django',
 'mptt',
 'rest_framework',
 'social_django',
 'taggit',
 'timezone_field',
 'core',
 'account',
 'circuits',
 'dcim',
 'ipam',
 'extras',
 'tenancy',
 'users',
 'utilities',
 'virtualization',
 'vpn',
 'wireless',
 'django_rq',
 'drf_spectacular',
 'drf_spectacular_sidecar']
Installed Middleware:
['graphiql_debug_toolbar.middleware.DebugToolbarMiddleware',
 'django_prometheus.middleware.PrometheusBeforeMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'netbox.middleware.RemoteUserMiddleware',
 'netbox.middleware.CoreMiddleware',
 'netbox.middleware.MaintenanceModeMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware']



Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/object_views.py", line 175, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/base.py", line 26, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/utilities/views.py", line 106, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/object_views.py", line 269, in post
    obj = form.save()
          ^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/forms/models.py", line 542, in save
    self.instance.save()
    ^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 1054, in save
    self._instantiate_components(self.device_type.inventoryitemtemplates.all(), bulk_create=False)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 999, in _instantiate_components
    components = [obj.instantiate(device=self) for obj in queryset]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 999, in <listcomp>
    components = [obj.instantiate(device=self) for obj in queryset]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/device_component_templates.py", line 765, in instantiate
    parent = InventoryItem.objects.get(name=self.parent.name, **kwargs) if self.parent else None
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
    ^

Exception Type: DoesNotExist at /dcim/devices/add/
Exception Value: InventoryItem matching query does not exist.


Discussion:
This issue looks a lot like #12118 which was fixed in PR #12170.
BTW - I opened this issue here (rather than in docker-netbox) because it does not seem like an issue with the Docker image or related tooling and rather more like an issue with schema or a script.
Thank you for opening a bug report. I was unable to reproduce the reported behavior on NetBox v3.7.4. Please re-confirm the reported behavior on the current stable release and adjust your post above as necessary. Remember to provide detailed steps that someone else can follow using a clean installation of NetBox to reproduce the issue. Remember to include the steps taken to create any initial objects or other data.
Replicates fine here following his steps. The offending commit is this:

https://github.com/netbox-community/netbox/commit/32264ac3e3496a9a9cd049eeef671179d3f9c41f

The cause is pretty obvious, inventory items are intentionally created with bulk_create=False to avoid trying to reference a non-existant object.

The above commit moves the instantiation of the inventory items outside the bulk_create check. Reverting the commit allows creating the device again.
Thanks @kkthxbye-code, would you like to submit a PR for this?
@jeremystretch @kkthxbye-code I got it.  I'll have a PR shortly.
![image](https://github.com/netbox-community/netbox/assets/3824438/832cdc4a-4864-42f3-88d1-f3fa294e2044)
![image](https://github.com/netbox-community/netbox/assets/3824438/737afce2-5166-4053-9bed-844509dadd5c)


Resolving patch:
diff --git a/netbox/dcim/models/devices.py b/netbox/dcim/models/devices.py
--- a/netbox/dcim/models/devices.py
+++ b/netbox/dcim/models/devices.py
@@ -996,17 +996,16 @@ def _instantiate_components(self, queryset, bulk_create=True):
             bulk_create: If True, bulk_create() will be called to create all components in a single query
                          (default). Otherwise, save() will be called on each instance individually.
         """
-        components = [obj.instantiate(device=self) for obj in queryset]
-        if not components:
-            return
-
-        # Set default values for any applicable custom fields
         model = queryset.model.component_model
-        if cf_defaults := CustomField.objects.get_defaults_for_model(model):
-            for component in components:
-                component.custom_field_data = cf_defaults
 
         if bulk_create:
+            components = [obj.instantiate(device=self) for obj in queryset]
+            if not components:
+                return
+            # Set default values for any applicable custom fields
+            if cf_defaults := CustomField.objects.get_defaults_for_model(model):
+                for component in components:
+                    component.custom_field_data = cf_defaults
             model.objects.bulk_create(components)
             # Manually send the post_save signal for each of the newly created components
             for component in components:
@@ -1019,7 +1018,11 @@ def _instantiate_components(self, queryset, bulk_create=True):
                     update_fields=None
                 )
         else:
-            for component in components:
+            for obj in queryset:
+                component = obj.instantiate(device=self)
+                # Set default values for any applicable custom fields
+                if cf_defaults := CustomField.objects.get_defaults_for_model(model):
+                    component.custom_field_data = cf_defaults
                 component.save()
 
     def save(self, *args, **kwargs):


Test patch: