Repository: DataDog/dd-trace-py

Issue:
Django failure after upgrading
### Which version of dd-trace-py are you using?

0.39.0

### Which version of the libraries are you using?

We use a LOT of dependencies, I've tried to extract out the ones you're going to care here but I can always provide a full list:

```
django==1.8.17
django-18-fast-migrations==0.1.1
django-admin-sortable2==0.5.1
django-adminplus==0.2.1
django-cors-headers==2.1.0
django-csp==3.5
django-debug-toolbar==1.9.1
django-discover-runner==1.0
django-extensions==1.6.7
django-filter==1.1.0
django-formtools==2.1
django-jinja==2.2.1
django-mssql==1.8
django-orderable==6.0.1
django-otp==0.4.3
django-otp-twilio==0.4.2
django-password-validation==0.1.1
django-pyodbc-azure==1.8.17.0
django-ratelimit==2.0.0
django-redis==4.8.0
django-rest-swagger==2.1.2
django-role-permissions==2.2.0
django-sqlserver==1.11
django-sslserver==0.15
django-suit==0.2.13
django-supervisor==0.3.4
django-webpack-loader==0.6.0
djangorestframework==3.6.3
```

```
ddtrace==0.39.0
datadog==0.37.1
```

### How can we reproduce your problem?

This one might be hard to reproduce.  We're upgrading:

```
-datadog==0.32.0
+datadog==0.37.1
 datetime-truncate==1.0.1
 datrie==0.7.1
-ddtrace==0.31.0
+ddtrace==0.39.0
```

And on py2, after upgrading (but not on py3 - we have different dependencies between py2 and py3 however as we transition over to fully py3) I see:

```
#48 1.476   new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
#48 1.476
#48 1.476 2020-07-02 19:58:26,607 [WARNING:140088484815680:8] py.warnings: /root/.local/lib/python2.7/s
ite-packages/django/db/models/base.py:309: RuntimeWarning: Model 'simon.asynctask' was already register
ed. Reloading models is not advised as it can lead to inconsistencies, most notably with related models
.
#48 1.476   new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
#48 1.476
#48 2.127 Traceback (most recent call last):
#48 2.127   File "/root/.local/bin/django-admin", line 8, in <module>
#48 2.127     sys.exit(execute_from_command_line())
#48 2.127   File "/root/.local/lib/python2.7/site-packages/django/core/management/__init__.py", line 35
4, in execute_from_command_line
#48 2.127     utility.execute()
#48 2.127   File "/root/.local/lib/python2.7/site-packages/django/core/management/__init__.py", line 32
8, in execute
#48 2.127     django.setup()
#48 2.127   File "/root/.local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
#48 2.127     apps.populate(settings.INSTALLED_APPS)
#48 2.127   File "/root/.local/lib/python2.7/site-packages/django/apps/registry.py", line 115, in popul
ate
#48 2.127     app_config.ready()
#48 2.127   File "/root/.local/lib/python2.7/site-packages/rolepermissions/apps.py", line 12, in ready
#48 2.127     load_roles_and_permissions()
#48 2.127   File "/root/.local/lib/python2.7/site-packages/rolepermissions/loader.py", line 28, in load
_roles_and_permissions
#48 2.127     app_name = get_app_name(app_name)
#48 2.127   File "/root/.local/lib/python2.7/site-packages/rolepermissions/loader.py", line 16, in get_
app_name
#48 2.127     type_ = locate(app_name)
#48 2.127   File "/usr/local/lib/python2.7/pydoc.py", line 1518, in locate
#48 2.127     nextmodule = safeimport(join(parts[:n+1], '.'), forceload)
#48 2.127   File "/usr/local/lib/python2.7/pydoc.py", line 342, in safeimport
#48 2.127     raise ErrorDuringImport(path, sys.exc_info())
#48 2.127 pydoc.ErrorDuringImport: problem in django.contrib.admin.apps.SimpleAdminConfig - <type 'exce
ptions.ImportError'>: No module named SimpleAdminConfig
```

This occurs after I change out settings/base/__init__.py to have these lines:

```
  from ddtrace import config
  config.django['service_name'] = 'django'
```

It dies in:

```
apps.populate(settings.INSTALLED_APPS)
```

Which looks like this (after removing the old ddtrace app)

```
  INSTALLED_APPS = (
      'identity',
      'suit',
      'debug_toolbar',
      'django_extensions',
      'sslserver',
>>) + COMMON_INSTALLED_APPS
```

COMMON_INSTALLED_APPS is:

```
COMMON_INSTALLED_APPS = (
      'django.contrib.auth',
      'django.contrib.sessions',
      'django.contrib.contenttypes',
      'django.contrib.messages',
      'django.contrib.staticfiles',

      'django.contrib.admin.apps.SimpleAdminConfig',
      'django.contrib.admindocs',

      # For 2FA
      'django_otp',
      'django_otp.plugins.otp_totp',
      'otp_twilio',

      'rolepermissions',

      'adminplus',
      'django_jinja',
      'orderable',
      'rest_framework',
      'corsheaders',
      'rest_framework_swagger',

      'accounts',
      'auth3',
      'api',
      'billing',
      'costs',
      'journeys_advancement',
      'journeys_core',
      'simon',
      'singerio',
      'saml',
      'results',
      'chef',
      'webpack_loader',
      'django_filters',
      'featurestore',
      'journeys_app',
```

Ideally what I'm hoping for is help to figure out what to troubleshoot next.  I know we're on an ancient version of Django, but I can't do anything about that in the short term.

Simply having 'from ddtrace import config' and nothing else in my settings is enough to break things, and removing this always fixes it.


### What is the result that you get?

The backtrace above.

### What is the result that you expected?

Beautiful tracing.


Discussion:
This started to fail at 0.35, so I assume it's related to this changelog entry:

New hook registry for patching on import

(given that my failures happen on import).
After a really long day troubleshooting this is caused by:

django-role-permissions.  If this is installed and in INSTALLED_APPS then I get this failure every time.  If I just remove it from INSTALLED_APPS everything seems to work.
Hi @apenney, wow thanks for the very detailed report and follow-up!

Just to make sure: it appears to be the combination of both django-role-permissions and ddtrace that is resulting in the crash, right? If so I'll see if I can reproduce the issue.
I think so, I was going to start tomorrow by trying to get a reproducible project.  I was able to repro with django 1.9, 1.11, and 1.8, but I'm still not sure if it's anything to do with how we use the roles-permissions module, so I'll try and make a cloneable repo that reproduces it (if you don't beat me to it)
Gotcha :+1: sounds good 🙂 
https://github.com/apenney/ddtrace-bug - should be a minimal (or as minimal as I can) repo case, cobbled together from the code that seems to trigger this in our real codebase!

With py2 I can ./manage.py collectstatic to cause this failure every time.
🙌 awesome, thanks!! I'll try to take a look today

Resolving patch:
diff --git a/ddtrace/__init__.py b/ddtrace/__init__.py
--- a/ddtrace/__init__.py
+++ b/ddtrace/__init__.py
@@ -1,10 +1,5 @@
 import pkg_resources
 
-# Always import and patch import hooks before loading anything else
-from .internal.import_hooks import patch as patch_import_hooks
-
-patch_import_hooks()  # noqa: E402
-
 from .monkey import patch, patch_all  # noqa: E402
 from .pin import Pin  # noqa: E402
 from .span import Span  # noqa: E402


Test patch:
diff --git a/tests/internal/import_hooks/test_integration.py b/tests/internal/import_hooks/test_integration.py
--- a/tests/internal/import_hooks/test_integration.py
+++ b/tests/internal/import_hooks/test_integration.py
@@ -9,6 +9,9 @@
 from tests.subprocesstest import run_in_subprocess, SubprocessTestCase
 
 
+import_hooks.patch()
+
+
 @pytest.fixture
 def hooks():
     import_hooks.hooks.reset()