Repository: HypothesisWorks/hypothesis
Pull request: conditionally import fields from django.contrib
Closes #3716

Problem statement:
hypothesis.extra.django.TestCase vs. Django INSTALLED_APPS
Hello, I'm trying out Hypothesis 6.82.2 on my Django 4.2 project.

Importing `hypothesis.extra.django.TestCase` fails with `"RuntimeError: Model class django.contrib.contenttypes.models.ContentType doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS."`

The error is triggered by `hypothesis/extra/django/_fields.py` doing `from django.contrib.auth.forms import UsernameField`

It can be fixed by adding `'django.contrib.contenttypes'` to the `INSTALLED_APPS` setting.

However, I would like to use Hypothesis without modifying my INSTALLED_APPS setting.

Ideally, Hypothesis should use UsernameField when it is available, and not register strategies for it when it is not available. UsernameField is available when `'django.contrib.contenttypes'` and `'django.contrib.auth'` are in `INSTALLED_APPS`, as far as I can tell.

Would you be open to such a change?

Hints from issue discussion:
@jams2 this might be relevant to your interests?
@Zac-HD certainly. I'm afk for 2 weeks, but can take a look if this is still open when I'm back.

Commit message: adjust release notes

Patch to review:
diff --git a/hypothesis-python/RELEASE.rst b/hypothesis-python/RELEASE.rst
new file mode 100644
index 0000000000..b495be4f7c
--- /dev/null
+++ b/hypothesis-python/RELEASE.rst
@@ -0,0 +1,5 @@
+RELEASE_TYPE: patch
+
+This patch fixes an error when importing :ref:`our django extra <hypothesis-django>` (via ``hypothesis.extra.django``) if ``django.contrib.auth`` was not in ``INSTALLED_APPS`` (:issue:`3716`).
+
+Thanks to Chris Wesseling for this fix!
diff --git a/hypothesis-python/src/hypothesis/extra/django/_fields.py b/hypothesis-python/src/hypothesis/extra/django/_fields.py
index 36b43dffe6..357a0e6a57 100644
--- a/hypothesis-python/src/hypothesis/extra/django/_fields.py
+++ b/hypothesis-python/src/hypothesis/extra/django/_fields.py
@@ -17,7 +17,7 @@
 
 import django
 from django import forms as df
-from django.contrib.auth.forms import UsernameField
+from django.conf import settings
 from django.core.validators import (
     validate_ipv4_address,
     validate_ipv6_address,
@@ -231,7 +231,6 @@ def _for_binary(field):
 @register_for(dm.TextField)
 @register_for(df.CharField)
 @register_for(df.RegexField)
-@register_for(UsernameField)
 def _for_text(field):
     # We can infer a vastly more precise strategy by considering the
     # validators as well as the field type.  This is a minimal proof of
@@ -262,6 +261,12 @@ def _for_text(field):
     return strategy
 
 
+if "django.contrib.auth" in settings.INSTALLED_APPS:
+    from django.contrib.auth.forms import UsernameField
+
+    register_for(UsernameField)(_for_text)
+
+
 @register_for(df.BooleanField)
 def _for_form_boolean(field):
     if field.required:
diff --git a/hypothesis-python/tests/django/toys/settings/no_urls.py b/hypothesis-python/tests/django/toys/settings/no_urls.py
new file mode 100644
index 0000000000..42913e3e38
--- /dev/null
+++ b/hypothesis-python/tests/django/toys/settings/no_urls.py
@@ -0,0 +1,11 @@
+# This file is part of Hypothesis, which may be found at
+# https://github.com/HypothesisWorks/hypothesis/
+#
+# Copyright the Hypothesis Authors.
+# Individual contributors are listed in AUTHORS.rst and the git log.
+#
+# This Source Code Form is subject to the terms of the Mozilla Public License,
+# v. 2.0. If a copy of the MPL was not distributed with this file, You can
+# obtain one at https://mozilla.org/MPL/2.0/.
+
+urlpatterns = []
diff --git a/hypothesis-python/tests/django/toys/settings.py b/hypothesis-python/tests/django/toys/settings/settings.py
similarity index 98%
rename from hypothesis-python/tests/django/toys/settings.py
rename to hypothesis-python/tests/django/toys/settings/settings.py
index 6f8ae97fb5..20568c0ff7 100644
--- a/hypothesis-python/tests/django/toys/settings.py
+++ b/hypothesis-python/tests/django/toys/settings/settings.py
@@ -61,7 +61,7 @@
     "django.middleware.clickjacking.XFrameOptionsMiddleware",
 )
 
-ROOT_URLCONF = "tests.django.toys.urls"
+ROOT_URLCONF = "tests.django.toys.settings.urls"
 
 WSGI_APPLICATION = "tests.django.toys.wsgi.application"
 
diff --git a/hypothesis-python/tests/django/toys/settings/settings_no_contrib.py b/hypothesis-python/tests/django/toys/settings/settings_no_contrib.py
new file mode 100644
index 0000000000..d1b77c13f6
--- /dev/null
+++ b/hypothesis-python/tests/django/toys/settings/settings_no_contrib.py
@@ -0,0 +1,19 @@
+# This file is part of Hypothesis, which may be found at
+# https://github.com/HypothesisWorks/hypothesis/
+#
+# Copyright the Hypothesis Authors.
+# Individual contributors are listed in AUTHORS.rst and the git log.
+#
+# This Source Code Form is subject to the terms of the Mozilla Public License,
+# v. 2.0. If a copy of the MPL was not distributed with this file, You can
+# obtain one at https://mozilla.org/MPL/2.0/.
+
+from .settings import *  # noqa: F403
+
+# We test django in two contexts: with some default django.contrib apps installed
+# (which is settings.py), and with no django.contrib apps installed (which is this
+# file). We set DJANGO_SETTINGS_MODULE in tox to select which settings file we
+# use during testing.
+
+INSTALLED_APPS = ["tests.django.toystore"]
+ROOT_URLCONF = "tests.django.toys.settings.no_urls"
diff --git a/hypothesis-python/tests/django/toys/urls.py b/hypothesis-python/tests/django/toys/settings/urls.py
similarity index 100%
rename from hypothesis-python/tests/django/toys/urls.py
rename to hypothesis-python/tests/django/toys/settings/urls.py
diff --git a/hypothesis-python/tests/django/toystore/forms.py b/hypothesis-python/tests/django/toystore/forms.py
index dfd482bbad..6714566edc 100644
--- a/hypothesis-python/tests/django/toystore/forms.py
+++ b/hypothesis-python/tests/django/toystore/forms.py
@@ -9,7 +9,7 @@
 # obtain one at https://mozilla.org/MPL/2.0/.
 
 from django import forms
-from django.contrib.auth.forms import ReadOnlyPasswordHashField, UsernameField
+from django.conf import settings
 from django.core.validators import (
     MaxLengthValidator,
     MaxValueValidator,
@@ -28,6 +28,9 @@
     Store,
 )
 
+if "django.contrib.auth" in settings.INSTALLED_APPS:
+    from django.contrib.auth.forms import ReadOnlyPasswordHashField, UsernameField
+
 
 class ReprModelForm(forms.ModelForm):
     def __repr__(self):
@@ -217,12 +220,17 @@ class ShortStringForm(ReprForm):
     _not_too_long = forms.CharField(max_length=20, required=False)
 
 
-class UsernameForm(ReprForm):
-    username = UsernameField()
+if "django.contrib.auth" in settings.INSTALLED_APPS:
+
+    class UsernameForm(ReprForm):
+        username = UsernameField()
 
+    class ReadOnlyPasswordHashFieldForm(ReprForm):
+        password = ReadOnlyPasswordHashField()
 
-class ReadOnlyPasswordHashFieldForm(ReprForm):
-    password = ReadOnlyPasswordHashField()
+else:
+    UsernameForm = None
+    ReadOnlyPasswordHashFieldForm = None
 
 
 class StoreForm(ReprModelForm):
diff --git a/hypothesis-python/tests/django/toystore/test_given_forms.py b/hypothesis-python/tests/django/toystore/test_given_forms.py
index be8fa470de..4b1addce8f 100644
--- a/hypothesis-python/tests/django/toystore/test_given_forms.py
+++ b/hypothesis-python/tests/django/toystore/test_given_forms.py
@@ -8,7 +8,10 @@
 # v. 2.0. If a copy of the MPL was not distributed with this file, You can
 # obtain one at https://mozilla.org/MPL/2.0/.
 
+from unittest import skipIf
+
 from django import forms
+from django.conf import settings
 
 from hypothesis import assume, given
 from hypothesis.extra.django import (
@@ -48,6 +51,8 @@
     BroadBooleanField, booleans() | sampled_from(["1", "0", "True", "False"])
 )
 
+has_contrib_auth = "django.contrib.auth" in settings.INSTALLED_APPS
+
 
 class TestGetsBasicForms(TestCase):
     @given(from_form(CustomerForm))
@@ -130,10 +135,12 @@ def test_tight_validators_form(self, x):
         self.assertTrue(1 <= x.data["_float_one_to_five"] <= 5)
         self.assertTrue(5 <= len(x.data["_string_five_to_ten"]) <= 10)
 
+    @skipIf(not has_contrib_auth, "contrib.auth not installed")
     @given(from_form(UsernameForm))
     def test_username_form(self, username_form):
         self.assertTrue(username_form.is_valid())
 
+    @skipIf(not has_contrib_auth, "contrib.auth not installed")
     @given(from_form(UsernameForm))
     def test_read_only_password_hash_field_form(self, password_form):
         self.assertTrue(password_form.is_valid())
diff --git a/hypothesis-python/tests/django/toystore/test_given_models.py b/hypothesis-python/tests/django/toystore/test_given_models.py
index 266b18bd44..75bd9cfac1 100644
--- a/hypothesis-python/tests/django/toystore/test_given_models.py
+++ b/hypothesis-python/tests/django/toystore/test_given_models.py
@@ -9,11 +9,11 @@
 # obtain one at https://mozilla.org/MPL/2.0/.
 
 import datetime as dt
+from unittest import skipIf
 from uuid import UUID
 
 import django
 from django.conf import settings as django_settings
-from django.contrib.auth.models import User
 
 from hypothesis import HealthCheck, assume, given, settings, strategies as st
 from hypothesis.errors import InvalidArgument
@@ -26,6 +26,11 @@
 from hypothesis.internal.conjecture.data import ConjectureData
 from hypothesis.strategies import just, lists
 
+if "django.contrib.auth" in django_settings.INSTALLED_APPS:
+    from django.contrib.auth.models import User
+else:
+    User = None
+
 from tests.common.debug import check_can_generate_examples
 from tests.django.toystore.models import (
     Car,
@@ -184,6 +189,7 @@ def test_constructs_valid_instance(self, instance):
         self.assertTrue(instance.non_blank_text_field)
 
 
+@skipIf(User is None, "contrib.auth not installed")
 class TestValidatorInference(TestCase):
     @given(from_model(User))
     def test_user_issue_1112_regression(self, user):
diff --git a/hypothesis-python/tox.ini b/hypothesis-python/tox.ini
index 9f463f96b0..9db7c46e8f 100644
--- a/hypothesis-python/tox.ini
+++ b/hypothesis-python/tox.ini
@@ -166,25 +166,18 @@ commands =
     niche: bash scripts/other-tests.sh
     custom: python -bb -X dev -m pytest {posargs}
 
-[testenv:django42]
-setenv=
-    PYTHONWARNDEFAULTENCODING=1
-commands =
-    pip install django==4.2.20
-    python -bb -X dev -m tests.django.manage test tests.django {posargs}
-
-[testenv:django51]
-setenv=
-    PYTHONWARNDEFAULTENCODING=1
-commands =
-    pip install django==5.1.8
-    python -bb -X dev -m tests.django.manage test tests.django {posargs}
-
-[testenv:django52]
+[testenv:django{42,51,52,-nocontrib}]
 setenv=
     PYTHONWARNDEFAULTENCODING=1
+    DJANGO_SETTINGS_MODULE=tests.django.toys.settings.settings
+    nocontrib: DJANGO_SETTINGS_MODULE=tests.django.toys.settings.settings_no_contrib
+deps=
+    -r../requirements/test.txt
+    django42: django==4.2.22
+    django51: django==5.1.8
+    django52: django==5.2
+    django-nocontrib: django==5.2
 commands =
-    pip install django==5.2
     python -bb -X dev -m tests.django.manage test tests.django {posargs}
 
 [testenv:py{39}-nose]
diff --git a/tooling/src/hypothesistooling/__main__.py b/tooling/src/hypothesistooling/__main__.py
index a81976e813..da8077fa78 100644
--- a/tooling/src/hypothesistooling/__main__.py
+++ b/tooling/src/hypothesistooling/__main__.py
@@ -635,6 +635,8 @@ def standard_tox_task(name, py=ci_version):
 
 for n in DJANGO_VERSIONS:
     standard_tox_task(f"django{n.replace('.', '')}")
+# we also test no-contrib on the latest django version
+standard_tox_task("django-nocontrib")
 
 for n in [13, 14, 15, 20, 21, 22]:
     standard_tox_task(f"pandas{n}")


Code review comments:
diff_hunk: @@ -166,25 +166,18 @@ commands =
     niche: bash scripts/other-tests.sh
     custom: python -bb -X dev -m pytest {posargs}
 
-[testenv:django42]
-setenv=
-    PYTHONWARNDEFAULTENCODING=1
-commands =
-    pip install django==4.2.20
-    python -bb -X dev -m tests.django.manage test tests.django {posargs}
-
-[testenv:django51]
-setenv=
-    PYTHONWARNDEFAULTENCODING=1
-commands =
-    pip install django==5.1.8
-    python -bb -X dev -m tests.django.manage test tests.django {posargs}
-
-[testenv:django52]
+[testenv:django{42,51,52,-nocontrib}]
 setenv=
     PYTHONWARNDEFAULTENCODING=1
+    DJANGO_SETTINGS_MODULE=tests.django.toys.settings.settings
+    nocontrib: DJANGO_SETTINGS_MODULE=tests.django.toys.settings.settings_no_contrib
+deps=
+    -r../requirements/test.txt
+    django42: django==4.2.22
+    django51: django==5.1.8
+    django52: django==5.2
+    django-nocontrib: django==5.2
path: hypothesis-python/tox.ini
text: @user1:
This is I think basically fine with our auto-update logic, which I wanted to check, except that there's an existing bug on master where we don't handle `major.minor` without `.patch` at the end.  Mind making that optional?