Repository: kraken-tech/django-integrity (language: python)

Issue:
Test marked as `xfail` non-strictly
https://github.com/kraken-tech/django-integrity/blob/fe258d9268147d46f8c0c87e19c275f09786037f/tests/django_integrity/test_constraints.py#L52 is marked as x-fail but isn't always failing.

There are two problems here:

1. inappropriate use of `xfail`: we should rewrite the test to have an explicit expected failure on a specific line
2. `xfail`s are not strict: we need to configure them to be strict in our pytest config.


Discussion / hints:

Fix patch:
diff --git a/pyproject.toml b/pyproject.toml
index 6ae176b..61bae90 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -156,5 +156,7 @@ django_settings_module = "tests.example_app.settings"
 [tool.pytest.ini_options]
 # Ensure error warnings are converted into test errors.
 filterwarnings = "error"
+# Ensure that tests fail if an xfail test unexpectedly passes.
+xfail_strict = true
 
 DJANGO_SETTINGS_MODULE = "tests.example_app.settings"


Test patch:
diff --git a/tests/django_integrity/test_constraints.py b/tests/django_integrity/test_constraints.py
index 5d6a733..93c7fc1 100644
--- a/tests/django_integrity/test_constraints.py
+++ b/tests/django_integrity/test_constraints.py
@@ -1,6 +1,7 @@
 import pytest
 from django import db as django_db
 from django.core import exceptions
+from django.db import transaction
 
 from django_integrity import constraints
 from tests.example_app import models as test_models
@@ -48,16 +49,18 @@ def test_all_constraints_set(self) -> None:
             # The ForeignKey constraint should be enforced immediately.
             test_models.ForeignKeyModel.objects.create(related_id=42)
 
-    @pytest.mark.django_db
-    @pytest.mark.xfail(raises=django_db.IntegrityError)
+    @pytest.mark.django_db(transaction=True)
     def test_constraint_not_set(self) -> None:
-        # This try block proves that the constraint isn't enforced immediately.
-        # It's deferred, so the error is raised in the shutdown phase of the test.
-        # We use xfail to catch the error and prevent the test from failing.
-        try:
-            test_models.ForeignKeyModel.objects.create(related_id=42)
-        except django_db.IntegrityError:
-            pytest.fail("The ForeignKey constraint should be deferred.")
+        # We handle transaction open/close manually in this test
+        # so that we can catch exceptions from the COMMIT.
+        transaction.set_autocommit(False)
+
+        # The related object does not exist.
+        # This would raise an IntegrityError if the FK constraint wasn't deferred.
+        test_models.ForeignKeyModel.objects.create(related_id=42)
+
+        with pytest.raises(django_db.IntegrityError):
+            transaction.commit()
 
     @pytest.mark.django_db(transaction=True)
     def test_not_in_transaction(self) -> None: