Repository: Kinto/kinto (language: python)

Issue:
Unit tests should be independent
While upgrading to pytest v8, it was discovered that our unit tests have a lot of interdependence. Creating this issue to act as an epic.

Unit tests in kinto should:
1. Be independent of each other. We should be able to run one test module with the same test results as if we had run all tests.
2. Not create side effects. If a test needs to create a mock or a test file, that mock or test file should be removed during teardown.

Examples of issues:
1. Running `pytest tests/test_views_version.py` results in a test error because it relies on something setup in another test
2. Several tests appear to have `setUp` and `setUpClass` methods that create test state, but no associated teardown methods that reset it

Given how extensive our test scenarios are in kinto (over 2000 at this time), this may take significant effort to research and correct all cases. We should probably do this in phases to make the PR's readable.


Discussion / hints:

Fix patch:
diff --git a/Makefile b/Makefile
index 5726bf295..8b935b34d 100644
--- a/Makefile
+++ b/Makefile
@@ -25,8 +25,9 @@ help:
 	@echo "  migrate                     run the kinto migrations"
 	@echo "  lint                        run the code linters"
 	@echo "  tests                       run all the tests with all the supported python interpreters (same as CI)"
+	@echo "  test-deps                   pulls and runs docker postgres and memcached containers for tests"
 	@echo "  tdd                         run pytest-watch to rerun tests automatically on changes for tdd"
-	@echo "  tests-once  	             only run the tests once with the default python interpreter"
+	@echo "  tests-once  	               only run the tests once with the default python interpreter"
 	@echo "  functional                  run functional test against a real kinto"
 	@echo "  browser-test                run browser test against a real kinto"
 	@echo "  clean                       remove *.pyc files and __pycache__ directory"
@@ -94,6 +95,14 @@ tests: install-postgres install-monitoring install-memcached version-file instal
 tests-raw: version-file install-dev
 	$(VENV)/bin/py.test
 
+test-deps:
+	docker pull memcached
+	docker pull postgres
+	docker run -p 11211:11211 --name kinto-memcached -d memcached || echo "cannot start memcached, already exists?"
+	docker run -p 5432:5432 --name kinto-postgres -e POSTGRES_PASSWORD=postgres -d postgres  || echo "cannot start postgres, already exists?"
+	sleep 2
+	PGPASSWORD=postgres psql -c "CREATE DATABASE testdb ENCODING 'UTF8' TEMPLATE template0;" -U postgres -h localhost
+
 lint: install-dev
 	$(VENV)/bin/ruff check kinto tests docs/conf.py
 	$(VENV)/bin/ruff format --check kinto tests docs/conf.py
@@ -123,6 +132,8 @@ clean:
 	find . -name '*.pyc' -delete
 	find . -name '__pycache__' -type d | xargs rm -fr
 	rm -fr docs/_build/
+	docker rm -f kinto-memcached || echo ""
+	docker rm -f kinto-postgres || echo ""
 
 distclean: clean
 	rm -fr *.egg *.egg-info/ dist/ build/
diff --git a/constraints.txt b/constraints.txt
index de3faafbd..b6ef5ef27 100644
--- a/constraints.txt
+++ b/constraints.txt
@@ -100,7 +100,7 @@ plaster-pastedeploy==1.0.1
     # via pyramid
 playwright==1.42.0
     # via -r constraints.in
-pluggy==1.3.0
+pluggy==1.4.0
     # via pytest
 psycopg2==2.9.9
     # via -r constraints.in
@@ -121,7 +121,7 @@ pyramid-multiauth==1.0.1
     # via -r constraints.in
 pyramid-tm==2.5
     # via -r constraints.in
-pytest==7.4.4
+pytest==8.1.1
     # via
     #   -r constraints.in
     #   pytest-cache
diff --git a/docs/community.rst b/docs/community.rst
index 3751913c3..a286dcd7d 100644
--- a/docs/community.rst
+++ b/docs/community.rst
@@ -119,7 +119,7 @@ Get started!
 
     git clone git@github.com:your_name_here/kinto.git
 
-3. Setup a local PostgreSQL database for the tests (:ref:`more details <postgresql-install>`)::
+3. Run ``make test-deps``, or setup a local PostgreSQL database (:ref:`more details <postgresql-install>`)::
 
     sudo apt-get install postgresql
     sudo -n -u postgres -s -- psql -c "ALTER USER postgres WITH PASSWORD 'postgres';" -U postgres


Test patch:
diff --git a/tests/core/resource/test_views_cors.py b/tests/core/resource/test_views_cors.py
index 6560dafde..b8d5eaf77 100644
--- a/tests/core/resource/test_views_cors.py
+++ b/tests/core/resource/test_views_cors.py
@@ -16,6 +16,10 @@ def setUpClass(cls):
         super().setUpClass()
         cls.headers["Origin"] = "notmyidea.org"
 
+    @classmethod
+    def tearDownClass(cls):
+        cls.headers.pop("Origin", None)
+
     def setUp(self):
         super().setUp()
         body = {"data": MINIMALIST_OBJECT}
@@ -217,6 +221,11 @@ def setUpClass(cls):
         super().setUpClass()
         cls.headers.update({"Origin": "lolnet.org", "Access-Control-Request-Method": "GET"})
 
+    @classmethod
+    def tearDownClass(cls):
+        cls.headers.pop("Access-Control-Request-Method", None)
+        cls.headers.pop("Origin", None)
+
     def test_cors_max_age_is_3600_seconds_by_default(self):
         app = self.make_app()
         resp = app.options("/", headers=self.headers)
diff --git a/tests/core/test_views_version.py b/tests/test_views_version.py
similarity index 100%
rename from tests/core/test_views_version.py
rename to tests/test_views_version.py