Repository: rdmorganiser/rdmo (language: python)

Issue:
Add front-end tests for react code
### Rationale / Begründung

From discussion in https://github.com/rdmorganiser/rdmo/pull/649:

>Since we use react now, we could also think about adding tests for the front-end. I have no clue about the ecosystem there. This is something for the next year.

_Originally posted by @jochenklar in https://github.com/rdmorganiser/rdmo/issues/649#issuecomment-1723633041_


> I think you can test the frontend code with javascript unit tests, like jest.
> 
> And the test the bundled frontend in combination with a django-liveserver you can use selenium, playwright or cypress nowadays.
> 
> I wrote a couple of functional tests with selenium for another project. But I think playwright might be taking over.
> 
> Refs:
> - https://playwright.dev/
> - https://github.com/microsoft/playwright-pytest
> - https://playwright.dev/docs/codegen-intro
> - https://www.selenium.dev/
> - https://splinter.readthedocs.io/
> - https://jestjs.io/
> 

_Originally posted by @afuetterer in https://github.com/rdmorganiser/rdmo/issues/649#issuecomment-1727470141_

### Affected 

Devs

### Minimal functionality

* functional tests for front-end 
  * multiple browsers, test should just pass
  * Test for mobile, will depend on what is tested
* included in ci actions

### Nice-to-have functionality
* tests of JS code with coverage
* 100 % coverage

### References / Verweise

* https://github.com/rdmorganiser/rdmo/issues/518





Discussion / hints:
I can offer to write a couple of playwright tests in Python, because I wanted to test this for a long time.

These tests would require a webpack bundled frontend and a running live-server pytest-fixture. That is no problem at all.

But it "only" tests the functionality of the frontend, it will not be able to measure coverage of the JS files. For that you might need to write the tests with a JS testing framework (jest, cypress or even playwright js).

What is the priority? High coverage?

Refs:
- https://github.com/microsoft/playwright-pytest
- https://playwright.dev/python/docs/test-runners
- https://github.com/Chris-May/django_playwright_pytest_example
thank you @afuetterer! Think it would be great if you could make the start with just the functional tests. The coverage part is then for later or nice-to-have.
Alright, will do. 

But maybe one of you RDMO-professionals can help me to come up with a meaningful UI test.
What would be a nice "click-path" through the React frontend?
ok, i'll suggest here a start for tests on the new "Management Interface ([docs](https://rdmo.readthedocs.io/en/latest/management/user-interface.html))" that uses the React frontend.

Under `/management/` each content type is known as an element. In the `#sidebar` there is the list of links to each element. Opening each link should show the list of items of each element in the `#main` part.  As a test, maybe you could count the number of items (in the `#list-group`) shown for each element based on the test fixtures. 

Some of the elements also have a special "nested" display (eg. `/catalogs/1/nested/`). As a start, those `/nested/` views could be tested just for functionality.

The edit buttons and the edit form for each element can be tested additionally.
Another typical "click-path" would be to create a new item, fill out some data and save it.

Sidebar, Navigation:

> <ul class="list-unstyled"><li><a href="/management/catalogs/" class="undefined">Catalogs</a></li><li><a href="/management/sections/" class="undefined">Sections</a></li><li><a href="/management/pages/" class="undefined">Pages</a></li><li><a href="/management/questionsets/" class="undefined">Question sets</a></li><li><a href="/management/questions/" class="undefined">Questions</a></li><li><a href="/management/attributes/" class="undefined">Attributes</a></li><li><a href="/management/optionsets/" class="undefined">Option sets</a></li><li><a href="/management/options/" class="undefined">Options</a></li><li><a href="/management/conditions/" class="undefined">Conditions</a></li><li><a href="/management/tasks/" class="undefined">Tasks</a></li><li><a href="/management/views/" class="undefined">Views</a></li></ul>


Here are the so-called "elements" fetched:
https://github.com/rdmorganiser/rdmo/blob/ee6bd484fc17437206fd8128a569dc58785da22b/rdmo/management/assets/js/actions/elementActions.js#L21
and the nested parts:
https://github.com/rdmorganiser/rdmo/blob/ee6bd484fc17437206fd8128a569dc58785da22b/rdmo/management/assets/js/actions/elementActions.js#L113

2cents
Everything @MyPyDavid suggested and, maybe some check that if you click a button (or trigger a redux action) the right API call gets fired to the backend. I don't know how JS testing does that, but to me that would be the biggest benefit of testing the front-end.
Thank you, I will give it a try and let you know if I need more info. I will use an admin user for this, which should definitively have the required permissions, right?

@jochenklar I think the playwright tests will test the output, not so much which API call gets fired. But I will let you know.
Interim Report: playwright is awesome.

I will submit a PR soon.

I figured out how to use pytest-playwright to request the /management section with a logged in admin user and click on all items in the navigation. Works like a charm.

We need to build the webpack frontend in CI then and install a browser to access the live_server from pytest-django.

I still struggle with using the database in multiple tests. The live_server fixture uses transactions and wipes the database after a test is run. That means the first test can access data in rhe db, the second test finds an empty db. Need to do some more research.

I will define a custom "e2e" pytest marker, that can (de)select test cases.

Should I put these tests in management/tests or in a specific e2e tests folder?

I suggest running CI like now:
- lint etc.
- lots of unit tests
- a few e2e tests

Sounds like a plan?
Further discussion in associated PR #769.

Fix patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 369086b358..fe1418d7b4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -92,6 +92,30 @@ jobs:
         GITHUB_DB_BACKEND: ${{ matrix.db-backend }}
         COVERALLS_FLAG_NAME: '${{ matrix.db-backend }}: ${{ matrix.python-version }}'
         COVERALLS_PARALLEL: true
+    # end-to-end tests
+    - uses: actions/setup-node@v4
+      with:
+        node-version: 18
+        cache: npm
+      if: matrix.python-version == '3.12' && matrix.db-backend == 'postgres'
+    - name: Cache Playwright browsers
+      uses: actions/cache@v3
+      with:
+        path: ~/.cache/ms-playwright/
+        key: playwright-browsers
+      if: matrix.python-version == '3.12' && matrix.db-backend == 'postgres'
+    - name: Install e2e tests dependencies
+      run: |
+        npm install
+        npm run build:prod
+        playwright install chromium
+      if: matrix.python-version == '3.12' && matrix.db-backend == 'postgres'
+    - name: Run end-to-end tests
+      run: pytest -p randomly -p no:cacheprovider --reuse-db --numprocesses=auto --dist=loadscope -m e2e --nomigrations
+      if: matrix.python-version == '3.12' && matrix.db-backend == 'postgres'
+      env:
+        DJANGO_DEBUG: True
+        GITHUB_DB_BACKEND: ${{ matrix.db-backend }}
 
   coveralls:
     name: Indicate completion to coveralls
@@ -140,20 +164,6 @@ jobs:
       - run: python -m pip freeze
       - run: python -m pip list --outdated
 
-  webpack-build:
-    name: Test webpack-build
-    needs: lint
-    runs-on: ubuntu-22.04
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-node@v3
-        with:
-          node-version: 18
-          cache: npm
-      - run: npm install --dev
-      - run: npm run build
-      - run: npm run build:prod
-
   required-checks-pass:
     if: always()
     needs:
@@ -162,7 +172,6 @@ jobs:
       - coveralls
       - dev-setup
       - optional-dependencies
-      - webpack-build
     runs-on: ubuntu-22.04
     steps:
       - uses: re-actors/alls-green@release/v1
diff --git a/pyproject.toml b/pyproject.toml
index 9665e8e7c5..7f396bc22f 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -92,6 +92,7 @@ pytest = [
   "pytest-cov~=4.1",
   "pytest-django~=4.5",
   "pytest-mock~=3.11",
+  "pytest-playwright~=0.4.3",
   "pytest-randomly~=3.15",
   "pytest-xdist~=3.3",
 ]
@@ -180,7 +181,10 @@ DJANGO_SETTINGS_MODULE = "config.settings"
 testpaths = ["rdmo"]
 python_files = "test_*[!.txt].py"
 pythonpath = [".", "testing"]
-addopts = "-p no:randomly"
+addopts = '-p no:randomly -m "not e2e"'
+markers = [
+  "e2e: marks tests as end-to-end tests using playwright (deselect with '-m \"not e2e\"')",
+]
 filterwarnings = [
   # fail on RemovedInDjango50Warning exception
   "error::django.utils.deprecation.RemovedInDjango50Warning",


Test patch:
diff --git a/conftest.py b/conftest.py
index b62eb83355..7a2d7ec8d9 100644
--- a/conftest.py
+++ b/conftest.py
@@ -10,29 +10,33 @@
 from rdmo.accounts.utils import set_group_permissions
 
 
+@pytest.fixture(scope="session")
+def fixtures():
+    allowed_file_stems = {
+        'accounts',
+        'conditions',
+        'domain',
+        'groups',
+        'options',
+        'overlays',
+        'projects',
+        'questions',
+        'sites',
+        'tasks',
+        'users',
+        'views'
+    }
+    fixtures = []
+    for fixture_dir in settings.FIXTURE_DIRS:
+        filenames = [filename for filename in Path(fixture_dir).iterdir() if filename.stem in allowed_file_stems]
+        fixtures.extend(filenames)
+    return fixtures
+
+
 @pytest.fixture(scope='session')
-def django_db_setup(django_db_setup, django_db_blocker):
+def django_db_setup(django_db_setup, django_db_blocker, fixtures):
     """Populate database with test data from fixtures directories."""
     with django_db_blocker.unblock():
-        fixtures = []
-        for fixture_dir in settings.FIXTURE_DIRS:
-            for file in Path(fixture_dir).iterdir():
-                if file.stem in [
-                    'accounts',
-                    'conditions',
-                    'domain',
-                    'groups',
-                    'options',
-                    'overlays',
-                    'projects',
-                    'questions',
-                    'sites',
-                    'tasks',
-                    'users',
-                    'views'
-                ]:
-                    fixtures.append(file)
-
         call_command('loaddata', *fixtures)
         set_group_permissions()
 
@@ -49,7 +53,4 @@ def files(settings, tmp_path):
 @pytest.fixture
 def json_data():
     json_file = Path(settings.BASE_DIR) / 'import' / 'catalogs.json'
-    json_data = {
-        'elements': json.loads(json_file.read_text())
-    }
-    return json_data
+    return {'elements': json.loads(json_file.read_text())}
diff --git a/rdmo/management/tests/test_frontend.py b/rdmo/management/tests/test_frontend.py
new file mode 100644
index 0000000000..e4d30cd18a
--- /dev/null
+++ b/rdmo/management/tests/test_frontend.py
@@ -0,0 +1,192 @@
+import os
+import re
+from dataclasses import dataclass
+from urllib.parse import urlparse
+
+import pytest
+
+from django.core.management import call_command
+
+from playwright.sync_api import Page, expect
+from pytest_django.live_server_helper import LiveServer
+
+from rdmo.accounts.utils import set_group_permissions
+from rdmo.conditions.models import Condition
+from rdmo.core.models import Model
+from rdmo.domain.models import Attribute
+from rdmo.options.models import Option, OptionSet
+from rdmo.questions.models import Catalog, Question, Section
+from rdmo.questions.models import Page as PageModel
+from rdmo.questions.models.questionset import QuestionSet
+from rdmo.tasks.models import Task
+from rdmo.views.models import View
+
+pytestmark = pytest.mark.e2e
+
+# needed for playwright to run
+os.environ.setdefault("DJANGO_ALLOW_ASYNC_UNSAFE", "true")
+
+
+@dataclass
+class ModelHelper:
+    """Helper class to bundle information about models for test cases."""
+
+    model: Model
+    form_field: str = "URI Path"
+    db_field: str = "uri_path"
+    has_nested: bool = False
+
+    @property
+    def url(self) -> str:
+        return f"{self.model._meta.model_name}s"
+
+    @property
+    def verbose_name(self) -> str:
+        """Return the verbose_name for the model."""
+        return self.model._meta.verbose_name
+
+    @property
+    def verbose_name_plural(self) -> str:
+        """Return the verbose_name_plural for the model."""
+        return self.model._meta.verbose_name_plural
+
+
+@pytest.fixture(scope="function")
+def e2e_tests_django_db_setup(django_db_setup, django_db_blocker, fixtures):
+    """Set up database and populate with fixtures, that get restored for every test case."""
+    with django_db_blocker.unblock():
+        call_command("loaddata", *fixtures)
+        set_group_permissions()
+
+
+@pytest.fixture(scope="session")
+def base_url(live_server: LiveServer) -> str:
+    """Enable playwright to address URLs with base URL automatically prefixed."""
+    return live_server.url
+
+
+@pytest.fixture
+def logged_in_admin_user(e2e_tests_django_db_setup, page: Page) -> Page:
+    """Log in as admin user through django login UI, returns logged in page for e2e tests."""
+    page.goto("/account/login")
+    page.get_by_label("Username").fill("admin", timeout=5000)
+    page.get_by_label("Password").fill("admin")
+    page.get_by_role("button", name="Login").click()
+    page.goto("/management")
+    yield page
+
+
+model_helpers = (
+    ModelHelper(Catalog, has_nested=True),
+    ModelHelper(Section, has_nested=True),
+    ModelHelper(PageModel, has_nested=True),
+    ModelHelper(QuestionSet, has_nested=True),
+    ModelHelper(Question),
+    ModelHelper(
+        Attribute, has_nested=True, form_field="Key", db_field="key"
+    ),
+    ModelHelper(OptionSet, has_nested=True),
+    ModelHelper(Option),
+    ModelHelper(Condition),
+    ModelHelper(Task),
+    ModelHelper(View),
+)
+
+@pytest.mark.parametrize("helper", model_helpers)
+def test_management_navigation(logged_in_admin_user: Page, helper: ModelHelper) -> None:
+    """Test that each content type is available through the navigation."""
+    page = logged_in_admin_user
+    expect(page.get_by_role("heading", name="Management")).to_be_visible()
+
+    # click a link in the navigation
+    name = helper.verbose_name_plural
+    page.get_by_role("link", name=name, exact=True).click()
+
+    # make sure the browser opened a new page
+    url_name = name.lower()
+    url_name = url_name.replace(" ", "")
+    expect(page).to_have_url(re.compile(rf".*/{url_name}/"))
+
+
+@pytest.mark.parametrize("helper", model_helpers)
+def test_management_has_items(logged_in_admin_user: Page, helper: ModelHelper) -> None:
+    """Test all items in database are visible in management UI."""
+    page = logged_in_admin_user
+    num_items_in_database = helper.model.objects.count()
+    page.goto(f"/management/{helper.url}")
+    items_in_ui = page.locator(".list-group > .list-group-item")
+    expect(items_in_ui).to_have_count(num_items_in_database)
+
+
+@pytest.mark.parametrize("helper", model_helpers)
+def test_management_nested_view(
+    logged_in_admin_user: Page, helper: ModelHelper
+) -> None:
+    """For each element type, that has a nested view, click the first example."""
+    page = logged_in_admin_user
+    page.goto(f"/management/{helper.url}")
+    # Open nested view for element type
+    if helper.has_nested:
+        page.get_by_title(f"View {helper.verbose_name} nested").first.click()
+        expect(page.locator(".panel-default").first).to_be_visible()
+        expect(page.locator(".panel-default > .panel-body").first).to_be_visible()
+
+
+@pytest.mark.parametrize("helper", model_helpers)
+def test_management_create_model(
+    logged_in_admin_user: Page, helper: ModelHelper
+) -> None:
+    """Test management UI can create objects in the database."""
+    page = logged_in_admin_user
+    num_objects_at_start = helper.model.objects.count()
+    page.goto(f"/management/{helper.url}")
+    # click "New" button
+    page.get_by_role("button", name="New").click()
+    # fill mandatory fields
+    value = "some-value"
+    page.get_by_label(helper.form_field).fill(value)
+    if helper.model == Condition:
+        # conditions need to have a source attribute
+        source_form = (
+            page.locator(".form-group")
+            .filter(has_text="Source")
+            .locator(".select-item > .react-select")
+        )
+        source_form.click()
+        page.keyboard.type(Attribute.objects.first().uri)
+        page.keyboard.press("Enter")
+
+    # save
+    page.get_by_role("button", name="Create").nth(1).click()
+    # check if new item is in list
+    items_in_ui = page.locator(".list-group > .list-group-item")
+    expect(items_in_ui).to_have_count(num_objects_at_start + 1)
+
+    num_objects_after_save = helper.model.objects.count()
+    assert num_objects_after_save - num_objects_at_start == 1
+    query = {helper.db_field: value}
+    assert helper.model.objects.get(**query)
+
+
+@pytest.mark.parametrize("helper", model_helpers)
+def test_management_edit_model(logged_in_admin_user: Page, helper: ModelHelper) -> None:
+    page = logged_in_admin_user
+    page.goto(f"/management/{helper.url}")
+    # click edit
+    edit_button_title = f"Edit {helper.verbose_name}"
+    page.get_by_title(f"{edit_button_title}").first.click()
+    # edit
+    page.get_by_label("Comment").click()
+    comment = "this is a comment."
+    page.get_by_label("Comment").fill(comment)
+    # save
+    page.get_by_role("button", name="Save").nth(1).click()
+    # click on edit element again
+    page.get_by_title(f"{edit_button_title}").first.click()
+    # check the updated comment
+    comment_locator = page.get_by_label("Comment")
+    expect(comment_locator).to_have_text(comment)
+    # compare the updated comment with element object from db
+    url_id = int(urlparse(page.url).path.rstrip("/").split("/")[-1])
+    model_obj = helper.model.objects.get(id=url_id)
+    assert model_obj.comment == comment
diff --git a/testing/config/settings/base.py b/testing/config/settings/base.py
index 270ec4ea20..e39edf6c80 100644
--- a/testing/config/settings/base.py
+++ b/testing/config/settings/base.py
@@ -2,7 +2,7 @@
 
 from django.utils.translation import gettext_lazy as _
 
-DEBUG = False
+DEBUG = os.getenv("DJANGO_DEBUG", False) == "True"
 TEMPLATE_DEBUG = False
 DEBUG_LOGGING = False