Repository: wagtail/wagtail

Issue:
Support custom configuration options in accessibility checker
## Is your proposal related to a problem?

Currently, the only supported configuration of the accessibility checker is to turn it on or off like all userbar items. With Axe having so many ways to be configured, it’d make sense to expose some of those options to site implementers.

## Describe the solution you'd like

Have a Wagtail-friendly way to configure:

- Which elements of the page the checker runs on – `main, title` for example (so we don’t report issues on the footer)
- Which elements of the page are ignored – for example `footer, .my-cool-widget`. Making sure we retain our ignoring of `wagtail-userbar`
- Which rules are on/off (if possible including the ability to create fully custom Axe rules) – including customising the error message of each rule if possible.

This can either be an almost direct reproduction of the relevant Axe configuration options, or something more opinionated.

In addition, we could consider supporting configuring the checker differently for:

- Different preview modes
- Different user roles / groups
- Page type by page type

## Describe alternatives you've considered

We could do this via Python/Django, templates, or JS. Python/Django feels much more promising.

---

## Implementation approach

In the interest of time, keep using the userbar hook? Add class attributes to define defaults, methods as relevant. To customise, use the hook to swap default `AccessibilityItem` with the subclass. Retain a way to override whole Axe configuration if possible (doesn’t have to be elegant, or documented).

Documentation: TBC (can be done during code freeze)


Discussion:

Resolving patch:
diff --git a/wagtail/admin/userbar.py b/wagtail/admin/userbar.py
--- a/wagtail/admin/userbar.py
+++ b/wagtail/admin/userbar.py
@@ -5,9 +5,12 @@
 class BaseItem:
     template = "wagtailadmin/userbar/item_base.html"
 
+    def get_context_data(self, request):
+        return {"self": self, "request": request}
+
     def render(self, request):
         return render_to_string(
-            self.template, {"self": self, "request": request}, request=request
+            self.template, self.get_context_data(request), request=request
         )
 
 
@@ -24,75 +27,138 @@ def render(self, request):
 
 
 class AccessibilityItem(BaseItem):
+    """A userbar item that runs the accessibility checker."""
+
+    #: The template to use for rendering the item.
     template = "wagtailadmin/userbar/item_accessibility.html"
 
-    def get_axe_configuration(self):
+    #: A list of CSS selector(s) to test specific parts of the page.
+    #: For more details, see `Axe documentation <https://github.com/dequelabs/axe-core/blob/master/doc/context.md#the-include-property>`__.
+    axe_include = ["body"]
+
+    #: A list of CSS selector(s) to exclude specific parts of the page from testing.
+    #: For more details, see `Axe documentation <https://github.com/dequelabs/axe-core/blob/master/doc/context.md#exclude-elements-from-test>`__.
+    axe_exclude = []
+
+    # Make sure that the userbar is not tested.
+    _axe_default_exclude = [{"fromShadowDOM": ["wagtail-userbar"]}]
+
+    #: A list of `axe-core tags <https://github.com/dequelabs/axe-core/blob/master/doc/API.md#axe-core-tags>`_
+    #: or a list of `axe-core rule IDs <https://github.com/dequelabs/axe-core/blob/master/doc/rule-descriptions.md>`_
+    #: (not a mix of both).
+    #: Setting this to a falsy value (e.g. ``None``) will omit the ``runOnly`` option and make Axe run with all non-experimental rules enabled.
+    axe_run_only = [
+        "button-name",
+        "empty-heading",
+        "empty-table-header",
+        "frame-title",
+        "heading-order",
+        "input-button-name",
+        "link-name",
+        "p-as-heading",
+    ]
+
+    #: A dictionary that maps axe-core rule IDs to a dictionary of rule options,
+    #: commonly in the format of ``{"enabled": True/False}``. This can be used in
+    #: conjunction with :attr:`axe_run_only` to enable or disable specific rules.
+    #: For more details, see `Axe documentation <https://github.com/dequelabs/axe-core/blob/master/doc/API.md#options-parameter-examples>`__.
+    axe_rules = {}
+
+    #: A dictionary that maps axe-core rule IDs to custom translatable strings
+    #: to use as the error messages. If an enabled rule does not exist in this
+    #: dictionary, Axe's error message for the rule will be used as fallback.
+    axe_messages = {
+        "button-name": _(
+            "Button text is empty. Use meaningful text for screen reader users."
+        ),
+        "empty-heading": _(
+            "Empty heading found. Use meaningful text for screen reader users."
+        ),
+        "empty-table-header": _(
+            "Table header text is empty. Use meaningful text for screen reader users."
+        ),
+        "frame-title": _(
+            "Empty frame title found. Use a meaningful title for screen reader users."
+        ),
+        "heading-order": _("Incorrect heading hierarchy. Avoid skipping levels."),
+        "input-button-name": _(
+            "Input button text is empty. Use meaningful text for screen reader users."
+        ),
+        "link-name": _(
+            "Link text is empty. Use meaningful text for screen reader users."
+        ),
+        "p-as-heading": _("Misusing paragraphs as headings. Use proper heading tags."),
+    }
+
+    def get_axe_include(self, request):
+        """Returns a list of CSS selector(s) to test specific parts of the page."""
+        return self.axe_include
+
+    def get_axe_exclude(self, request):
+        """Returns a list of CSS selector(s) to exclude specific parts of the page from testing."""
+        return self.axe_exclude + self._axe_default_exclude
+
+    def get_axe_run_only(self, request):
+        """Returns a list of axe-core tags or a list of axe-core rule IDs (not a mix of both)."""
+        return self.axe_run_only
+
+    def get_axe_rules(self, request):
+        """Returns a dictionary that maps axe-core rule IDs to a dictionary of rule options."""
+        return self.axe_rules
+
+    def get_axe_messages(self, request):
+        """Returns a dictionary that maps axe-core rule IDs to custom translatable strings."""
+        return self.axe_messages
+
+    def get_axe_context(self, request):
+        """
+        Returns the `context object <https://github.com/dequelabs/axe-core/blob/develop/doc/context.md>`_
+        to be passed as the
+        `context parameter <https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#context-parameter>`_
+        for ``axe.run``.
+        """
         return {
-            # See https://github.com/dequelabs/axe-core/blob/develop/doc/context.md.
-            "context": {
-                "include": "body",
-                "exclude": {"fromShadowDOM": ["wagtail-userbar"]},
-            },
-            # See https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#options-parameter.
-            "options": {
-                "runOnly": {
-                    "type": "rule",
-                    "values": [
-                        "button-name",
-                        "empty-heading",
-                        "empty-table-header",
-                        "frame-title",
-                        "heading-order",
-                        "input-button-name",
-                        "link-name",
-                        "p-as-heading",
-                    ],
-                }
-            },
-            # Wagtail-specific translatable custom error messages.
-            "messages": {
-                "button-name": _(
-                    "Button text is empty. Use meaningful text for screen reader users."
-                ),
-                "empty-heading": _(
-                    "Empty heading found. Use meaningful text for screen reader users."
-                ),
-                "empty-table-header": _(
-                    "Table header text is empty. Use meaningful text for screen reader users."
-                ),
-                "frame-title": _(
-                    "Empty frame title found. Use a meaningful title for screen reader users."
-                ),
-                "heading-order": _(
-                    "Incorrect heading hierarchy. Avoid skipping levels."
-                ),
-                "input-button-name": _(
-                    "Input button text is empty. Use meaningful text for screen reader users."
-                ),
-                "link-name": _(
-                    "Link text is empty. Use meaningful text for screen reader users."
-                ),
-                "p-as-heading": _(
-                    "Misusing paragraphs as headings. Use proper heading tags."
-                ),
-            },
+            "include": self.get_axe_include(request),
+            "exclude": self.get_axe_exclude(request),
         }
 
-    def render(self, request):
+    def get_axe_options(self, request):
+        """
+        Returns the options object to be passed as the
+        `options parameter <https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#options-parameter>`_
+        for ``axe.run``.
+        """
+        options = {
+            "runOnly": self.get_axe_run_only(request),
+            "rules": self.get_axe_rules(request),
+        }
+        # If the runOnly option is omitted, Axe will run all rules except those
+        # with the "experimental" flag or that are disabled in the rules option.
+        # The runOnly has to be omitted (instead of set to an empty list or null)
+        # for this to work, so we remove it if it's falsy.
+        if not options["runOnly"]:
+            options.pop("runOnly")
+        return options
+
+    def get_axe_configuration(self, request):
+        return {
+            "context": self.get_axe_context(request),
+            "options": self.get_axe_options(request),
+            "messages": self.get_axe_messages(request),
+        }
 
+    def get_context_data(self, request):
+        return {
+            **super().get_context_data(request),
+            "axe_configuration": self.get_axe_configuration(request),
+        }
+
+    def render(self, request):
         # Don't render if user doesn't have permission to access the admin area
         if not request.user.has_perm("wagtailadmin.access_admin"):
             return ""
 
-        return render_to_string(
-            self.template,
-            {
-                "self": self,
-                "request": request,
-                "axe_configuration": self.get_axe_configuration(),
-            },
-            request=request,
-        )
+        return super().render(request)
 
 
 class AddPageItem(BaseItem):


Test patch:
diff --git a/wagtail/admin/tests/test_userbar.py b/wagtail/admin/tests/test_userbar.py
--- a/wagtail/admin/tests/test_userbar.py
+++ b/wagtail/admin/tests/test_userbar.py
@@ -1,9 +1,14 @@
-from django.contrib.auth.models import AnonymousUser
+import json
+
+from bs4 import BeautifulSoup
+from django.contrib.auth.models import AnonymousUser, Permission
 from django.template import Context, Template
 from django.test import TestCase
-from django.test.client import RequestFactory
 from django.urls import reverse
 
+from wagtail import hooks
+from wagtail.admin.userbar import AccessibilityItem
+from wagtail.coreutils import get_dummy_request
 from wagtail.models import PAGE_TEMPLATE_VAR, Page
 from wagtail.test.testapp.models import BusinessChild, BusinessIndex
 from wagtail.test.utils import WagtailTestUtils
@@ -25,7 +30,7 @@ def dummy_request(
         revision_id=None,
         is_editing=False,
     ):
-        request = RequestFactory().get("/")
+        request = get_dummy_request()
         request.user = user or AnonymousUser()
         request.is_preview = is_preview
         request.is_editing = is_editing
@@ -178,24 +183,181 @@ def test_userbar_not_in_preview_panel(self):
         # Make sure nothing was rendered
         self.assertEqual(content, "")
 
-    def test_userbar_accessibility_configuration(self):
+
+class TestAccessibilityCheckerConfig(WagtailTestUtils, TestCase):
+    def setUp(self):
+        self.user = self.login()
+        self.request = get_dummy_request()
+        self.request.user = self.user
+
+    def get_script(self):
         template = Template("{% load wagtailuserbar %}{% wagtailuserbar %}")
-        content = template.render(
-            Context(
-                {
-                    PAGE_TEMPLATE_VAR: self.homepage,
-                    "request": self.dummy_request(self.user),
-                }
-            )
-        )
+        content = template.render(Context({"request": self.request}))
+        soup = BeautifulSoup(content, "html.parser")
 
         # Should include the configuration as a JSON script with the specific id
-        self.assertIn(
-            '<script id="accessibility-axe-configuration" type="application/json">',
-            content,
+        return soup.find("script", id="accessibility-axe-configuration")
+
+    def get_config(self):
+        return json.loads(self.get_script().string)
+
+    def get_hook(self, item_class):
+        def customise_accessibility_checker(request, items):
+            items[:] = [
+                item_class() if isinstance(item, AccessibilityItem) else item
+                for item in items
+            ]
+
+        return customise_accessibility_checker
+
+    def test_config_json(self):
+        script = self.get_script()
+        # The configuration should be a valid non-empty JSON script
+        self.assertIsNotNone(script)
+        self.assertEqual(script.attrs["type"], "application/json")
+        config_string = script.string.strip()
+        self.assertGreater(len(config_string), 0)
+        config = json.loads(config_string)
+        self.assertIsInstance(config, dict)
+        self.assertGreater(len(config.keys()), 0)
+
+    def test_messages(self):
+        # Should include Wagtail's error messages
+        config = self.get_config()
+        self.assertIsInstance(config.get("messages"), dict)
+        self.assertEqual(
+            config["messages"]["empty-heading"],
+            "Empty heading found. Use meaningful text for screen reader users.",
         )
-        # Should include the custom error message
-        self.assertIn("Empty heading found", content)
+
+    def test_custom_message(self):
+        class CustomMessageAccessibilityItem(AccessibilityItem):
+            # Override via class attribute
+            axe_messages = {
+                "empty-heading": "Headings should not be empty!",
+            }
+
+            # Override via method
+            def get_axe_messages(self, request):
+                return {
+                    **super().get_axe_messages(request),
+                    "color-contrast-enhanced": "Increase colour contrast!",
+                }
+
+        with hooks.register_temporarily(
+            "construct_wagtail_userbar",
+            self.get_hook(CustomMessageAccessibilityItem),
+        ):
+            config = self.get_config()
+            self.assertEqual(
+                config["messages"],
+                {
+                    "empty-heading": "Headings should not be empty!",
+                    "color-contrast-enhanced": "Increase colour contrast!",
+                },
+            )
+
+    def test_unset_run_only(self):
+        class UnsetRunOnlyAccessibilityItem(AccessibilityItem):
+            # Example config that unsets the runOnly property so that all
+            # non-experimental rules are run, but the experimental
+            # focus-order-semantics rule is explicitly enabled
+            axe_run_only = None
+            axe_rules = {"focus-order-semantics": {"enabled": True}}
+
+        with hooks.register_temporarily(
+            "construct_wagtail_userbar",
+            self.get_hook(UnsetRunOnlyAccessibilityItem),
+        ):
+            config = self.get_config()
+            self.assertEqual(
+                config["options"],
+                # Should not include the runOnly property, but should include
+                # the focus-order-semantics rule
+                {"rules": {"focus-order-semantics": {"enabled": True}}},
+            )
+
+    def test_custom_context(self):
+        class CustomContextAccessibilityItem(AccessibilityItem):
+            axe_include = ["article", "section"]
+            axe_exclude = [".sr-only"]
+
+            def get_axe_exclude(self, request):
+                return [*super().get_axe_exclude(request), "[data-please-ignore]"]
+
+        with hooks.register_temporarily(
+            "construct_wagtail_userbar",
+            self.get_hook(CustomContextAccessibilityItem),
+        ):
+            config = self.get_config()
+            self.assertEqual(
+                config["context"],
+                {
+                    # Override via class attribute
+                    "include": ["article", "section"],
+                    "exclude": [
+                        # Override via class attribute
+                        ".sr-only",
+                        # Should include the default exclude selectors
+                        {"fromShadowDOM": ["wagtail-userbar"]},
+                        # Override via method
+                        "[data-please-ignore]",
+                    ],
+                },
+            )
+
+    def test_custom_run_only_and_rules_per_request(self):
+        class CustomRunOnlyAccessibilityItem(AccessibilityItem):
+            # Enable all rules within these tags
+            axe_run_only = [
+                "wcag2a",
+                "wcag2aa",
+                "wcag2aaa",
+                "wcag21a",
+                "wcag21aa",
+                "wcag22aa",
+                "best-practice",
+            ]
+            # Turn off the color-contrast-enhanced rule
+            axe_rules = {
+                "color-contrast-enhanced": {"enabled": False},
+            }
+
+            def get_axe_rules(self, request):
+                # Do not turn off any rules for superusers
+                if request.user.is_superuser:
+                    return {}
+                return super().get_axe_rules(request)
+
+        with hooks.register_temporarily(
+            "construct_wagtail_userbar",
+            self.get_hook(CustomRunOnlyAccessibilityItem),
+        ):
+            config = self.get_config()
+            self.assertEqual(
+                config["options"],
+                {
+                    "runOnly": CustomRunOnlyAccessibilityItem.axe_run_only,
+                    "rules": {},
+                },
+            )
+
+            self.user.is_superuser = False
+            self.user.user_permissions.add(
+                Permission.objects.get(
+                    content_type__app_label="wagtailadmin", codename="access_admin"
+                )
+            )
+            self.user.save()
+
+            config = self.get_config()
+            self.assertEqual(
+                config["options"],
+                {
+                    "runOnly": CustomRunOnlyAccessibilityItem.axe_run_only,
+                    "rules": CustomRunOnlyAccessibilityItem.axe_rules,
+                },
+            )
 
 
 class TestUserbarFrontend(WagtailTestUtils, TestCase):