Repository: freedomofpress/securedrop

Issue:
semantic (HTML5/ARIA) page structure in Journalist Interface
# Description

[Accessibility Lab recommendations](https://docs.google.com/document/d/1jZlKco2YgOrrI4PmTkG_Q8lKuV5VxCds/edit#) include:

- `h1`...`hn` hierarchy
- HTML5 section elements
- ARIA roles ("landmarks"), labels, and descriptions

Our findings include:

- moving decorative `img` elements to CSS styles on semantic elements
- Keep in mind per @eloquence in <https://github.com/freedomofpress/securedrop/issues/5972#issuecomment-947317684>:
    > One thing we may want to look into for the next round of changes is how to minimize the need to re-translate strings - e.g., should we generate our own translations in cases where only HTML elements or attributes change (can we do so in the main repo and then import these into Weblate?).

## User Research Evidence

[WCAG 1.3.1](https://www.w3.org/TR/UNDERSTANDING-WCAG20/content-structure-separation-programmatic.html) per #5972.

## Progress

To invert the way things are usually done :-):

1. [x] **First pass:**  Refactor for semantic markup (test in developer tools and screen-readers) and keep `journalist.js` in sync
    - [x] decorative `img`/`hr`/etc. elements: comment out as `FIXME`s to move to CSS styles on semantic elements 
    - [x] linters pass
    - [x] tests pass
2. [x] **Second pass:**  Audit in validators and accessibility tools ([axe](https://www.deque.com/axe/), [WAVE](https://wave.webaim.org/)), e.g.:
    - `h1`...`hn` hierarchy
    - HTML5 section elements
    - ARIA roles ("landmarks"), labels, and descriptions
3. [x] Accessibility-focused review of draft PR before continuing further
4. [x] Review string changes for translation per <https://github.com/freedomofpress/securedrop/issues/5972#issuecomment-947317684> 
5. [x] **Third pass:**  CSS and images broken in first pass (test visually in Tor Browser)
    - [x] `FIXME`s
        - [x] Revert 6d4602da288b398dea3680dfb9faba67cf11e85a
    - [x] tests pass
6. [x] Write and run through test plan in #6165

| Path | First Pass | Second Pass | Third Pass |
| --- | --- | --- | --- |
| `securedrop/journalist_templates` | ✓ | ✓ | ✓ |
| `├──_confirmation_modal.html` | ✓ | ✓| ✓ |
| `├──_source_row.html` | ✓ | ✓ | ✓ |
| `├──_sources_confirmation_final_modal.html` | ✓ | ✓ | ✓ |
| `├──_sources_confirmation_modal.html` | ✓ | ✓ | ✓ |
| `├──account_edit_hotp_secret.html` | ✓ | ✓ | ✓ |
| `├──account_new_two_factor.html` | ✓ | ✓ | ✓ |
| `├──admin.html` | ✓ | ✓ | ✓ |
| `├──admin_add_user.html` | ✓ | ✓ | ✓ |
| `├──admin_edit_hotp_secret.html` | ✓ | ✓ | ✓ |
| `├──admin_new_user_two_factor.html` | ✓ | ✓ | ✓ |
| `├──base.html` | ✓ | ✓ | ✓ |
| `├──col.html` | ✓ | ✓ | ✓ |
| `├──config.html` | ✓ | ✓ | ✓ |
| `├──delete.html` | ✓ | ✓ | ✓ |
| `├──edit_account.html` | ✓ | ✓ | ✓ |
| `├──error.html` | ✓ | ✓ | ✓ |
| `├──flashed.html` | ✓ | ✓ | ✓ |
| `├──index.html` | ✓ | ✓ | ✓ |
| `├──js-strings.html` | ✓ | ✓ | ✓ |
| `├──locales.html` | ✓ | ✓ | ✓ |
| `├──login.html` | ✓ | ✓ | ✓ |
| `└──preferences_saved_flash.html` | ✓ | ✓ | ✓ |

## Specific cases

- [x] Address review feedback in https://github.com/freedomofpress/securedrop/pull/6041#discussion_r673883016
- [x] Refactor `*modal.html` as `dialog`s à la Source Interface's `.{confirm,hidden}-prompt`

### `add_user.html`
- [x] `h1` should not be `.visually-hidden`

### `col.html`
- [x] Listing of a source collection could be a semantic `table`
- [x] `securedrop/journalist_templates/col.html`: `p.breadcrumbs` is a set of unstructured inline elements; could be `ul` styled with `li:{before,after}` selectors
- [x] `div dialog`s are still being read (unlike those in `index.html`)

### `index.html`
- [x] List of sources could be a semantic `table`
    - [x] `td.filename` should wrap more elegantly (or not at all)


Discussion:

Resolving patch:
diff --git a/securedrop/journalist_app/forms.py b/securedrop/journalist_app/forms.py
--- a/securedrop/journalist_app/forms.py
+++ b/securedrop/journalist_app/forms.py
@@ -93,9 +93,13 @@ class NewUserForm(FlaskForm):
     username = StringField('username', validators=[
         InputRequired(message=gettext('This field is required.')),
         minimum_length_validation, check_invalid_usernames
-    ])
-    first_name = StringField('first_name', validators=[name_length_validation, Optional()])
-    last_name = StringField('last_name', validators=[name_length_validation, Optional()])
+        ],
+        render_kw={'aria-describedby': 'username-notes'},
+    )
+    first_name = StringField('first_name', validators=[name_length_validation, Optional()],
+                             render_kw={'aria-describedby': 'name-notes'})
+    last_name = StringField('last_name', validators=[name_length_validation, Optional()],
+                            render_kw={'aria-describedby': 'name-notes'})
     password = HiddenField('password')
     is_admin = BooleanField('is_admin')
     is_hotp = BooleanField('is_hotp')


Test patch:
diff --git a/securedrop/tests/functional/journalist_navigation_steps.py b/securedrop/tests/functional/journalist_navigation_steps.py
--- a/securedrop/tests/functional/journalist_navigation_steps.py
+++ b/securedrop/tests/functional/journalist_navigation_steps.py
@@ -93,7 +93,7 @@ def _login_user(self, username, password, otp, maxtries=3):
             self._try_login_user(username, password, token)
             # Successful login should redirect to the index
             self.wait_for(
-                lambda: self.driver.find_element_by_id("logout"), timeout=self.timeout * 2
+                lambda: self.driver.find_element_by_id("link-logout"), timeout=self.timeout * 2
             )
             if self.driver.current_url != self.journalist_location + "/":
                 new_token = str(otp.now())
@@ -122,11 +122,11 @@ def _journalist_logs_in(self):
         assert self._is_on_journalist_homepage()
 
     def _journalist_visits_col(self):
-        self.wait_for(lambda: self.driver.find_element_by_id("cols"))
+        self.wait_for(lambda: self.driver.find_element_by_css_selector("table#collections"))
 
         self.safe_click_by_id("un-starred-source-link-1")
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("ul#submissions"))
+        self.wait_for(lambda: self.driver.find_element_by_css_selector("table#submissions"))
 
     def _journalist_selects_first_doc(self):
         self.safe_click_by_css_selector('input[type="checkbox"][name="doc_names_selected"]')
@@ -197,7 +197,7 @@ def _journalist_clicks_delete_link(self, click_id, displayed_id):
         self.wait_for(lambda: self.driver.find_element_by_id(displayed_id))
 
     def _journalist_clicks_delete_selected_link(self):
-        self.safe_click_by_css_selector("a#delete-selected-link > button.danger")
+        self.safe_click_by_css_selector("a#delete-selected-link")
         self.wait_for(lambda: self.driver.find_element_by_id("delete-selected-confirmation-modal"))
 
     def _journalist_clicks_delete_collections_link(self):
@@ -298,9 +298,9 @@ def one_source_no_files():
                 flash_msg = self.driver.find_element_by_css_selector(".flash")
                 assert "The files and messages have been deleted" in flash_msg.text
             if not self.accept_languages:
-                count_div = self.driver.find_element_by_css_selector(".submission-count")
-                assert "0 docs" in count_div.text
-                assert "0 messages" in count_div.text
+                counts = self.driver.find_elements_by_css_selector(".submission-count")
+                assert "0 docs" in counts[0].text
+                assert "0 messages" in counts[1].text
 
         self.wait_for(one_source_no_files)
         time.sleep(5)
@@ -471,7 +471,7 @@ def user_token_added():
     def _admin_deletes_user(self):
         for i in range(CLICK_ATTEMPTS):
             try:
-                self.safe_click_by_css_selector(".delete-user")
+                self.safe_click_by_css_selector(".delete-user a")
                 self.wait_for(
                     lambda: expected_conditions.element_to_be_clickable((By.ID, "delete-selected"))
                 )
@@ -570,11 +570,8 @@ def edit_page_loaded():
     def _edit_user(self, username, is_admin=False):
         self.wait_for(lambda: self.driver.find_element_by_id("users"))
 
-        new_user_edit_links = [
-            el
-            for el in self.driver.find_elements_by_tag_name("a")
-            if el.get_attribute("data-username") == username
-        ]
+        selector = 'a.edit-user[data-username="{}"]'.format(username)
+        new_user_edit_links = self.driver.find_elements_by_css_selector(selector)
 
         assert 1 == len(new_user_edit_links)
         new_user_edit_links[0].click()
@@ -622,15 +619,12 @@ def _admin_editing_invalid_username(self):
         # Go to the admin interface
         self.safe_click_by_id("link-admin-index")
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("button#add-user"))
+        self.wait_for(lambda: self.driver.find_element_by_id("add-user"))
 
         # Click the "edit user" link for the new user
         # self._edit_user(self.new_user['username'])
-        new_user_edit_links = [
-            el
-            for el in self.driver.find_elements_by_tag_name("a")
-            if (el.get_attribute("data-username") == self.new_user["username"])
-        ]
+        selector = 'a.edit-user[data-username="{}"]'.format(self.new_user["username"])
+        new_user_edit_links = self.driver.find_elements_by_css_selector(selector)
         assert len(new_user_edit_links) == 1
         new_user_edit_links[0].click()
 
@@ -665,15 +659,12 @@ def _admin_can_edit_new_user(self):
         # Go to the admin interface
         self.safe_click_by_id("link-admin-index")
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("button#add-user"))
+        self.wait_for(lambda: self.driver.find_element_by_id("add-user"))
 
         # Click the "edit user" link for the new user
         # self._edit_user(self.new_user['username'])
-        new_user_edit_links = [
-            el
-            for el in self.driver.find_elements_by_tag_name("a")
-            if (el.get_attribute("data-username") == self.new_user["username"])
-        ]
+        selector = 'a.edit-user[data-username="{}"]'.format(self.new_user["username"])
+        new_user_edit_links = self.driver.find_elements_by_css_selector(selector)
         assert len(new_user_edit_links) == 1
         new_user_edit_links[0].click()
 
@@ -723,9 +714,9 @@ def can_edit_user2():
         # Go to the admin interface
         self.safe_click_by_id("link-admin-index")
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("button#add-user"))
+        self.wait_for(lambda: self.driver.find_element_by_id("add-user"))
 
-        selector = 'a[data-username="{}"]'.format(self.new_user["username"])
+        selector = 'a.edit-user[data-username="{}"]'.format(self.new_user["username"])
         new_user_edit_links = self.driver.find_elements_by_css_selector(selector)
         assert len(new_user_edit_links) == 1
         self.safe_click_by_css_selector(selector)
@@ -760,7 +751,7 @@ def _journalist_checks_messages(self):
 
         if not self.accept_languages:
             # There should be a "1 unread" span in the sole collection entry
-            unread_span = self.driver.find_element_by_css_selector("span.unread")
+            unread_span = self.driver.find_element_by_css_selector("tr.unread")
             assert "1 unread" in unread_span.text
 
     def _journalist_stars_and_unstars_single_message(self):
@@ -808,7 +799,7 @@ def _journalist_selects_all_documents(self):
     def _journalist_downloads_message(self):
         self._journalist_selects_the_first_source()
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("ul#submissions"))
+        self.wait_for(lambda: self.driver.find_element_by_css_selector("table#submissions"))
 
         submissions = self.driver.find_elements_by_css_selector("#submissions a")
         assert 1 == len(submissions)
@@ -840,7 +831,7 @@ def cookie_string_from_selenium_cookies(cookies):
     def _journalist_downloads_message_missing_file(self):
         self._journalist_selects_the_first_source()
 
-        self.wait_for(lambda: self.driver.find_element_by_css_selector("ul#submissions"))
+        self.wait_for(lambda: self.driver.find_element_by_css_selector("table#submissions"))
 
         submissions = self.driver.find_elements_by_css_selector("#submissions a")
         assert 1 == len(submissions)
@@ -915,14 +906,14 @@ def _visit_edit_totp_secret(self):
         )
 
     def _admin_visits_add_user(self):
-        add_user_btn = self.driver.find_element_by_css_selector("button#add-user")
+        add_user_btn = self.driver.find_element_by_id("add-user")
         self.wait_for(lambda: add_user_btn.is_enabled() and add_user_btn.is_displayed())
         add_user_btn.click()
 
         self.wait_for(lambda: self.driver.find_element_by_id("username"))
 
     def _admin_visits_edit_user(self):
-        selector = 'a[data-username="{}"]'.format(self.new_user["username"])
+        selector = 'a.edit-user[data-username="{}"]'.format(self.new_user["username"])
         new_user_edit_links = self.driver.find_elements_by_css_selector(selector)
         assert len(new_user_edit_links) == 1
         self.safe_click_by_css_selector(selector)
@@ -972,9 +963,9 @@ def _admin_visits_reset_2fa_hotp_step():
             time.sleep(1)
 
             tip_opacity = self.driver.find_elements_by_css_selector(
-                "#button-reset-two-factor-hotp span")[0].value_of_css_property('opacity')
+                "#button-reset-two-factor-hotp span.tooltip")[0].value_of_css_property('opacity')
             tip_text = self.driver.find_elements_by_css_selector(
-                "#button-reset-two-factor-hotp span")[0].text
+                "#button-reset-two-factor-hotp span.tooltip")[0].text
 
             assert tip_opacity == "1"
 
@@ -1005,9 +996,9 @@ def _admin_visits_reset_2fa_totp_step():
             time.sleep(1)
 
             tip_opacity = self.driver.find_elements_by_css_selector(
-                "#button-reset-two-factor-totp span")[0].value_of_css_property('opacity')
+                "#button-reset-two-factor-totp span.tooltip")[0].value_of_css_property('opacity')
             tip_text = self.driver.find_elements_by_css_selector(
-                "#button-reset-two-factor-totp span")[0].text
+                "#button-reset-two-factor-totp span.tooltip")[0].text
 
             assert tip_opacity == "1"
             if not self.accept_languages:
@@ -1068,7 +1059,7 @@ def _journalist_delete_none(self):
 
     def _journalist_delete_all_confirmation(self):
         self.safe_click_all_by_css_selector("[name=doc_names_selected]")
-        self.safe_click_by_css_selector("a#delete-selected-link > button.danger")
+        self.safe_click_by_css_selector("a#delete-selected-link")
 
     def _journalist_delete_one(self):
         self.safe_click_by_css_selector("[name=doc_names_selected]")
@@ -1209,7 +1200,7 @@ def _journalist_sees_missing_file_error_message(self, single_file=False):
             )
 
         if self.accept_languages is None:
-            assert notification.text == error_msg
+            assert notification.text in error_msg
 
     def _journalist_is_on_collection_page(self):
         return self.wait_for(
@@ -1217,7 +1208,9 @@ def _journalist_is_on_collection_page(self):
         )
 
     def _journalist_clicks_source_unread(self):
-        self.driver.find_element_by_css_selector("span.unread a").click()
+        self.driver.find_element_by_css_selector(
+            "table#collections tr.source > td.unread a"
+            ).click()
 
     def _journalist_selects_first_source_then_download_all(self):
         checkboxes = self.driver.find_elements_by_name("cols_selected")
diff --git a/securedrop/tests/test_integration.py b/securedrop/tests/test_integration.py
--- a/securedrop/tests/test_integration.py
+++ b/securedrop/tests/test_integration.py
@@ -68,19 +68,19 @@ def test_submit_message(journalist_app, source_app, test_journo):
 
         # The source should have a "download unread" link that
         # says "1 unread"
-        col = soup.select('ul#cols > li')[0]
-        unread_span = col.select('span.unread a')[0]
+        col = soup.select('table#collections tr.source')[0]
+        unread_span = col.select('td.unread a')[0]
         assert "1 unread" in unread_span.get_text()
 
-        col_url = soup.select('ul#cols > li a')[0]['href']
+        col_url = soup.select('table#collections th.designation a')[0]['href']
         resp = app.get(col_url)
         assert resp.status_code == 200
         text = resp.data.decode('utf-8')
         soup = BeautifulSoup(text, 'html.parser')
-        submission_url = soup.select('ul#submissions li a')[0]['href']
+        submission_url = soup.select('table#submissions th.filename a')[0]['href']
         assert "-msg" in submission_url
-        span = soup.select('ul#submissions li span.info span')[0]
-        assert re.compile(r'\d+ bytes').match(span['title'])
+        size = soup.select('table#submissions td.info')[0]
+        assert re.compile(r'\d+ bytes').match(size['title'])
 
         resp = app.get(submission_url)
         assert resp.status_code == 200
@@ -97,7 +97,8 @@ def test_submit_message(journalist_app, source_app, test_journo):
         text = resp.data.decode('utf-8')
         soup = BeautifulSoup(text, 'html.parser')
         doc_name = soup.select(
-            'ul > li > input[name="doc_names_selected"]')[0]['value']
+            'table#submissions > tr.submission > td.status input[name="doc_names_selected"]'
+            )[0]['value']
         resp = app.post('/bulk', data=dict(
             action='confirm_delete',
             filesystem_id=filesystem_id,
@@ -170,19 +171,19 @@ def test_submit_file(journalist_app, source_app, test_journo):
 
         # The source should have a "download unread" link that says
         # "1 unread"
-        col = soup.select('ul#cols > li')[0]
-        unread_span = col.select('span.unread a')[0]
+        col = soup.select('table#collections tr.source')[0]
+        unread_span = col.select('td.unread a')[0]
         assert "1 unread" in unread_span.get_text()
 
-        col_url = soup.select('ul#cols > li a')[0]['href']
+        col_url = soup.select('table#collections th.designation a')[0]['href']
         resp = app.get(col_url)
         assert resp.status_code == 200
         text = resp.data.decode('utf-8')
         soup = BeautifulSoup(text, 'html.parser')
-        submission_url = soup.select('ul#submissions li a')[0]['href']
+        submission_url = soup.select('table#submissions th.filename a')[0]['href']
         assert "-doc" in submission_url
-        span = soup.select('ul#submissions li span.info span')[0]
-        assert re.compile(r'\d+ bytes').match(span['title'])
+        size = soup.select('table#submissions td.info')[0]
+        assert re.compile(r'\d+ bytes').match(size['title'])
 
         resp = app.get(submission_url)
         assert resp.status_code == 200
@@ -206,7 +207,8 @@ def test_submit_file(journalist_app, source_app, test_journo):
         text = resp.data.decode('utf-8')
         soup = BeautifulSoup(text, 'html.parser')
         doc_name = soup.select(
-            'ul > li > input[name="doc_names_selected"]')[0]['value']
+            'table#submissions > tr.submission > td.status input[name="doc_names_selected"]'
+            )[0]['value']
         resp = app.post('/bulk', data=dict(
             action='confirm_delete',
             filesystem_id=filesystem_id,
@@ -273,7 +275,7 @@ def _helper_test_reply(journalist_app, source_app, config, test_journo,
         text = resp.data.decode('utf-8')
         assert "Sources" in text
         soup = BeautifulSoup(resp.data, 'html.parser')
-        col_url = soup.select('ul#cols > li a')[0]['href']
+        col_url = soup.select('table#collections tr.source > th.designation a')[0]['href']
 
         resp = app.get(col_url)
         assert resp.status_code == 200
@@ -458,7 +460,7 @@ def test_delete_collection(mocker, source_app, journalist_app, test_journo):
         resp = app.get('/')
         # navigate to the collection page
         soup = BeautifulSoup(resp.data.decode('utf-8'), 'html.parser')
-        first_col_url = soup.select('ul#cols > li a')[0]['href']
+        first_col_url = soup.select('table#collections tr.source > th.designation a')[0]['href']
         resp = app.get(first_col_url)
         assert resp.status_code == 200
 
@@ -561,7 +563,7 @@ def test_filenames(source_app, journalist_app, test_journo):
         _login_user(app, test_journo)
         resp = app.get('/')
         soup = BeautifulSoup(resp.data.decode('utf-8'), 'html.parser')
-        first_col_url = soup.select('ul#cols > li a')[0]['href']
+        first_col_url = soup.select('table#collections tr.source > th.designation a')[0]['href']
         resp = app.get(first_col_url)
         assert resp.status_code == 200
 
@@ -569,7 +571,7 @@ def test_filenames(source_app, journalist_app, test_journo):
         soup = BeautifulSoup(resp.data.decode('utf-8'), 'html.parser')
         submission_filename_re = r'^{0}-[a-z0-9-_]+(-msg|-doc\.gz)\.gpg$'
         for i, submission_link in enumerate(
-                soup.select('ul#submissions li a .filename')):
+                soup.select('table#submissions tr.submission > th.filename a')):
             filename = str(submission_link.contents[0])
             assert re.match(submission_filename_re.format(i + 1), filename)
 
@@ -588,7 +590,7 @@ def test_filenames_delete(journalist_app, source_app, test_journo):
         _login_user(app, test_journo)
         resp = app.get('/')
         soup = BeautifulSoup(resp.data.decode('utf-8'), 'html.parser')
-        first_col_url = soup.select('ul#cols > li a')[0]['href']
+        first_col_url = soup.select('table#collections tr.source > th.designation a')[0]['href']
         resp = app.get(first_col_url)
         assert resp.status_code == 200
         soup = BeautifulSoup(resp.data.decode('utf-8'), 'html.parser')
@@ -601,13 +603,13 @@ def test_filenames_delete(journalist_app, source_app, test_journo):
         # test filenames and sort order
         submission_filename_re = r'^{0}-[a-z0-9-_]+(-msg|-doc\.gz)\.gpg$'
         filename = str(
-            soup.select('ul#submissions li a .filename')[0].contents[0])
+            soup.select('table#submissions tr.submission > th.filename a')[0].contents[0])
         assert re.match(submission_filename_re.format(1), filename)
         filename = str(
-            soup.select('ul#submissions li a .filename')[1].contents[0])
+            soup.select('table#submissions tr.submission > th.filename a')[1].contents[0])
         assert re.match(submission_filename_re.format(3), filename)
         filename = str(
-            soup.select('ul#submissions li a .filename')[2].contents[0])
+            soup.select('table#submissions tr.submission > th.filename a')[2].contents[0])
         assert re.match(submission_filename_re.format(4), filename)