Repository: e-valuation/EvaP

Issue:
Evaluation Edit: Button Groups don't work, submitting fails with "This field is required".
Since the django4 update (#1746, https://github.com/e-valuation/EvaP/commit/f2ce2c5a686070b153549559ac614c6aad0c8e73), going to any evaluation edit page as a staff user and clicking on "Save" fails with
>  This field is required

on the extra form in the contribution formset. The same is probably true for any formset on our page.

At the same time, the button groups on the page can not be used properly.

I don't remember (and currently also cannot find) any explanation of changed behavior regarding formsets on the django4 [release notes](https://docs.djangoproject.com/en/4.0/releases/4.0/). [This ticket](https://code.djangoproject.com/ticket/33534) also implies django's behavior shouldn't have changed. Maybe we do something to the form so that `has_changed()` evaluates to true.

@niklasmohrin is currently working on a minrep example (for maybe creating a django issue).

Edit: Explanation of what's going on is below in https://github.com/e-valuation/EvaP/issues/1769#issuecomment-1146813897.
The root cause is that django4 removed `id_for_label` from choices of a choice field, but we attempt to use it in `choice_buttom.html`. Now, we always put "None" there in the html, breaking both button groups and sortable.js.


Discussion:
Django should ignore forms that have `has_changed() == False`.
However, for our extra form, it says `has_changed() == True`, because the "order" field has changed from its initial value "-1" to whatever value it got assigned by sortable.js (in my debug-run: 2).

Removing the script-tag that loads "sortable.min.js" makes this go away.

That leaves the question: Why did this work before?

Edit: It seems that with django3.2, sortable.js didn't touch the order for the extra formset, so the request.POST data contains "order: -1" (unchanged).

That leaves the question: Why does it touch the order now?

It seems like the order is changed on submit, maybe even using the formdata event. Will have to debug sortable to find out why.
We pass a custom `rowChanged` method to `makeFormSortable`, which is used to detect what a row was changed by the user. If this returns `true`(=`changed`), the hidden order-input is changed.

The implementation in `evaluation_edit.js` uses jquery for this:
https://github.com/e-valuation/EvaP/blob/d3d3bf7c0f854fce206c8dded8f7ab6c74b7b6aa/evap/evaluation/templates/evap_evaluation_edit_js.html#L4-L20

Now, in jquery, if `row.find("some_selector")` returns an empty result, it is still valid to call `.prop("checked")` on it, and it returns `undefined`.

With the django4 update, the row and visibility inputs no longer have their IDs set correctly, so this is what happened: The `checked` test always returned undefined, and negating that returns `true`.

The id is not set correctly anymore because in staff_evaluation_form.py, we use contribution_formset.html, which uses role_buttons.html and textanswer_visibility_buttons.html, which in turn use choice_button.html.

`choice_button.html` tries to access `choice.id_for_label` to get the id for the label. This seems to always be `None` from django 4.0 on. `formelement` is a `BoundField` instance here and has a `id_for_label`, [as is documented](https://docs.djangoproject.com/en/4.0/ref/forms/api/#django.forms.BoundField.id_for_label), but `choice` is on of the `BoundWidget` instances that this field is made up of, so it isn't required by the documentation to have an `id_for_label`. This makes some sense, as from the point of django, they are expecting the widget to be a select input, and there, you usually don't need an ID for each option.

So:
* We need to fix the IDs. Maybe we can build our own id by concatenating `formelement.id_for_label` with some identifying data from the widget. Or, maybe there is a django widget designed for radio-selections, where it's more usual to have a label for each option (and thus need an id).
* jquery and javascript are at it again. We should consider making the whole "rowChanged" implementation more robust. Most of our sortable formsets get a template row passed anyway, so it should be easy to just compare the current form's data with the hidden template row's data, instead of manually checking every field.
This should wait until #1758 is done because the templating is tightly coupled with select2 right now
The same base issue (no more `id_for_label` for the choices of a choice field) also currently breaks the "responsibility" and "text answer visibility" button groups on the evaluation edit page:

All buttons (= html labels) have `for="None"`, and all underlying radio inputs have `id="None"`. Thus, no matter which button you click, it will always select the button button in a button group on the page.

The voting page as well as the results page filtering seem to work fine with their button groups.

Resolving patch:
diff --git a/evap/staff/forms.py b/evap/staff/forms.py
--- a/evap/staff/forms.py
+++ b/evap/staff/forms.py
@@ -579,6 +579,9 @@ class Meta:
         fields = ("evaluation", "contributor", "questionnaires", "role", "textanswer_visibility", "label", "order")
         widgets = {
             "order": forms.HiddenInput(),
+            # RadioSelects are necessary so each value gets a id_for_label, see #1769.
+            "role": forms.RadioSelect(),
+            "textanswer_visibility": forms.RadioSelect(),
         }
 
     def __init__(self, *args, evaluation=None, **kwargs):


Test patch:
diff --git a/evap/staff/tests/test_views.py b/evap/staff/tests/test_views.py
--- a/evap/staff/tests/test_views.py
+++ b/evap/staff/tests/test_views.py
@@ -1756,6 +1756,8 @@ def get_post_params(cls):
     ]
 )
 class TestEvaluationEditView(WebTestStaffMode):
+    render_pages_url = "/staff/semester/PK/evaluation/PK/edit"
+
     @classmethod
     def setUpTestData(cls):
         cls.manager = make_manager()
@@ -1797,6 +1799,12 @@ def setUpTestData(cls):
         cls.contribution1.questionnaires.set([cls.contributor_questionnaire])
         cls.contribution2.questionnaires.set([cls.contributor_questionnaire])
 
+    @render_pages
+    def render_pages(self):
+        return {
+            "normal": self.app.get(self.url, user=self.manager).content,
+        }
+
     def test_edit_evaluation(self):
         page = self.app.get(self.url, user=self.manager)
 
diff --git a/evap/static/ts/tests/frontend/staff-evaluation-edit.ts b/evap/static/ts/tests/frontend/staff-evaluation-edit.ts
new file mode 100644
--- /dev/null
+++ b/evap/static/ts/tests/frontend/staff-evaluation-edit.ts
@@ -0,0 +1,36 @@
+import { test, expect } from "@jest/globals";
+import { ElementHandle } from "puppeteer";
+
+import { pageHandler } from "../utils/page";
+
+// regression test for #1769
+test("changes form data", pageHandler(
+    "/staff/semester/PK/evaluation/PK/edit/normal.html",
+    async page => {
+        const managerId = await page.evaluate(() => {
+            const tomselect = (document.getElementById("id_contributions-0-contributor") as any).tomselect;
+            const options = tomselect.options;
+            const managerOption = Object.keys(options).find(key => options[key].text == "manager (manager)");
+            tomselect.setValue(managerOption);
+            return managerOption;
+        });
+
+        const editorLabels = await page.$x("//label[contains(text(), 'Editor')]");
+        const ownAndGeneralLabels = await page.$x("//label[contains(text(), 'Own and general')]");
+        if (editorLabels.length < 1 || ownAndGeneralLabels.length < 1) {
+          throw new Error("Button group buttons not found.");
+        }
+
+        await (editorLabels[0] as ElementHandle<Element>).click()
+        await (ownAndGeneralLabels[0] as ElementHandle<Element>).click();
+
+        const formData = await page.evaluate(() => {
+            return Object.fromEntries(new FormData(document.getElementById("evaluation-form") as HTMLFormElement));
+        });
+
+        expect(formData["contributions-0-contributor"]).toBe(managerId);
+        expect(formData["contributions-0-order"]).toBe("0");
+        expect(formData["contributions-0-role"]).toBe("1");
+        expect(formData["contributions-0-textanswer_visibility"]).toBe("GENERAL");
+    },
+));