Repository: comic/grand-challenge.org

Issue:
Remove validation for shuffling cases + case navigation
Continuing from:
- https://github.com/DIAGNijmegen/rse-roadmap/issues/12#issuecomment-1118667097

Disabling validation was based on the assumption that when shuffling an `index` would not be provided because of performance issues.

However, a quick test of the endpoint on a test reader-study would suggest it already works. Not sure if this was solved by @MikeOverkamp-diag since the validation was introduced in:
- https://github.com/comic/grand-challenge.org/pull/2415

From the CIRRUS side, it seems there are no explicit checks and it should already work.


Discussion:
It was not, the current index in the detail view when the ordering is shuffled is incorrect. The reason is that in order to determine the index in the case of shuffling, we need to determine the index in the full list of display sets. That means that we would need to load the entire list of display sets for a reader study to determine the index in the detail view.
> It was not, the current index in the detail view when the ordering is shuffled is incorrect. 

That's when we return `None`, right? So that value is never used.

I still don't really understand why this combination isn't allowed. Can you walk me through this? Some documentation and a sequence diagram would really help.
The check in gc was added because of this comment: https://github.com/DIAGNijmegen/rse-roadmap/issues/135#issuecomment-1105199651 (ans the ones following that). There is no reason that it shouldn't be allowed, other than the fact that the index on the detail view is not set, so that cannot be used.
So is it used? I don't see what the problem is. That comment does not make things any clearer to me.
As far as I know it is not used in cirrus, @chrisvanrun can you confirm? I just went off what was discussed in that issue and added  the check.
To clarify: this only holds for the detail view. Using the list view with a `limit=1` filter works as it should. I was under the impression that that would be used in cirrus.
It seems like some confusion happened between list and detail views and the support therein for `index`. (Since I didn't realise the detail view doesn't provide it.)

Detail view is never used by CIRRUS. At least, currently, that will likely change when we implement loading a specific display set per:
- https://github.com/DIAGNijmegen/rse-roadmap/issues/148

That puts us in a bit of bind:

## Regular reader-study loading
Can easily and already support `allow_case_navigation` even when shuffling the cases.

## Directly loading a display-set
No (current) way of knowing the index of the display-set in the reader study. This would break a few features in CIRRUS, including but not limited to:
* Case navigation plug-in, partial fixable but only by replacing it with a more naked got-to-index `x` functionality.
* Pre-loading images for next cases, thus resulting in slower case-to-case transitions

I suggest we bite the proverbial bullet and take the performance hit for a detail view and generate the index.
Already fixed here: https://github.com/comic/grand-challenge.org/pull/2450
If it was fixed, why does the validation still exist?

https://github.com/comic/grand-challenge.org/blob/d6d79ec44ea5c80e64b48179310d4bea007a330c/app/grandchallenge/reader_studies/forms.py#L150-L161
Ah sorry, you ar right. I'm looking into it now and it actually seems like the index no longer works on the list view if `shuffle_hanging_list` is set to True. 
Nevermind, I didn't specify a reader study, so it makes sense.

Resolving patch:
diff --git a/app/grandchallenge/reader_studies/forms.py b/app/grandchallenge/reader_studies/forms.py
--- a/app/grandchallenge/reader_studies/forms.py
+++ b/app/grandchallenge/reader_studies/forms.py
@@ -147,18 +147,6 @@ def clean(self):
                 ),
                 field=None,
             )
-        if (
-            self.cleaned_data["shuffle_hanging_list"]
-            and self.cleaned_data["allow_case_navigation"]
-        ):
-            self.add_error(
-                error=ValidationError(
-                    "Shuffled hanging lists cannot be used together with "
-                    "case navigation.",
-                    code="invalid",
-                ),
-                field=None,
-            )
         if self.cleaned_data["roll_over_answers_for_n_cases"] > 0 and (
             self.cleaned_data["allow_case_navigation"]
             or self.cleaned_data["shuffle_hanging_list"]
diff --git a/app/grandchallenge/reader_studies/serializers.py b/app/grandchallenge/reader_studies/serializers.py
--- a/app/grandchallenge/reader_studies/serializers.py
+++ b/app/grandchallenge/reader_studies/serializers.py
@@ -86,9 +86,7 @@ def get_index(self, obj):
             try:
                 return self.context["view"].randomized_qs.index(obj)
             except ValueError:
-                # The list is empty in the detail view. Return None to
-                # indicate that the value is invalid for shuffled lists
-                # in this case.
+                # The list is empty if no reader study is specified.
                 return None
         else:
             return obj.standard_index


Test patch:
diff --git a/app/tests/reader_studies_tests/test_forms.py b/app/tests/reader_studies_tests/test_forms.py
--- a/app/tests/reader_studies_tests/test_forms.py
+++ b/app/tests/reader_studies_tests/test_forms.py
@@ -180,14 +180,6 @@ def try_create_rs(
     assert "error_1_id_workstation" not in response.rendered_content
     assert case_navigation_error in response.rendered_content
 
-    shuffled_case_navigation_error = (
-        "Shuffled hanging lists cannot be used together with case navigation."
-    )
-    response = try_create_rs(
-        allow_case_navigation=True, shuffle_hanging_list=True
-    )
-    assert shuffled_case_navigation_error in response.rendered_content
-
     response = try_create_rs(allow_case_navigation=True)
     assert "error_1_id_workstation" not in response.rendered_content
     assert case_navigation_error not in response.rendered_content