internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: comic/grand-challenge.org

Issue:
Count of results displayed on the challenge card does not match leaderboard count
**Describe the bug**
The card for the node21 challenge currently notes there are 21 results.  Clicking on this brings you to the leaderboard where only 2 results are present.   It seems that the count is including submissions which failed and/or submissions where the evaluation failed, which is misleading.  

**To Reproduce**
Steps to reproduce the behavior:
 - Choose a challenge where the database includes many failed submissions or failed evaluations (e.g. node21 at present)
- View the card for this challenge (currently it is on the GC front page)
 - Verify that the number of results shown on the card does not match the number of results on the leaderboard (click the number shown on the card).

**Expected behavior**
The number of reported results should match the number of results on the leaderboard

**Screenshots**
![image](https://user-images.githubusercontent.com/6442576/139235716-012dc618-4fba-46e5-b510-2fbc81ae61b7.png)




Discussion:
These are the number of submissions, not the number of results on the leaderboard. If we changed this, we would still have the same problem for challenges with multiple leaderboards.

Resolving patch:
diff --git a/app/grandchallenge/challenges/tasks.py b/app/grandchallenge/challenges/tasks.py
--- a/app/grandchallenge/challenges/tasks.py
+++ b/app/grandchallenge/challenges/tasks.py
@@ -13,7 +13,7 @@
 def update_challenge_results_cache():
     challenges = Challenge.objects.all()
     evaluation_info = (
-        Evaluation.objects.filter(published=True)
+        Evaluation.objects.filter(published=True, rank__gt=0)
         .values("submission__phase__challenge_id")
         .annotate(
             cached_num_results=Count("submission__phase__challenge_id"),


Test patch:
Repository: comic/grand-challenge.org

Issue:
Subscription count does not correspond with displayed subscriptions
**Problem**
We do not show all subscriptions to a user - only those that we allow them to unsubscribe from. Subscriptions that are not shown to the user include permission request follows and upload-session follows, as well as "inactive" algorithm job follows (a necessary work around to allow unsubscribing from failed job notifications). The paginator on the listview, however, counts all subscriptions of the user, which leads to empty pages and a discrepancy in the shown vs. indicated number of subscriptions. 

**Solution**
We need to make sure that the paginator counts only the subscriptions that are actually shown to the user. 
Simply showing all subscriptions sounds like a good alternative, but is not possible, because the inactive algorithm job follows should not be exposed to the user. 


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/notifications/views.py b/app/grandchallenge/notifications/views.py
--- a/app/grandchallenge/notifications/views.py
+++ b/app/grandchallenge/notifications/views.py
@@ -1,6 +1,7 @@
 from actstream.models import Follow
 from django.contrib import messages
 from django.contrib.messages.views import SuccessMessageMixin
+from django.db.models import Q
 from django.views.generic import CreateView, DeleteView, ListView
 from guardian.mixins import (
     LoginRequiredMixin,
@@ -119,14 +120,33 @@ class FollowList(
     permission_required = "view_follow"
     filter_class = FollowFilter
     paginate_by = 50
-
-    def get_queryset(self, *args, **kwargs):
-        return (
-            super()
-            .get_queryset()
-            .select_related("user", "content_type")
-            .order_by("content_type")
+    queryset = (
+        Follow.objects.exclude(
+            Q(
+                content_type__app_label="archives",
+                content_type__model="archivepermissionrequest",
+            )
+            | Q(
+                content_type__app_label="algorithms",
+                content_type__model="algorithmpermissionrequest",
+            )
+            | Q(
+                content_type__app_label="reader_studies",
+                content_type__model="readerstudypermissionrequest",
+            )
+            | Q(
+                content_type__app_label="participants",
+                content_type__model="registrationrequest",
+            )
+            | Q(
+                content_type__app_label="cases",
+                content_type__model="rawimageuploadsessions",
+            )
         )
+        .exclude(flag="job-inactive")
+        .select_related("user", "content_type")
+        .order_by("content_type")
+    )
 
     def get_context_data(self, **kwargs):
         context = super().get_context_data(**kwargs)
@@ -136,6 +156,7 @@ def get_context_data(self, **kwargs):
         context["object_list"] = prefetch_generic_foreign_key_objects(
             context["object_list"]
         )
+
         return context
 
 


Test patch:
Repository: comic/grand-challenge.org

Issue:
'NoneType' object has no attribute 'values'
Sentry issue https://sentry.io/organizations/grand-challenge/issues/3127690895/?project=303639&query=is%3Aunresolved

```
slugs = {slug for viewport in mapping.values() for slug in viewport}
```

Added in https://github.com/comic/grand-challenge.org/pull/2322


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/hanging_protocols/forms.py b/app/grandchallenge/hanging_protocols/forms.py
--- a/app/grandchallenge/hanging_protocols/forms.py
+++ b/app/grandchallenge/hanging_protocols/forms.py
@@ -28,7 +28,7 @@ class Meta:
 
 class ViewContentMixin:
     def clean_view_content(self):
-        mapping = self.cleaned_data["view_content"]
+        mapping = self.cleaned_data["view_content"] or {}
         hanging_protocol = self.cleaned_data["hanging_protocol"]
         if mapping and not hanging_protocol:
             self.add_error(


Test patch:
Repository: comic/grand-challenge.org

Issue:
Cannot resolve keyword 'name' into field.
See https://sentry.io/organizations/grand-challenge/issues/3186162930/?alert_rule_id=389098&alert_timestamp=1649749100615&alert_type=email&environment=production&project=303639&referrer=alert_email


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/reader_studies/views.py b/app/grandchallenge/reader_studies/views.py
--- a/app/grandchallenge/reader_studies/views.py
+++ b/app/grandchallenge/reader_studies/views.py
@@ -340,7 +340,7 @@ class ReaderStudyDisplaySetList(
     raise_exception = True
     template_name = "reader_studies/readerstudy_images_list.html"
     row_template = "reader_studies/readerstudy_display_sets_row.html"
-    search_fields = ["pk", "name"]
+    search_fields = ["pk", "values__image__name", "values__file"]
     columns = [
         Column(title="Name", sort_field="order"),
     ]


Test patch:
Repository: comic/grand-challenge.org

Issue:
Add a time column to challenge requests table
**Is your feature request related to a problem? Please describe.**
Its not clear right away when each challenge request was submitted.


**Describe the solution you'd like**
I would like to add a column that shows the time when challenge request was submitted. It would be handy. 

**Describe alternatives you've considered**
Looking at a specific challenge request page to see the time.

**Additional context**
Idea shown on the picture below. Color coding is not necessary, but could be nice.
![requesttime](https://user-images.githubusercontent.com/37930396/164007865-b92bb7c9-dc2e-461d-a9d9-857779ed932f.png)


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/challenges/admin.py b/app/grandchallenge/challenges/admin.py
--- a/app/grandchallenge/challenges/admin.py
+++ b/app/grandchallenge/challenges/admin.py
@@ -27,8 +27,22 @@ class ExternalChallengeAdmin(ModelAdmin):
 class ChallengeRequestAdmin(ModelAdmin):
     readonly_fields = ("creator",)
     ordering = ("-created",)
-    list_display = ("title", "short_name", "creator", "created", "status")
+    list_display = (
+        "title",
+        "short_name",
+        "creator",
+        "created",
+        "status",
+        "challenge_type",
+        "total_cost",
+        "budget_for_hosting_challenge",
+    )
     actions = ["create_challenge", "send_status_update_email"]
+    list_filter = ["status", "challenge_type"]
+
+    @admin.display(description="Total cost")
+    def total_cost(self, obj):
+        return "%s" % (obj.budget["Total"])
 
     @admin.action(description="Create challenge for this request")
     def create_challenge(self, request, queryset):
diff --git a/app/grandchallenge/core/templatetags/naturaldelta.py b/app/grandchallenge/core/templatetags/naturaldelta.py
--- a/app/grandchallenge/core/templatetags/naturaldelta.py
+++ b/app/grandchallenge/core/templatetags/naturaldelta.py
@@ -1,5 +1,6 @@
 import humanize
 from django import template
+from django.utils import timezone
 
 register = template.Library()
 
@@ -7,3 +8,8 @@
 @register.filter
 def naturaldelta(value):
     return humanize.naturaldelta(value, months=False)
+
+
+@register.filter
+def timedifference(value):
+    return (timezone.now() - value).days


Test patch:
Repository: comic/grand-challenge.org

Issue:
UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list
Found when running `tests/emails_tests/test_views.py::test_email_list_permission`


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/emails/migrations/0002_alter_email_options.py b/app/grandchallenge/emails/migrations/0002_alter_email_options.py
new file mode 100644
--- /dev/null
+++ b/app/grandchallenge/emails/migrations/0002_alter_email_options.py
@@ -0,0 +1,17 @@
+# Generated by Django 3.2.13 on 2022-06-02 11:23
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("emails", "0001_initial"),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name="email",
+            options={"ordering": ["pk"]},
+        ),
+    ]
diff --git a/app/grandchallenge/emails/models.py b/app/grandchallenge/emails/models.py
--- a/app/grandchallenge/emails/models.py
+++ b/app/grandchallenge/emails/models.py
@@ -17,6 +17,9 @@ class Email(models.Model):
         help_text="This stores the page number of the last successfully sent email batch for this email.",
     )
 
+    class Meta:
+        ordering = ["pk"]
+
     def __str__(self):
         return self.subject
 


Test patch:
Repository: comic/grand-challenge.org

Issue:
Update reader study question gives error
**Describe the bug**
It is not possible to update a reader study question.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to any reader study that has answers
2. Click on questions
3. Click on Edit for a question
4. Update the question (for example Order)
5. Click Save
GC returns an error: "Something went wrong"
Sentry event:
https://sentry.io/organizations/grand-challenge/issues/3344966516/?project=303639&query=is%3Aunresolved&statsPeriod=24h

**Expected behavior**
Question should be saved





Discussion:

Resolving patch:
diff --git a/app/grandchallenge/reader_studies/forms.py b/app/grandchallenge/reader_studies/forms.py
--- a/app/grandchallenge/reader_studies/forms.py
+++ b/app/grandchallenge/reader_studies/forms.py
@@ -283,6 +283,8 @@ def __init__(self, *args, **kwargs):
 
     def interface_choices(self):
         answer_type = self["answer_type"].value()
+        if answer_type is None:
+            return ComponentInterface.objects.none()
         return ComponentInterface.objects.filter(
             kind__in=ANSWER_TYPE_TO_INTERFACE_KIND_MAP[answer_type]
         )


Test patch:
diff --git a/app/tests/reader_studies_tests/test_forms.py b/app/tests/reader_studies_tests/test_forms.py
--- a/app/tests/reader_studies_tests/test_forms.py
+++ b/app/tests/reader_studies_tests/test_forms.py
@@ -394,6 +394,16 @@ def test_question_form_interface_field(answer_type, interface_kind):
     assert not form.interface_choices().filter(pk=ci_img.pk).exists()
 
 
+@pytest.mark.django_db
+def test_question_form_interface_field_no_answer_type():
+    assert ComponentInterface.objects.count() > 0
+    form = QuestionForm(initial={"answer_type": None})
+    # No answer_type provided, this happens for answers that already have
+    # answers. The form shouldn't error and the interface_choices should
+    # be empty.
+    assert form.interface_choices().count() == 0
+
+
 @pytest.mark.django_db
 @pytest.mark.parametrize(
     "answer_type,port,questions_created",
Repository: comic/grand-challenge.org

Issue:
Display set does not expand
**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to https://grand-challenge.org/reader-studies/demo-educational-reader-study/
2. Click on 'Cases', 'View/Delete Cases'
3. Try and expand the display sets
4. Console errors:
Response Status Error Code 500 from /reader-studies/display-sets/REDACTED/ 

Sentry issue:
https://sentry.io/organizations/grand-challenge/issues/3345009229/?project=303639&query=is%3Aunresolved&statsPeriod=24h



**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**

https://user-images.githubusercontent.com/51329283/173305909-d1a5b0f0-06d6-43be-86c3-47cf2917909b.mp4



Discussion:

Resolving patch:
diff --git a/app/grandchallenge/reader_studies/models.py b/app/grandchallenge/reader_studies/models.py
--- a/app/grandchallenge/reader_studies/models.py
+++ b/app/grandchallenge/reader_studies/models.py
@@ -849,7 +849,7 @@ def description(self):
                 [
                     md2html(case_text[val.image.name])
                     for val in self.values.all()
-                    if val.image.name in case_text
+                    if val.image and val.image.name in case_text
                 ]
             )
         else:


Test patch:
Repository: comic/grand-challenge.org

Issue:
`unanswered_by_user` and `user` options are missing from the Display Set documentation
**Describe the bug**
The swagger documentation should show all the filters available, they are missing from https://grand-challenge.org/api/#/reader-studies/reader_studies_display_sets_list

**Screenshots**
![image](https://user-images.githubusercontent.com/12661555/171641441-bd564f31-b16a-4778-9546-2f83c157f295.png)



Discussion:

Resolving patch:
diff --git a/app/grandchallenge/reader_studies/views.py b/app/grandchallenge/reader_studies/views.py
--- a/app/grandchallenge/reader_studies/views.py
+++ b/app/grandchallenge/reader_studies/views.py
@@ -38,6 +38,12 @@
     View,
 )
 from django_filters.rest_framework import DjangoFilterBackend
+from drf_spectacular.types import OpenApiTypes
+from drf_spectacular.utils import (
+    OpenApiParameter,
+    extend_schema,
+    extend_schema_view,
+)
 from guardian.mixins import LoginRequiredMixin, PermissionListMixin
 from guardian.mixins import (
     PermissionRequiredMixin as ObjectPermissionRequiredMixin,
@@ -985,6 +991,16 @@ def ground_truth(self, request, pk=None, case_pk=None):
         )
 
 
+@extend_schema_view(
+    list=extend_schema(
+        parameters=[
+            OpenApiParameter(
+                "unanswered_by_user", OpenApiTypes.BOOL, OpenApiParameter.QUERY
+            ),
+            OpenApiParameter("user", OpenApiTypes.STR, OpenApiParameter.QUERY),
+        ],
+    ),
+)
 class DisplaySetViewSet(
     mixins.CreateModelMixin,
     mixins.RetrieveModelMixin,


Test patch:
diff --git a/app/tests/reader_studies_tests/test_api.py b/app/tests/reader_studies_tests/test_api.py
--- a/app/tests/reader_studies_tests/test_api.py
+++ b/app/tests/reader_studies_tests/test_api.py
@@ -2,6 +2,7 @@
 from pathlib import Path
 
 import pytest
+from django.urls import reverse
 from django_capture_on_commit_callbacks import capture_on_commit_callbacks
 
 from grandchallenge.cases.models import RawImageUploadSession
@@ -1145,6 +1146,22 @@ def test_question_accepts_image_type_answers(client, settings):
     )
 
 
+@pytest.mark.django_db
+def test_display_set_extended_schema(client):
+    """Ensure that the added params are still included if we ever upgrade drf_spectacular."""
+    response = get_view_for_user(
+        viewname="api:schema",
+        client=client,
+        data={"format": "json"},
+    )
+    params = response.json()["paths"][
+        reverse("api:reader-studies-display-set-list")
+    ]["get"]["parameters"]
+    param_names = [param["name"] for param in params]
+    assert "unanswered_by_user" in param_names
+    assert "user" in param_names
+
+
 @pytest.mark.django_db
 def test_display_set_list_filters(client, settings):
     settings.task_eager_propagates = (True,)
Repository: comic/grand-challenge.org

Issue:
Markdown Editor jumps around too much
When editing in Markdown the Grand Challenge editor jumps around a lot, so the writer loses their place.

https://user-images.githubusercontent.com/12661555/173570208-c2567b82-bb78-441c-9286-2f70a8f66745.mp4

It would be better if the size of the text area was kept constant, and the content only resized when switching to the preview tab, like on GitHub:

https://user-images.githubusercontent.com/12661555/173570346-97f51fa9-fc23-49e8-b587-f656ff3cb7db.mp4





Discussion:

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -716,6 +716,7 @@
 )
 MARKDOWNX_MARKDOWN_EXTENSION_CONFIGS = {}
 MARKDOWNX_IMAGE_MAX_SIZE = {"size": (2000, 0), "quality": 90}
+MARKDOWNX_EDITOR_RESIZABLE = "False"
 
 HAYSTACK_CONNECTIONS = {
     "default": {"ENGINE": "haystack.backends.simple_backend.SimpleEngine"}


Test patch:
Repository: comic/grand-challenge.org

Issue:
Flatpages and Redirects broken when using ASGI
Examples:

- https://grand-challenge.org/about/ (flatpage, exists)
- https://grand-challenge.org/about (should redirect to https://grand-challenge.org/about/)
- https://parse2022.grand-challenge.org/Participation (should redirect to https://parse2022.grand-challenge.org/Participation/)
- https://parse2022.grand-challenge.org/gfsdfgdfdsg (should redirect to https://parse2022.grand-challenge.org/gfsdfgdfdsg/, which should 404).

Error occurs in the clickjacking middleware:

https://sentry.io/organizations/grand-challenge/issues/3374906811/?project=303639&query=is%3Aignored


Discussion:
Switching from the uvicorn to the sync worker in production fixed this.
Downgrading uvicorn didn't make a difference, the points to the problem being in the middleware. Anne is trying to isolate that now.
Must have occurred in https://github.com/comic/grand-challenge.org/compare/d326e7a...f5b2437

Resolving patch:
diff --git a/app/grandchallenge/core/middleware.py b/app/grandchallenge/core/middleware.py
--- a/app/grandchallenge/core/middleware.py
+++ b/app/grandchallenge/core/middleware.py
@@ -17,9 +17,6 @@ class TwoFactorMiddleware(MiddlewareMixin):
     half logged in by mistake.
     """
 
-    def __init__(self, get_response):
-        self.get_response = get_response
-
     def process_request(self, request):
         try:
             match = get_resolver(request.urlconf).resolve(request.path)
@@ -33,4 +30,4 @@ def process_request(self, request):
                 except KeyError:
                     pass
         except Resolver404:
-            return self.get_response(request)
+            pass


Test patch:
Repository: comic/grand-challenge.org

Issue:
Handle capacity errors with SageMaker
Sometimes SageMaker is out of capacity, these jobs need to be retried.

```js
FailureReason: "CapacityError: Unable to provision requested ML compute capacity. Please retry using a different ML instance type."
```


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/components/backends/amazon_sagemaker_batch.py b/app/grandchallenge/components/backends/amazon_sagemaker_batch.py
--- a/app/grandchallenge/components/backends/amazon_sagemaker_batch.py
+++ b/app/grandchallenge/components/backends/amazon_sagemaker_batch.py
@@ -561,10 +561,14 @@ def _get_job_data_log(self):
         return [event["message"] for event in response["events"]]
 
     def _set_runtime_metrics(self, *, event):
-        query_id = "q"
+        try:
+            start_time = ms_timestamp_to_datetime(event["TransformStartTime"])
+            end_time = ms_timestamp_to_datetime(event["TransformEndTime"])
+        except KeyError:
+            logger.warning("Invalid start or end time, metrics undetermined")
+            return
 
-        start_time = ms_timestamp_to_datetime(event["TransformStartTime"])
-        end_time = ms_timestamp_to_datetime(event["TransformEndTime"])
+        query_id = "q"
         query = f"SEARCH('{{{self._log_group_name},Host}} Host={self._transform_job_name}/i-', 'Average', 60)"
 
         instance_type = get(
@@ -663,7 +667,7 @@ def _handle_failed_job(self, *, event):
             raise ComponentException("Time limit exceeded")
 
         # Anything else needs investigation by a site administrator
-        raise RuntimeError("Job failed for an unknown reason")
+        raise RuntimeError(failure_reason)
 
     def _stop_running_jobs(self):
         try:


Test patch:
Repository: comic/grand-challenge.org

Issue:
`add-repo` view only displays limited number of repos
The dropdown for selecting a repo to link to an algorithm from GitHub only displays at most 30 results. If a user has access to more than 30 repos, the repo that they want to link might not be there.


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/algorithms/views.py b/app/grandchallenge/algorithms/views.py
--- a/app/grandchallenge/algorithms/views.py
+++ b/app/grandchallenge/algorithms/views.py
@@ -954,10 +954,12 @@ def get_form_kwargs(self):
             "Accept": "application/vnd.github.v3+json",
             "Authorization": f"token {user_token.access_token}",
         }
+        params = {"per_page": 100, "page": 1}
 
         installations = requests.get(
             "https://api.github.com/user/installations",
             headers=headers,
+            params=params,
             timeout=5,
         ).json()
         repos = []
@@ -965,6 +967,7 @@ def get_form_kwargs(self):
             response = requests.get(
                 f"https://api.github.com/user/installations/{installation['id']}/repositories",
                 headers=headers,
+                params=params,
                 timeout=5,
             ).json()
 


Test patch: