internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: liqd/a4-opin

Issue:
Password reset attempt for non-existant user causes 500 and sentry error on DEV, unstyled response on STAGE/PROD
**URL:** 
500 only on DEV: https://opin-dev.liqd.net/de/accounts/password/reset/
Sentry: https://sentry.liqd.net/organizations/liqd/issues/1979/?environment=production&project=11&referrer=alert_email

Stage throws no error, but is not styled right (see screenshot)
**user:** unregistered
**expected behaviour:** proper information
**behaviour:** I get a 500 error and a sentry error on DEV, and an iunnformative red bar on STAGE and PROD
**important screensize:** any
**device & browser:** any
**Comment/Question:** 
Same as https://github.com/liqd/a4-meinberlin/issues/4290

Screenshot?
On Stage:
![image](https://user-images.githubusercontent.com/17978375/159471434-fb521d13-614a-4ef5-8e40-d9251a9c6658.png)



Discussion:

Resolving patch:
diff --git a/euth_wagtail/settings/base.py b/euth_wagtail/settings/base.py
--- a/euth_wagtail/settings/base.py
+++ b/euth_wagtail/settings/base.py
@@ -381,6 +381,7 @@
 ACCOUNT_USER_DISPLAY = 'euth.users.services.account_user_display'
 ACCOUNT_USER_MODEL_USERNAME_FIELD = 'username'
 ACCOUNT_USERNAME_REQUIRED = True
+ACCOUNT_PREVENT_ENUMERATION = True
 ACCOUNT_LOGIN_ATTEMPTS_LIMIT = 10
 ACCOUNT_LOGIN_ATTEMPTS_TIMEOUT = 300  # seconds
 ACCOUNT_LOGIN_ON_EMAIL_CONFIRMATION = True


Test patch:
Repository: comic/grand-challenge.org

Issue:
Do not print page title above each page
Having a h1 HOME on your home page looks stupid. Either remove this completely and show currently selected page in menu, or put page title at top of content by default, so it is there by default  but can be edited away if needed 



Discussion:
I would go for highlighted page titles in the link bar at the top. Should be simple by css, no? On the other hand, related to issue 22, what will you do with hidden-link-in-title-bar pages?


Resolving patch:
diff --git a/django/comicsite/views.py b/django/comicsite/views.py
--- a/django/comicsite/views.py
+++ b/django/comicsite/views.py
@@ -27,7 +27,7 @@ def site(request, site_short_name):
                     
     pages = getPages(site_short_name)
     
-    return render_to_response('page.html', {'site': site, 'page': pages[0], "pages":pages },context_instance=RequestContext(request))
+    return render_to_response('page.html', {'site': site, 'currentpage': pages[0], "pages":pages },context_instance=RequestContext(request))
     
 
 def page(request, site_short_name, page_title):
@@ -39,7 +39,7 @@ def page(request, site_short_name, page_title):
         raise Http404
     pages = getPages(site_short_name)
     
-    return render_to_response('page.html', {'site': p.ComicSite, 'page': p, "pages":pages },context_instance=RequestContext(request))
+    return render_to_response('page.html', {'site': p.ComicSite, 'currentpage': p, "pages":pages },context_instance=RequestContext(request))
                 
     
     


Test patch:
Repository: comic/grand-challenge.org

Issue:
Country is not stored in db on signup
When a user signs up the country is not stored in the db



Discussion:

Resolving patch:
diff --git a/django/profiles/forms.py b/django/profiles/forms.py
--- a/django/profiles/forms.py
+++ b/django/profiles/forms.py
@@ -44,6 +44,7 @@ def save(self):
         user_profile = user.get_profile()
         user_profile.institution = self.cleaned_data['institution']
         user_profile.department = self.cleaned_data['department']
+        user_profile.country = self.cleaned_data['country']
         user_profile.save()
 
         return user


Test patch:
Repository: comic/grand-challenge.org

Issue:
Make error message emails less verbose
The entire stacktrace is included right now.


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/evaluation/emails.py b/app/grandchallenge/evaluation/emails.py
--- a/app/grandchallenge/evaluation/emails.py
+++ b/app/grandchallenge/evaluation/emails.py
@@ -4,6 +4,7 @@
 from django.core.mail import send_mail
 
 from grandchallenge.core.urlresolvers import reverse
+from grandchallenge.evaluation.templatetags.evaluation_extras import user_error
 
 
 def send_failed_job_email(job):
@@ -11,7 +12,7 @@ def send_failed_job_email(job):
         f'Unfortunately the evaluation for the submission to '
         f'{job.challenge.short_name} failed with an error. The error message '
         f'is:\n\n'
-        f'{job.output}\n\n'
+        f'{user_error(job.output)}\n\n'
         f'You may wish to try and correct this, or contact the challenge '
         f'organizers. The following information may help them:\n'
         f'User: {job.submission.creator.username}\n'
diff --git a/app/grandchallenge/evaluation/views.py b/app/grandchallenge/evaluation/views.py
--- a/app/grandchallenge/evaluation/views.py
+++ b/app/grandchallenge/evaluation/views.py
@@ -72,7 +72,7 @@ class SubmissionCreate(
     form_class = SubmissionForm
     success_message = (
         "Your submission was successful. "
-        "Please keep checking this page for your result."
+        "You will receive an email when your result is ready."
     )
 
     def get_form_kwargs(self):


Test patch:
Repository: comic/grand-challenge.org

Issue:
Add apple icons
There are many errors in the logs about missing apple icons, we should provide these for all challenges.

apple-touch-icon.png/
apple-touch-icon-precomposed.png/
site/luna16/apple-touch-icon.png/
site/luna16/apple-touch-icon-precomposed.png/
site/luna16/apple-touch-icon-120x120.png/
site/luna16/apple-touch-icon-120x120-precomposed.png/



Discussion:

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -234,6 +234,7 @@
     'rest_framework', # provides REST API
     'rest_framework.authtoken', # token auth for REST API
     'crispy_forms', # bootstrap forms
+    'favicon', # favicon management
 ]
 
 LOCAL_APPS = [
diff --git a/app/grandchallenge/challenges/permissions.py b/app/grandchallenge/challenges/permissions.py
--- a/app/grandchallenge/challenges/permissions.py
+++ b/app/grandchallenge/challenges/permissions.py
@@ -60,8 +60,13 @@ def _required_permission(path, challenge_short_name):
             " to your .conf file."
         )
 
-    if challenge_short_name.lower() in ['logos', 'banners', 'mugshots']:
-        # Anyone can download logos, banners and mugshots
+    if challenge_short_name.lower() in [
+        'logos',
+        'banners',
+        'mugshots',
+        'favicon',
+    ]:
+        # Anyone can download logos, banners, mugshots and favicons
         return ComicSiteModel.ALL
 
     if challenge_short_name.lower() == 'evaluation':


Test patch:
Repository: comic/grand-challenge.org

Issue:
CKEditor Image upload makes the GUI undismissable
The browser keeps the changed fields state after the image is uploaded, and for some reason we're unable to dismiss the gui.


Discussion:

Resolving patch:
diff --git a/app/config/urls.py b/app/config/urls.py
--- a/app/config/urls.py
+++ b/app/config/urls.py
@@ -73,8 +73,12 @@ def handler500(request):
         "challenges/",
         include("grandchallenge.challenges.urls", namespace="challenges"),
     ),
-    re_path(
-        r"^(?i)all_challenges/$",
+    path(
+        "all_challenges/",
+        RedirectView.as_view(pattern_name="challenges:list", permanent=False),
+    ),
+    path(
+        "All_Challenges/",
         RedirectView.as_view(pattern_name="challenges:list", permanent=False),
     ),
     path("cases/", include("grandchallenge.cases.urls", namespace="cases")),


Test patch:
Repository: comic/grand-challenge.org

Issue:
No response when uploading a new algorithm using the wrong file format
# Recipe

1. Go to https://grand-challenge.org/algorithms/create/
2. Upload, for example, a `.tar.gz` file

# Result

Upload completes, nothing happens.


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/algorithms/forms.py b/app/grandchallenge/algorithms/forms.py
--- a/app/grandchallenge/algorithms/forms.py
+++ b/app/grandchallenge/algorithms/forms.py
@@ -29,7 +29,7 @@ class AlgorithmForm(forms.ModelForm):
         validators=[ExtensionValidator(allowed_extensions=(".tar",))],
         help_text=(
             "Tar archive of the container image produced from the command "
-            "`docker save IMAGE > IMAGE.tar`. See "
+            "'docker save IMAGE > IMAGE.tar'. See "
             "https://docs.docker.com/engine/reference/commandline/save/"
         ),
     )
diff --git a/app/grandchallenge/evaluation/forms.py b/app/grandchallenge/evaluation/forms.py
--- a/app/grandchallenge/evaluation/forms.py
+++ b/app/grandchallenge/evaluation/forms.py
@@ -89,7 +89,7 @@ class MethodForm(forms.ModelForm):
         validators=[ExtensionValidator(allowed_extensions=(".tar",))],
         help_text=(
             "Tar archive of the container image produced from the command "
-            "`docker save IMAGE > IMAGE.tar`. See "
+            "'docker save IMAGE > IMAGE.tar'. See "
             "https://docs.docker.com/engine/reference/commandline/save/"
         ),
     )


Test patch:
Repository: comic/grand-challenge.org

Issue:
Docker fails to read results
The results of evaluations cannot be read with the latest version of docker


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/container_exec/backends/docker.py b/app/grandchallenge/container_exec/backends/docker.py
--- a/app/grandchallenge/container_exec/backends/docker.py
+++ b/app/grandchallenge/container_exec/backends/docker.py
@@ -169,22 +169,30 @@ def _execute_container(self):
             raise RuntimeError(exc.stderr.decode())
 
     def _get_result(self) -> dict:
+        """
+        Read and parse the created results file. Due to a bug in the docker
+        client, copy the file to memory first rather than cat and read
+        stdout.
+        """
         try:
-            result = self._client.containers.run(
-                image=self._io_image,
-                volumes={
-                    self._output_volume: {"bind": "/output/", "mode": "ro"}
-                },
-                command=f"cat {self._results_file}",
-                remove=True,
-                **self._run_kwargs,
-            )
-        except ContainerError as exc:
-            raise RuntimeError(exc.stderr.decode())
+            with cleanup(
+                self._client.containers.run(
+                    image=self._io_image,
+                    volumes={
+                        self._output_volume: {"bind": "/output/", "mode": "ro"}
+                    },
+                    detach=True,
+                    tty=True,
+                    **self._run_kwargs,
+                )
+            ) as reader:
+                result = get_file(container=reader, src=self._results_file)
+        except Exception as e:
+            raise RuntimeError(str(e))
 
         try:
             result = json.loads(
-                result.decode(),
+                result.read().decode(),
                 parse_constant=lambda x: None,  # Removes -inf, inf and NaN
             )
         except JSONDecodeError as exc:


Test patch:
diff --git a/app/tests/evaluation_tests/resources/docker/evaluate_submission.py b/app/tests/evaluation_tests/resources/docker/evaluate_submission.py
--- a/app/tests/evaluation_tests/resources/docker/evaluate_submission.py
+++ b/app/tests/evaluation_tests/resources/docker/evaluate_submission.py
@@ -13,7 +13,7 @@ def get_classes(csvfile: str):
 
 def write_metrics(metrics: dict):
     with open("/output/metrics.json", "w") as f:
-        f.write(json.dumps(metrics, ensure_ascii=True))
+        f.write(json.dumps(metrics, ensure_ascii=True, indent=2))
 
 
 if __name__ == "__main__":
Repository: comic/grand-challenge.org

Issue:
Unpin redis
Celery does not support the redis v3 client so this is pinned for now. Unpin it when it is supported by Celery See: celery/celery#5178


Discussion:

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -506,7 +506,7 @@ def strtobool(val) -> bool:
 CONTAINER_EXEC_MEMORY_LIMIT = "4g"
 CONTAINER_EXEC_IO_IMAGE = "alpine:3.8"
 CONTAINER_EXEC_IO_SHA256 = (
-    "sha256:196d12cf6ab19273823e700516e98eb1910b03b17840f9d5509f03858484d321"
+    "sha256:3f53bb00af943dfdf815650be70c0fa7b426e56a66f5e3362b47a129d57d5991"
 )
 CONTAINER_EXEC_CPU_QUOTA = 100000
 CONTAINER_EXEC_CPU_PERIOD = 100000


Test patch:
Repository: comic/grand-challenge.org

Issue:
Resize the logos
The logos on the all challenges page take a while to download, and are mostly served at full resolution. Make these the appropriate size (both native a 2x) but keep the originals for regeneration in the future.


Discussion:
Try: https://github.com/jazzband/sorl-thumbnail

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -263,6 +263,7 @@ def strtobool(val) -> bool:
     "favicon",  # favicon management
     "django_select2",  # for multiple choice widgets
     "django_summernote",  # for WYSIWYG page editing
+    "sorl.thumbnail",  # for dynamic thumbnails
 ]
 
 LOCAL_APPS = [
@@ -567,6 +568,7 @@ def strtobool(val) -> bool:
     "evaluation-supplementary",
     "favicon",
     "i",
+    "cache",  # for sorl-thumbnails
     JQFILEUPLOAD_UPLOAD_SUBIDRECTORY,
     *USERNAME_DENYLIST,
 ]
diff --git a/app/grandchallenge/challenges/models.py b/app/grandchallenge/challenges/models.py
--- a/app/grandchallenge/challenges/models.py
+++ b/app/grandchallenge/challenges/models.py
@@ -256,14 +256,11 @@ def __str__(self):
         return self.short_name
 
     @property
-    def thumb_image_url(self):
-        try:
-            return self.logo.url
-        except ValueError:
-            return (
-                f"https://www.gravatar.com/avatar/"
-                f"{hashlib.md5(self.creator.email.lower().encode()).hexdigest()}"
-            )
+    def gravatar_url(self):
+        return (
+            f"https://www.gravatar.com/avatar/"
+            f"{hashlib.md5(self.creator.email.lower().encode()).hexdigest()}"
+        )
 
     def get_absolute_url(self):
         raise NotImplementedError
diff --git a/app/grandchallenge/serving/urls.py b/app/grandchallenge/serving/urls.py
--- a/app/grandchallenge/serving/urls.py
+++ b/app/grandchallenge/serving/urls.py
@@ -11,6 +11,7 @@
     path("mugshots/<path:path>", serve_folder, {"folder": "mugshots"}),
     path("favicon/<path:path>", serve_folder, {"folder": "favicon"}),
     path("i/<path:path>", serve_folder, {"folder": "i"}),
+    path("cache/<path:path>", serve_folder, {"folder": "cache"}),
     path(
         "evaluation-supplementary/<path:path>",
         serve_folder,


Test patch:
Repository: comic/grand-challenge.org

Issue:
Serve static files from a CDN
We have a lot of users in China and America, some of whom complain about the speed of the site. We would gain quite a bit by serving the static assets from a CDN. Integrate Whitenoise http://whitenoise.evans.io/en/stable/ and CloudFront with static file restrictions http://whitenoise.evans.io/en/stable/django.html#restricting-cloudfront


Discussion:

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -193,20 +193,17 @@ def strtobool(val) -> bool:
 # Example: "/home/media/media.lawrence.com/static/"
 STATIC_ROOT = "/static/"
 
-# Serve files using django (debug only)
-STATIC_URL = "/static/"
+STATIC_HOST = os.environ.get("DJANGO_STATIC_HOST", "")
+STATIC_URL = f"{STATIC_HOST}/static/"
 
 # List of finder classes that know how to find static files in
 # various locations.
 STATICFILES_FINDERS = (
     "django.contrib.staticfiles.finders.FileSystemFinder",
     "django.contrib.staticfiles.finders.AppDirectoriesFinder",
-    #    'django.contrib.staticfiles.finders.DefaultStorageFinder',
 )
 
-STATICFILES_STORAGE = (
-    "django.contrib.staticfiles.storage.ManifestStaticFilesStorage"
-)
+STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
 
 # Make this unique, and don't share it with anybody.
 SECRET_KEY = os.environ.get(
@@ -236,10 +233,11 @@ def strtobool(val) -> bool:
 ]
 
 MIDDLEWARE = (
+    "django.middleware.security.SecurityMiddleware",  # Keep security at top
+    "whitenoise.middleware.WhiteNoiseMiddleware",  # Keep whitenoise after security and before all else
     "django.middleware.common.BrokenLinkEmailsMiddleware",
     # Keep BrokenLinkEmailsMiddleware near the top
     "raven.contrib.django.raven_compat.middleware.SentryResponseErrorIdMiddleware",
-    "django.middleware.security.SecurityMiddleware",
     "django.contrib.sessions.middleware.SessionMiddleware",
     "django.middleware.common.CommonMiddleware",
     "django.middleware.csrf.CsrfViewMiddleware",
@@ -261,6 +259,7 @@ def strtobool(val) -> bool:
     "django.contrib.sessions",
     "django.contrib.sites",
     "django.contrib.messages",
+    "whitenoise.runserver_nostatic",  # Keep whitenoise above staticfiles
     "django.contrib.staticfiles",
     "django.contrib.humanize",
     "django.contrib.admin",


Test patch:
Repository: comic/grand-challenge.org

Issue:
Make the container runtime configurable
The celery workers should be able to set the container runtime that they will use, this should be configurable by environment variables.


Discussion:

Resolving patch:
diff --git a/app/config/settings.py b/app/config/settings.py
--- a/app/config/settings.py
+++ b/app/config/settings.py
@@ -513,8 +513,10 @@ def strtobool(val) -> bool:
 CELERY_BROKER_URL = os.environ.get("CELERY_BROKER_URL", "redis://redis:6379/0")
 CELERY_RESULT_BACKEND = os.environ.get("CELERY_RESULT_BACKEND", "django-db")
 CELERY_RESULT_PERSISTENT = True
-CELERY_TASK_SOFT_TIME_LIMIT = 7200
-CELERY_TASK_TIME_LIMIT = 7260
+CELERY_TASK_SOFT_TIME_LIMIT = int(
+    os.environ.get("CELERY_TASK_SOFT_TIME_LIMIT", "7200")
+)
+CELERY_TASK_TIME_LIMIT = int(os.environ.get("CELERY_TASK_TIME_LIMIT", "7260"))
 
 CONTAINER_EXEC_DOCKER_BASE_URL = os.environ.get(
     "CONTAINER_EXEC_DOCKER_BASE_URL", "unix://var/run/docker.sock"
@@ -531,13 +533,22 @@ def strtobool(val) -> bool:
 CONTAINER_EXEC_DOCKER_TLSKEY = os.environ.get(
     "CONTAINER_EXEC_DOCKER_TLSKEY", ""
 )
-CONTAINER_EXEC_MEMORY_LIMIT = "4g"
+CONTAINER_EXEC_MEMORY_LIMIT = os.environ.get(
+    "CONTAINER_EXEC_MEMORY_LIMIT", "4g"
+)
 CONTAINER_EXEC_IO_IMAGE = "alpine:3.9"
 CONTAINER_EXEC_IO_SHA256 = (
     "sha256:caf27325b298a6730837023a8a342699c8b7b388b8d878966b064a1320043019"
 )
-CONTAINER_EXEC_CPU_QUOTA = 100000
-CONTAINER_EXEC_CPU_PERIOD = 100000
+CONTAINER_EXEC_CPU_QUOTA = int(
+    os.environ.get("CONTAINER_EXEC_CPU_QUOTA", "100000")
+)
+CONTAINER_EXEC_CPU_PERIOD = int(
+    os.environ.get("CONTAINER_EXEC_CPU_PERIOD", "100000")
+)
+CONTAINER_EXEC_DOCKER_RUNTIME = os.environ.get(
+    "CONTAINER_EXEC_DOCKER_RUNTIME", None
+)
 
 CELERY_BEAT_SCHEDULE = {
     "cleanup_stale_uploads": {
diff --git a/app/grandchallenge/container_exec/backends/docker.py b/app/grandchallenge/container_exec/backends/docker.py
--- a/app/grandchallenge/container_exec/backends/docker.py
+++ b/app/grandchallenge/container_exec/backends/docker.py
@@ -16,6 +16,7 @@
 from docker.api.container import ContainerApiMixin
 from docker.errors import ContainerError, APIError
 from docker.tls import TLSConfig
+from docker.types import LogConfig
 from requests import HTTPError
 
 
@@ -61,6 +62,10 @@ def __init__(
             "mem_limit": settings.CONTAINER_EXEC_MEMORY_LIMIT,
             "cpu_period": settings.CONTAINER_EXEC_CPU_PERIOD,
             "cpu_quota": settings.CONTAINER_EXEC_CPU_QUOTA,
+            "runtime": settings.CONTAINER_EXEC_DOCKER_RUNTIME,
+            "log_config": LogConfig(
+                type=LogConfig.types.JSON, config={"max-size": "1g"}
+            ),
         }
 
     def __enter__(self):


Test patch:
Repository: comic/grand-challenge.org

Issue:
Catch ConnectionError for container cleanup
A connection error can sometimes occur on this line:

https://github.com/comic/grand-challenge.org/blob/9b7230e63040ad08654cb6280594f19eeb033372/app/grandchallenge/container_exec/backends/docker.py#L85

Catch it and return a sensible error message, or schedule the task to be retried later.


Discussion:

Resolving patch:
diff --git a/app/grandchallenge/container_exec/backends/docker.py b/app/grandchallenge/container_exec/backends/docker.py
--- a/app/grandchallenge/container_exec/backends/docker.py
+++ b/app/grandchallenge/container_exec/backends/docker.py
@@ -14,7 +14,7 @@
 from django.conf import settings
 from django.core.files import File
 from docker.api.container import ContainerApiMixin
-from docker.errors import ContainerError, APIError
+from docker.errors import ContainerError, APIError, NotFound
 from docker.tls import TLSConfig
 from docker.types import LogConfig
 from requests import HTTPError
@@ -82,11 +82,18 @@ def __enter__(self):
     def __exit__(self, exc_type, exc_val, exc_tb):
         flt = {"label": f"job_id={self._job_id}"}
 
-        for container in self._client.containers.list(filters=flt):
-            container.stop()
+        try:
+            for c in self._client.containers.list(filters=flt):
+                c.stop()
 
-        self.__retry_docker_obj_prune(obj=self._client.containers, filters=flt)
-        self.__retry_docker_obj_prune(obj=self._client.volumes, filters=flt)
+            self.__retry_docker_obj_prune(
+                obj=self._client.containers, filters=flt
+            )
+            self.__retry_docker_obj_prune(
+                obj=self._client.volumes, filters=flt
+            )
+        except ConnectionError:
+            raise RuntimeError("Could not connect to worker.")
 
     @staticmethod
     def __retry_docker_obj_prune(*, obj, filters: dict):
@@ -205,6 +212,15 @@ def _get_result(self) -> dict:
                 )
             ) as reader:
                 result = get_file(container=reader, src=self._results_file)
+        except NotFound:
+            # The container exited without error, but no results file was
+            # produced. This shouldn't happen, but does with poorly programmed
+            # evaluation containers.
+            raise RuntimeError(
+                "The evaluation failed for an unknown reason as no results "
+                "file was produced. Please contact the organisers for "
+                "assistance."
+            )
         except Exception as e:
             raise RuntimeError(str(e))
 


Test patch: