Repository: comic/grand-challenge.org

Issue:
Users cannot create archive items without images in the html templates
**Describe the bug**
The only way for a user to create a new archive item through the html templates is to upload an image. Some algorithms work without images, so a user needs a way to upload non-image files directly to new archive items. I think that they can use the API client to work around this, but unsure, @MikeOverkamp-diag could you confirm ASAP?

**Expected behavior**
A user uploads non-image interfaces and archive items are created for them.

**Additional context**
Related to #2506 



Discussion:
Yes, they should be able to use `client.archive_items.create` to create an
item and then add the values using `client.update_archive_item`.

On Fri, Jun 10, 2022, 12:25 James ***@***.***> wrote:

> Assigned #2508 <https://github.com/comic/grand-challenge.org/issues/2508>
> to @MikeOverkamp-diag <https://github.com/MikeOverkamp-diag>.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/comic/grand-challenge.org/issues/2508#event-6784838008>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ANU2ZKQFIDH2FDXLHIIEPODVOMJ2HANCNFSM5YNGO4QA>
> .
> You are receiving this because you were assigned.Message ID:
> ***@***.***>
>

Great - can you create a code snippet and test it - eg I have a bunch of pdf files in a folder, upload them all to an archive.
Ah sorry, that doesn't work. The archive item viewset is currently a ReadOnlyModelViewSet with the UpdateModelMixin, so create is not allowed on the gc side. Do you want me to create a pr to add the create mixin as well?
Currently testing against my local gc instance and making changes to make this work.
What are the implications of this approach? At the moment the only way to create an Archive Item is by uploading images, so I think that it means that we never have empty Archive Items. There might be assumptions about Archive Items being empty or not elsewhere in the codebase - in particular with type 2 challenges and algorithm linking.
That may very well be true, I'll have a look at what happens if there are archive items with no values. If that turns out to be problematic we need to implement a proper create function in the serializer that can handle nested values and error out if no values are supplied on create.
An intermediate solution would be to add a feature flag to the archive model that allows users to create display sets, then tell the user we set this for to always fill them and warn them about possible unexpected behavior while we work on the proper create serializer?
I think a convenience method in gcapi like we have for updating archive items would be nice and shouldn't be too hard to implement (but that also requires a create endpoint for archive items on the GC side, of course). 
Ok, I can now create an empty archive item via the api locally. I like Anne's suggestion, but that would still leave us with temorarily empty archive items. I'll have a look at the implications of that.
For algorithm jobs, any items with invalid values for the algorithm are filtered out before the job is created in the task, so that should not lead to any issues.
I cannot easily find how challenges are connected to archives, I assume through algorithms? In that case there should be no issue there either. Can someone confirm?
It's through evaluations: https://github.com/comic/grand-challenge.org/blob/70d9ad3b4e98f4b94e54f489e826040cd6329e96/app/grandchallenge/evaluation/tasks.py#L123 and it uses the same way of creating algorithms jobs, so should be fine indeed. 
Everything works via gcapi locally now, just not in the tests. PR should be out soon.


Resolving patch:
diff --git a/app/grandchallenge/archives/apps.py b/app/grandchallenge/archives/apps.py
--- a/app/grandchallenge/archives/apps.py
+++ b/app/grandchallenge/archives/apps.py
@@ -19,6 +19,10 @@ def init_archiveitem_permissions(*_, **__):
         f"{ArchiveItem._meta.app_label}.change_{ArchiveItem._meta.model_name}",
         g,
     )
+    assign_perm(
+        f"{ArchiveItem._meta.app_label}.add_{ArchiveItem._meta.model_name}",
+        g,
+    )
 
 
 class ArchivesConfig(AppConfig):
diff --git a/app/grandchallenge/archives/serializers.py b/app/grandchallenge/archives/serializers.py
--- a/app/grandchallenge/archives/serializers.py
+++ b/app/grandchallenge/archives/serializers.py
@@ -1,6 +1,7 @@
 from django.db.transaction import on_commit
 from guardian.shortcuts import get_objects_for_user
 from rest_framework import serializers
+from rest_framework.exceptions import ValidationError
 from rest_framework.fields import JSONField, ReadOnlyField, URLField
 from rest_framework.relations import HyperlinkedRelatedField
 
@@ -79,9 +80,14 @@ def __init__(self, *args, **kwargs):
             user = self.context["request"].user
 
             self.fields["archive"].queryset = get_objects_for_user(
-                user, "archives.use_archive", accept_global_perms=False
+                user, "archives.upload_archive", accept_global_perms=False
             )
 
+    def create(self, validated_data):
+        if validated_data.pop("values") != []:
+            raise ValidationError("Values can only be added via update")
+        return super().create(validated_data)
+
     def update(self, instance, validated_data):
         civs = validated_data.pop("values")
 
diff --git a/app/grandchallenge/archives/views.py b/app/grandchallenge/archives/views.py
--- a/app/grandchallenge/archives/views.py
+++ b/app/grandchallenge/archives/views.py
@@ -28,7 +28,7 @@
     PermissionRequiredMixin as ObjectPermissionRequiredMixin,
 )
 from rest_framework.decorators import action
-from rest_framework.mixins import UpdateModelMixin
+from rest_framework.mixins import CreateModelMixin, UpdateModelMixin
 from rest_framework.permissions import DjangoObjectPermissions
 from rest_framework.response import Response
 from rest_framework.settings import api_settings
@@ -705,7 +705,9 @@ def studies(self, request, pk=None):
         return Response(studies)
 
 
-class ArchiveItemViewSet(UpdateModelMixin, ReadOnlyModelViewSet):
+class ArchiveItemViewSet(
+    CreateModelMixin, UpdateModelMixin, ReadOnlyModelViewSet
+):
     queryset = ArchiveItem.objects.all().prefetch_related(
         "archive__hanging_protocol"
     )
@@ -715,7 +717,7 @@ class ArchiveItemViewSet(UpdateModelMixin, ReadOnlyModelViewSet):
     filterset_fields = ["archive"]
 
     def get_serializer_class(self):
-        if "update" in self.action:
+        if "update" in self.action or "create" in self.action:
             return ArchiveItemPostSerializer
         else:
             return ArchiveItemSerializer


Test patch:
diff --git a/app/tests/archives_tests/test_views.py b/app/tests/archives_tests/test_views.py
--- a/app/tests/archives_tests/test_views.py
+++ b/app/tests/archives_tests/test_views.py
@@ -460,6 +460,57 @@ def test_api_archive_item_add_and_update_non_image_file(client, settings):
     assert new_civ != civ
 
 
+@pytest.mark.django_db
+def test_api_archive_item_create(client, settings):
+    settings.task_eager_propagates = (True,)
+    settings.task_always_eager = (True,)
+    archive = ArchiveFactory()
+    editor, user = UserFactory.create_batch(2)
+    archive.add_editor(editor)
+    archive.add_user(user)
+
+    response = get_view_for_user(
+        viewname="api:archives-item-list",
+        client=client,
+        method=client.post,
+        data={"archive": archive.api_url, "values": []},
+        user=user,
+        content_type="application/json",
+        follow=True,
+    )
+
+    # User does not have access to the archive
+    assert response.status_code == 400
+    assert response.json()["archive"] == [
+        "Invalid hyperlink - Object does not exist."
+    ]
+
+    assert archive.items.count() == 0
+    response = get_view_for_user(
+        viewname="api:archives-item-list",
+        client=client,
+        method=client.post,
+        data={"archive": archive.api_url, "values": [{}]},
+        user=editor,
+        content_type="application/json",
+        follow=True,
+    )
+    assert response.status_code == 400
+    assert archive.items.count() == 0
+
+    response = get_view_for_user(
+        viewname="api:archives-item-list",
+        client=client,
+        method=client.post,
+        data={"archive": archive.api_url, "values": []},
+        content_type="application/json",
+        user=editor,
+        follow=True,
+    )
+    assert response.status_code == 201
+    assert archive.items.count() == 1
+
+
 @pytest.mark.django_db
 def test_archive_items_to_reader_study_update(client, settings):
     settings.task_eager_propagates = (True,)