Repository: Cloud-CV/EvalAI

Issue:
Add a feature to re-reun a submission by challenge host using UI
## Current Behaviour
If a challenge host wants to re-run a submission then the host has to email EvalAI admin to re-run a particular submission from admin.

## Expected Behaviour
The challenge host should be able to re-run the submission from the UI.

Please feel free to post the approach first.


Discussion:
@RishabhJain2018 I'd like to work on this 
@yashdusing You can take this up.
@RishabhJain2018 I'd like to work on this as well.
@RishabhJain2018 also, could you please verify the doubts I've asked on gitter please
Sure! I'll take a look at it soon.


@RishabhJain2018 So, the challenge host no longer has the access to the annotation file he/she previously submitted. So, the challenge host approaches the admin to fetch the submission annotation file to re-run it on the challenge ? Is that right or am I missing something ?
>  So, the challenge host no longer has the access to the annotation file he/she previously submitted. So, the challenge host approaches the admin to fetch the submission annotation file to re-run it on the challenge ? Is that right or am I missing something ?

I'm not sure why you're talking about annotation file here?
@RishabhJain2018 the file submitted by the participant which the challenge host wants to re-evaluate (re-run)
@RishabhJain2018 How about the challenge host having a re-run submission button on the leaderboard itself ? 
Also, should I use the existing run_submission function in the workers folder or write a function to get the existing files which might be saved onto the temp directory when worker is initiated ?
Hey, @yashdusing The file submitted by the participant is the `user_submission_file`. Please don't confuse it with `test_annotation_file`.

> How about the challenge host having a re-run submission button on the leaderboard itself ?

No, since the leaderboard doesn't list all the entries (as participants can make them private), I think it'd be ideal to have it under view all submissions tab.

> Also, should I use the existing run_submission function in the workers folder or write a function to get the existing files which might be saved onto the temp directory when worker is initiated ?

No, please try to understand the architecture, how it works. You need to pass the submission message to the SQS queue and it'll be processed automatically. I've already implemented an admin feature for it. Please take a look at that.  
@yashdusing  I guess publishing the submission parameters to the submission Queue would do the work, a similar kind of approach that we used here.
https://github.com/Cloud-CV/EvalAI/blob/343089d6b246bd0d5815ad9bb7288a958195211a/apps/jobs/admin.py#L57-L70
Yes @Ayukha . Thanks for pointing to the code.
@RishabhJain2018 Thanks for clarifying. I was a little confused and thought I needed to make changes directly to worker.py 😅
@RishabhJain2018 could you please check my PRs since most of them are not reviewed. Also, my proposal as well 

Resolving patch:
diff --git a/apps/jobs/urls.py b/apps/jobs/urls.py
--- a/apps/jobs/urls.py
+++ b/apps/jobs/urls.py
@@ -20,6 +20,10 @@
         views.get_remaining_submissions,
         name="get_remaining_submissions",
     ),
+    url(
+        r'^submissions/(?P<submission_pk>[0-9]+)/re-run/$',
+        views.re_run_submission, name='re_run_submission'
+    ),
     url(
         r"^challenge_phase_split/(?P<challenge_phase_split_id>[0-9]+)/leaderboard/$",
         views.leaderboard,
diff --git a/apps/jobs/views.py b/apps/jobs/views.py
--- a/apps/jobs/views.py
+++ b/apps/jobs/views.py
@@ -958,6 +958,42 @@ def update_submission(request, challenge_pk):
         return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
 
 
+@api_view(['POST', ])
+@throttle_classes([UserRateThrottle, ])
+@permission_classes((permissions.IsAuthenticated, HasVerifiedEmail))
+@authentication_classes((ExpiringTokenAuthentication,))
+def re_run_submission(request, submission_pk):
+    """
+    API endpoint to re-run a submission.
+    Only challenge host has access to this endpoint.
+    """
+    try:
+        submission = Submission.objects.get(pk=submission_pk)
+    except Submission.DoesNotExist:
+        response_data = {'error': 'Submission {} does not exist'.format(submission_pk)}
+        return Response(response_data, status=status.HTTP_404_NOT_FOUND)
+
+    # get the challenge and challenge phase object
+    challenge_phase = submission.challenge_phase
+    challenge = challenge_phase.challenge
+
+    if not is_user_a_host_of_challenge(request.user, challenge.pk):
+        response_data = {
+            "error": "Only challenge hosts are allowed to re-run a submission"
+        }
+        return Response(response_data, status=status.HTTP_403_FORBIDDEN)
+
+    if not challenge.is_active:
+        response_data = {'error': 'Challenge {} is not active'.format(challenge.title)}
+        return Response(response_data, status=status.HTTP_406_NOT_ACCEPTABLE)
+
+    publish_submission_message(challenge.pk, challenge_phase.pk, submission.pk)
+    response_data = {
+        'success': 'Submission is successfully submitted for re-running'
+    }
+    return Response(response_data, status=status.HTTP_200_OK)
+
+
 @api_view(["GET"])
 @throttle_classes([UserRateThrottle])
 @permission_classes((permissions.IsAuthenticated, HasVerifiedEmail))


Test patch: