Repository: Cloud-CV/EvalAI

Issue:
Improve dataset_split selection logic in submission worker
After integrating #1560, there can be multiple dataset_splits with the same name and codename and the code at  

https://github.com/Cloud-CV/EvalAI/blob/master/scripts/workers/submission_worker.py#L335

will fail as it will return `MultipleObjectsReturned` exception in the case where there are two or more dataset splits which have the same values of codename and split name. 
Improve dataset_split selection logic in submission worker
After integrating #1560, there can be multiple dataset_splits with the same name and codename and the code at  

https://github.com/Cloud-CV/EvalAI/blob/master/scripts/workers/submission_worker.py#L335

will fail as it will return `MultipleObjectsReturned` exception in the case where there are two or more dataset splits which have the same values of codename and split name. 


Discussion:

Resolving patch:
diff --git a/scripts/workers/submission_worker.py b/scripts/workers/submission_worker.py
--- a/scripts/workers/submission_worker.py
+++ b/scripts/workers/submission_worker.py
@@ -48,7 +48,6 @@
 from challenges.models import (Challenge,
                                ChallengePhase,
                                ChallengePhaseSplit,
-                               DatasetSplit,
                                LeaderboardData) # noqa
 
 from jobs.models import Submission          # noqa
@@ -329,24 +328,26 @@ def run_submission(challenge_id, challenge_phase, submission_id, submission, use
             leaderboard_data_list = []
             for split_result in submission_output['result']:
 
-                # Check if the dataset_split exists for the codename in the result
+                # get split_code_name that is the key of the result
+                split_code_name = split_result.items()[0][0]
+
+                # Check if the challenge_phase_split exists for the challenge_phaseand dataset_split
                 try:
-                    split_code_name = split_result.items()[0][0]  # get split_code_name that is the key of the result
-                    dataset_split = DatasetSplit.objects.get(codename=split_code_name)
+                    challenge_phase_split = ChallengePhaseSplit.objects.get(challenge_phase=challenge_phase,
+                                                                            dataset_split__codename=split_code_name)
                 except:
-                    stderr.write("ORGINIAL EXCEPTION: The codename specified by your Challenge Host doesn't match"
-                                 " with that in the evaluation Script.\n")
+                    stderr.write("ORGINIAL EXCEPTION: No such relation between Challenge Phase and DatasetSplit"
+                                 " specified by Challenge Host \n")
                     stderr.write(traceback.format_exc())
                     successful_submission_flag = False
                     break
 
-                # Check if the challenge_phase_split exists for the challenge_phase and dataset_split
+                # Check if the dataset_split exists for the codename in the result
                 try:
-                    challenge_phase_split = ChallengePhaseSplit.objects.get(challenge_phase=challenge_phase,
-                                                                            dataset_split=dataset_split)
+                    dataset_split = challenge_phase_split.dataset_split
                 except:
-                    stderr.write("ORGINIAL EXCEPTION: No such relation between between Challenge Phase and DatasetSplit"
-                                 " specified by Challenge Host \n")
+                    stderr.write("ORGINIAL EXCEPTION: The codename specified by your Challenge Host doesn't match"
+                                 " with that in the evaluation Script.\n")
                     stderr.write(traceback.format_exc())
                     successful_submission_flag = False
                     break


Test patch: