Repository: apluslms/a-plus

Issue:
Chronological order for the students in the "assess next submitter manually" feature
In the assignment submissions list, there is a button "Start manual assessment" at the top. What is the order of the students when they are traversed through the "Assess next submitter manually" links (at the top of the inspect submission view)? The students could be sorted chronologically based on the submission time stamps (model field `Submission.submission_time`). The students could then be manually graded starting from the earliest students and progressing towards the latest submitters.

Note that students have multiple submissions and each of those has a different submission time. Should the students be sorted based on their latest submission time stamps or the time stamps of the best submissions?

Chronological order is useful in the grading of the Y1 course exam because the grading may be started before all students have taken the exam. Then, it is best to start from the student who took the exam at the start of the exam period.


Discussion:
> Note that students have multiple submissions and each of those has a different submission time. Should the students be sorted based on their latest submission time stamps or the time stamps of the best submissions?

One teaching assistant preferred **the latest submission time stamps**. It describes best when the student is finishing the exam (when manual assessment is used in exams in particular; manual assessment could be used during the course too). It is then convenient to assess in succession those students who took the exam at the same time. This may also help with spotting cheating in the exam studios.


The chronological sorting helps to avoid this unfortunate scenario:

1. assistants start manual assessment during the exam period before all students have taken the exam
2. assistant moves on to the next student using the "Assess next submitter manually" link.
3. If the students are not sorted, the assistant may start to assess students who are actually taking the exam right at that time. The student could still make new submissions and the student is not supposed to see the feedback during the exam. It is hard for the assistant to spot that the student has submitted on the same day when the assistant is assessing the students.

The problem (3) has already occurred in a real exam.

Resolving patch:
diff --git a/exercise/staff_views.py b/exercise/staff_views.py
--- a/exercise/staff_views.py
+++ b/exercise/staff_views.py
@@ -5,7 +5,7 @@
 from django.contrib.auth.models import User
 from django.core.exceptions import PermissionDenied, ValidationError
 from django.core.validators import URLValidator
-from django.db.models import Count, Max, Prefetch, Q
+from django.db.models import Count, Max, Prefetch, Q, F
 from django.http.request import HttpRequest
 from django.http.response import HttpResponse, JsonResponse, Http404
 from django.shortcuts import get_object_or_404
@@ -324,27 +324,42 @@ class NextUnassessedSubmitterView(ExerciseBaseView, BaseRedirectView):
     access_mode = ACCESS.ASSISTANT
 
     def get(self, request: HttpRequest, *args: Any, **kwargs: Any) -> HttpResponse:
-        # Query submitters who have not been assessed yet.
         submitter = None
-        submitters = (UserProfile.objects
-            .filter(submissions__exercise=self.exercise)
-            .annotate(
-                count_assessed=Count(
-                    'submissions__id',
-                    filter=(Q(submissions__grader__isnull=False)),
-                ),
-            )
-            .filter(count_assessed=0)
-            .order_by('id'))
+
+        # submissions for this exercise
+        submissions = Submission.objects.filter(exercise=self.exercise)
+
+        # filter only the latest submissions
+        submissions = (submissions.annotate(latest=Max('submission_time'))
+           .filter(submission_time=F('latest'))
+           .order_by('latest'))
 
         previous_user_id = request.GET.get('prev')
+        previous_submission_time = None
+
+        # if we had a previous user then we need to find their submission time before we filter already assessed users.
         if previous_user_id:
-            # Find specifically the submitter AFTER the previously inspected one.
-            submitters_after = submitters.filter(id__gt=previous_user_id)
-            submitter = submitters_after.first()
+            previous_submission_time = submissions.filter(submitters__id=previous_user_id).first().submission_time
+
+        # filter users who's have submissions that have been assessed
+        submissions = (submissions.annotate(
+            count_assessed=Count(
+                'id',
+                filter=(Q(grader__isnull=False)),
+            ),
+        )
+        .filter(count_assessed=0))
+
+        # if we had a previous time then try find the next submitter after this time
+        if previous_submission_time:
+            submission = submissions.filter(submission_time__gt=previous_submission_time).first()
+            if submission:
+                submitter = submission.submitters.first()
 
+        # if we don't have one just take the first submitter
         if not submitter:
-            submitter = submitters.first()
+            if submissions.first():
+                submitter = submissions.first().submitters.first()
 
         if not submitter:
             # There are no more unassessed submitters.


Test patch:
diff --git a/exercise/tests.py b/exercise/tests.py
--- a/exercise/tests.py
+++ b/exercise/tests.py
@@ -1349,3 +1349,86 @@ def test_enrollment_questionaire_opening_time(self):
             self.enrollment_exercise.check_submission_allowed(self.user.userprofile)[0],
             BaseExercise.SUBMIT_STATUS.CANNOT_ENROLL,
         )
+
+    def test_next_unassessed_submitter_view(self):
+        # parses the user ID from the URL response of NextUnassessedSubmitterView
+        def get_url_user_id(args=''):
+            response = self.client.get(
+                f"{exercise.get_absolute_url()}submitters/next-unassessed/{args}")
+
+            return int(response.url.split('/submissions/')[1].split('/inspect/')[0])
+
+        self.client.login(username="staff", password="staffPassword")
+
+        exercise = BaseExercise.objects.create(
+            order=10,
+            name="Unassessed Exercise Submitter View",
+            course_module=self.course_module,
+            category=self.learning_object_category,
+            url="unassessed",
+            max_submissions=1,
+        )
+        exercise.save()
+
+        user_submission = Submission.objects.create(
+            exercise=exercise,
+        )
+        user_submission.submitters.add(self.user.userprofile)
+        user_submission.submission_time = self.tomorrow
+        user_submission.save()
+
+        user2_submission = Submission.objects.create(
+            exercise=exercise,
+        )
+        user2_submission.submitters.add(self.user2.userprofile)
+        user2_submission.submission_time = self.two_days_from_now
+        user2_submission.save()
+
+        # user submission day before user2
+        self.assertEqual(user_submission.id, get_url_user_id())
+        self.assertEqual(user2_submission.id,
+                         get_url_user_id(f"?prev={self.user.id}"))
+
+        user_submission.submission_time = self.three_days_from_now
+        user_submission.save()
+
+        # now user2 submission is earlier
+        self.assertEqual(user2_submission.id, get_url_user_id())
+
+
+        # we should now expect to get user1's submission first since user2s earlier submission has been graded
+        user2_submission.grader = self.teacher.userprofile
+        user2_submission.save()
+        self.assertEqual(user_submission.id, get_url_user_id())
+
+        # test with prev parameter
+        self.assertEqual(user_submission.id, get_url_user_id(f"?prev={self.user2.id}"))
+
+        # remove grader for further tests
+        user2_submission.grader = None
+        user2_submission.save()
+
+        # create a submission for user so the newest submission was made by user again and not user2
+        user_earlier_submission = Submission.objects.create(
+            exercise=exercise,
+        )
+        user_earlier_submission.submitters.add(self.user.userprofile)
+        user_earlier_submission.submission_time = self.today
+        user_earlier_submission.save()
+
+        self.assertEqual(user_submission.id, get_url_user_id())
+
+        user2_other_exercise_submission = Submission.objects.create(
+            exercise=self.base_exercise,
+        )
+        user2_other_exercise_submission.submitters.add(self.user.userprofile)
+        user2_other_exercise_submission.submission_time = self.yesterday
+        user2_other_exercise_submission.save()
+
+        # an even earlier submission in another exercise doesn't matter
+        self.assertEqual(user_submission.id, get_url_user_id())
+
+        # we should now expect user2's submission to be shown because one of user1's submissions has been graded
+        user_earlier_submission.grader = self.teacher.userprofile
+        user_earlier_submission.save()
+        self.assertEqual(user2_submission.id, get_url_user_id())