internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: apluslms/a-plus

Issue:
Remove the HTML attribute id="exercise" when chapters and exercises are rendered in the A+ frontend
HTML id attributes must be unique. However, currently, the `id="exercise"` is used multiple times. It seems that the chapter content itself has one `<div id="exercise">` and each embedded exercise also has one `<div id="exercise">`.

One approach would be to remove the attribute `id="exercise"` when A-plus retrieves chapter and exercise HTML from the backend. A-plus already manipulates the HTML a bit before rendering it to the web browser.

Another approach is to change the templates in the MOOC-Grader so that the `id="exercise"` is never even included anywhere, but I think it is usually used to pick the correct part of the HTML document retrieved from the backend (Git manager or MOOC-Grader).

Very related to one subtask in https://github.com/apluslms/a-plus/issues/667.
A little related to https://github.com/apluslms/a-plus/issues/593.
Id attribute issue in active elements: https://github.com/apluslms/a-plus/issues/823.

Related code:
* https://github.com/apluslms/a-plus/blob/660f98d0906b05a925fca64c6f2c3906497fc8f6/exercise/protocol/aplus.py#L157-L164
* https://github.com/apluslms/mooc-grader/blob/af39777890fc62805d12963abae994d85e11525b/access/templates/access/exercise_frame.html#L9
* https://github.com/apluslms/a-plus/blob/660f98d0906b05a925fca64c6f2c3906497fc8f6/exercise/templates/exercise/exercise_plain.html#L193


Discussion:

Resolving patch:
diff --git a/lib/remote_page.py b/lib/remote_page.py
--- a/lib/remote_page.py
+++ b/lib/remote_page.py
@@ -175,6 +175,8 @@ def select_element_or_body(self, search_attributes):
 
     def element_or_body(self, search_attributes):
         element = self.select_element_or_body(search_attributes)
+        if element.get('id') == 'exercise':
+            del element['id']
         return str(element) if element else ""
 
     def clean_element_or_body(self, search_attributes):


Test patch:
Repository: apluslms/a-plus

Issue:
A+ must not try to render binary files in plain text in the submitted file views
Student's submitted files are usually opened in a modal dialog for a preview of the contents. In the teacher's inspect submission page, the preview is within the page without a modal dialog. The preview works for plain text files. A+ does not recognize all binary file types, thus A+ tries to render previews for some binary files too. That results in bad user experience because the preview shows only garbled text and non-printable characters.

A+ must not try to render binary files in plain text. Binary files may only be downloaded without a preview in the web page.

A+ already recognizes some binary file types, but their MIME types are hardcoded, which is not optimal:
https://github.com/apluslms/a-plus/blob/8811feab20862474ac69951ec04315a98e79c659/exercise/submission_models.py#L606

Microsoft Office files (.doc, .docx, .xls, .xlsx) are particularly missing. They are needed urgently.

If there were a better way to recognize binary files without listing them manually, that would be the best. There are libraries that try to recognize files types from the header of the file or try to check if the file contains only printable ASCII/Unicode characters/bytes.

Students see submitted files in submission feedback pages and modal dialogs. Course staff see submitted files in the inspect submission page in addition to the student participant overview and the submission feedback pages.

Related to #728.

Internal ticket:
https://rt.cs.aalto.fi/Ticket/Display.html?id=20621


Discussion:

Resolving patch:
diff --git a/exercise/submission_models.py b/exercise/submission_models.py
--- a/exercise/submission_models.py
+++ b/exercise/submission_models.py
@@ -4,6 +4,7 @@
 import json
 from typing import IO, Dict, Iterable, List, Tuple, TYPE_CHECKING
 
+from binaryornot.check import is_binary
 from django.conf import settings
 from django.core.files.storage import default_storage
 from django.db import models, DatabaseError
@@ -642,7 +643,7 @@ def get_mime(self):
         return guess_type(self.file_object.path)[0]
 
     def is_passed(self):
-        return self.get_mime() in SubmittedFile.PASS_MIME
+        return is_binary(self.file_object.path)
 
 
     ABSOLUTE_URL_NAME = "submission-file"


Test patch:
Repository: apluslms/a-plus

Issue:
Add button to regenerate the API access token
Currently it is not possible for the user to re-generate new API token in the user interface. The user profile view should therefore have a button for regenerating one.


Discussion:

Resolving patch:
diff --git a/userprofile/models.py b/userprofile/models.py
--- a/userprofile/models.py
+++ b/userprofile/models.py
@@ -145,6 +145,13 @@ def get_url(self, instance):
         kwargs = dict(user_id=self.user.id, **instance.get_url_kwargs())
         return reverse('user-results', kwargs=kwargs)
 
+    def regenerate_api_token(self):
+        # FIXME: implement support for more than 1 token
+        token, created = Token.objects.get_or_create(user=self.user)
+        if not created:
+            token.delete()
+            Token.objects.create(user=self.user)
+
 
 def create_user_profile(sender, instance, created, **kwargs):
     """
diff --git a/userprofile/urls.py b/userprofile/urls.py
--- a/userprofile/urls.py
+++ b/userprofile/urls.py
@@ -17,5 +17,6 @@
     url(r'^accounts/$', views.ProfileView.as_view(),
         name="profile"),
     url(r'^setlang', views.set_user_language, name="set-user-language"),
+    url(r'^regentoken/$', views.regenerate_access_token, name="regenerate-access-token"),
     url(r'^teachers/$', views.TeacherListView.as_view(), name="teachers"),
 ]
diff --git a/userprofile/views.py b/userprofile/views.py
--- a/userprofile/views.py
+++ b/userprofile/views.py
@@ -4,6 +4,7 @@
 from urllib.parse import unquote
 
 from django.conf import settings
+from django.contrib.auth.decorators import login_required
 from django.contrib.auth.views import LoginView
 from django.core.cache import cache
 from django.core.cache.utils import make_template_fragment_key
@@ -18,6 +19,7 @@
 )
 from django.utils.translation import gettext_lazy as _
 from django.utils.translation import gettext
+from django.views.decorators.http import require_POST
 
 from authorization.permissions import ACCESS
 from course.models import CourseInstance
@@ -73,7 +75,7 @@ def get_context_data(self, **kwargs):
 
 
 def set_user_language(request):
-    """"Overrides set_language function from  django.views.i18n."""
+    """Overrides set_language function from django.views.i18n."""
     LANGUAGE_PARAMETER = 'language'
 
     next = remove_query_param_from_url(request.POST.get('next', request.GET.get('next')), 'hl')
@@ -111,6 +113,14 @@ def set_user_language(request):
     return response
 
 
+@login_required
+@require_POST
+def regenerate_access_token(request):
+    """Regenerates the API access token."""
+    request.user.userprofile.regenerate_api_token()
+    return HttpResponseRedirect(reverse('profile'))
+
+
 def try_get_template(name):
     try:
         return get_template(name)


Test patch:
Repository: apluslms/a-plus

Issue:
Remove sign in as another user
It shouldn't be possible to log in as another user in a production system. Remove the old code for the SignInAsUser view completely.

* https://github.com/apluslms/a-plus/blob/e832ba304b22392e8acc0e47292a4ccd3dae24b0/edit_course/views.py#L396-L413
* https://github.com/apluslms/a-plus/blob/e832ba304b22392e8acc0e47292a4ccd3dae24b0/edit_course/urls.py#L46-L48


Discussion:
is this possible for end-users? Needs to be removed, indeed :)
Is this needed for admins, though? Compared to Moodle; admins can log in as another user, see https://docs.moodle.org/400/en/Accounts_FAQ#How_can_I_log_in_as_another_user.3F but of course this is logged, as other actions as well.
Not possible for end-users and not needed by admins. I have never used it and wouldn't want to since it is suspicious.
ok!
It's mostly just reverting this:
https://github.com/apluslms/a-plus/commit/4e2bc94c1006b7a65b89bbc30a94457dd4baab35

There are just a few lines that are used for another purpose in that commit

Resolving patch:
diff --git a/edit_course/urls.py b/edit_course/urls.py
--- a/edit_course/urls.py
+++ b/edit_course/urls.py
@@ -43,9 +43,6 @@
     url(EDIT_URL_PREFIX + r'batch-assess/$',
         views.BatchCreateSubmissionsView.as_view(),
         name='batch-assess'),
-    url(r'^admin/signin-as-user$',
-        views.SignInAsUser.as_view(),
-        name='signin-as-user'),
 
     url(MODEL_URL_PREFIX + r'add/$',
         views.ModelEditView.as_view(),
diff --git a/edit_course/views.py b/edit_course/views.py
--- a/edit_course/views.py
+++ b/edit_course/views.py
@@ -391,23 +391,3 @@ def get_context_data(self, *args, **kwargs):
         context = super().get_context_data(*args, **kwargs)
         context.update(get_build_log(self.instance))
         return context
-
-
-class SignInAsUser(BaseRedirectMixin, BaseTemplateView):
-    access_mode = ACCESS.SUPERUSER
-    template_name = "edit_course/signin_as_user.html"
-
-    def post(self, request, *args, **kwargs):
-        username = request.POST.get('username', None)
-        try:
-            user = User.objects.get(username=username)
-        except User.DoesNotExist:
-            messages.error(request,
-                format_lazy(
-                    _('ERROR_USERNAME_DOESNT_EXIST -- {username}'),
-                    username=html.escape(username),
-                )
-            )
-            return self.redirect(reverse('signin-as-user'))
-        auth_login(request, user, backend="django.contrib.auth.backends.ModelBackend")
-        return self.redirect("/")


Test patch:
Repository: apluslms/a-plus

Issue:
Clarify the batch assess instructions
Make small improvements to the teacher's instructions in the batch assess page (Edit course section - Batch assess or in Finnish Joukkoarviointi). This is a very small task.

**The part "Z1 an Z2 (integers)" refers to assignment ids, but teachers do not know the assignment ids in the A+ database.** Explain that they can be read from two places:

1. Look at the course exercises API. A link to the exercises API page of the current course should be added here. Since the course id varies, the message string needs a variable for the course. It is best to supply the whole URL as a variable to the translated message since the whole URL is received from the URL reverse function. A relative URL without the server domain should work ("/api/v2/courses/COURSEID/exercises") since the web browser uses the domain of the current site automatically. The browser search Ctrl+F works in the exercise list API page.
2. Look at the web address of the Edit assignment button under the Edit course section - Content. The Edit assignment address includes a number after the course keys.

* https://github.com/apluslms/a-plus/blob/d33fa893d426f165b62a1acbe4de4adfe4c7b647/locale/en/LC_MESSAGES/django.po#L2247-L2256
* https://github.com/apluslms/a-plus/blob/d33fa893d426f165b62a1acbe4de4adfe4c7b647/edit_course/templates/edit_course/batch_assess.html#L22
* https://github.com/apluslms/a-plus/blob/d33fa893d426f165b62a1acbe4de4adfe4c7b647/edit_course/views.py#L301

For creating the course URL "/api/v2/courses/COURSEID/exercises":
`api_reverse` function
* https://github.com/apluslms/a-plus/blob/d33fa893d426f165b62a1acbe4de4adfe4c7b647/aplus/api/__init__.py#L1-L8
* Django URL name `course-exercises`
* https://github.com/apluslms/a-plus/blob/d33fa893d426f165b62a1acbe4de4adfe4c7b647/api/urls_v2.py#L35-L37


Internal ticket:
https://rt.cs.aalto.fi/Ticket/Display.html?id=20789


Discussion:

Resolving patch:
diff --git a/edit_course/views.py b/edit_course/views.py
--- a/edit_course/views.py
+++ b/edit_course/views.py
@@ -25,6 +25,7 @@
     BaseRedirectView,
     BaseFormView,
 )
+from aplus.api import api_reverse
 from authorization.permissions import ACCESS
 from course.models import CourseInstance, UserTag, UserTagging
 from course.viewbase import CourseInstanceBaseView, CourseInstanceMixin
@@ -316,6 +317,12 @@ def post(self, request, *args, **kwargs):
             messages.success(request, _('NEW_SUBMISSIONS_STORED'))
         return self.render_to_response(self.get_context_data())
 
+    def get_context_data(self, **kwargs):
+        context = super().get_context_data(**kwargs)
+        exercises_api_url = api_reverse("course-exercises-list", kwargs={'course_id': self.instance.id})
+        context.update({'exercises_api_url': exercises_api_url})
+        return context
+
 
 class CloneInstanceView(CourseInstanceMixin, BaseFormView):
     access_mode = ACCESS.TEACHER


Test patch:
Repository: apluslms/a-plus

Issue:
Problems with Enrollment questionnaire and SISU enrollments
Quite recently a new "PENDING" enrollment state was added to support enrollment questionnaires with courses where students are enrolled from SISU. On summer Y1 course this feature was first time in use in production, but does not appear to work properly. Students are set to PENDING state properly when fetched from SISU, but are not redirected to enrollment questionnaire properly as they should. Instead they get "You must enrol on this course through Student Information System." note which should not be shown in this case.


Discussion:
Status update: I managed to reproduce this locally. The problem occurs if content visibility is set to "Enrolled students". In this case "PENDING" students get the above message, and are not redirected to the questionnaire as they should.

Resolving patch:
diff --git a/course/viewbase.py b/course/viewbase.py
--- a/course/viewbase.py
+++ b/course/viewbase.py
@@ -11,13 +11,14 @@
 from lib.helpers import remove_query_param_from_url, update_url_params
 from lib.viewbase import BaseTemplateView
 from userprofile.viewbase import UserProfileMixin
+from exercise.models import LearningObject
 from .cache.students import CachedStudent
 from .exceptions import TranslationNotFound
 from .permissions import (
     CourseVisiblePermission,
     CourseModulePermission,
 )
-from .models import Course, CourseInstance, CourseModule, UserTagging
+from .models import Course, CourseInstance, CourseModule, UserTagging, Enrollment
 
 
 class CourseMixin(UserProfileMixin):
@@ -158,6 +159,15 @@ def handle_no_permission(self):
             # - the user is signed in but not enrolled or staff
             # - the page is not a teacher page (e.g. edit course)
             # - the course is visible only to enrolled students
+            #
+            # If SIS enrollment is applied and course requires enrollment questionnaire,
+            # redirect to the questionnaire instead.
+            enrollment = self.user_course_data
+            if enrollment and enrollment.status == Enrollment.ENROLLMENT_STATUS.PENDING:
+                exercise = LearningObject.objects.find_enrollment_exercise(
+                    self.instance, self.profile.is_external)
+                if exercise:
+                    return self.redirect(exercise.get_absolute_url())
             return redirect(self.instance.get_url('enroll'))
         return super().handle_no_permission()
 


Test patch:
Repository: apluslms/a-plus

Issue:
LTI Tool: Some links not working properly when embedded in iframe
When run as embedded iframe from MyCourses, some of the links in A+ bluntly show (in my case) "minus.cs.aalto.fi refused to connect", but when opened as separate window they work, or at least show something more sensible. Something should be done about them so that they show something more informative also when opened in iframe. For example, these buttons/links have the issue:

- [x] "Edit assignment" button when operating as teacher (the button could be removed in LTI mode?)
- [ ] "View all submissions" button (the button could be removed in LTI mode?)
- [x] When submitting solution to an exercise with expired deadline. It seems, however, that when trying to submit excess submissions after full submission count, it works ok, showing a banner complaining about this.

Also the legendary "403 Gandalf" page (see issue #858 ) causes the same, for example when A+ course is hidden from students, or the user has another reason for not allowed to see the page.


Discussion:

Resolving patch:
diff --git a/authorization/views.py b/authorization/views.py
--- a/authorization/views.py
+++ b/authorization/views.py
@@ -1,7 +1,8 @@
 from django.contrib.auth.mixins import AccessMixin
 from django.contrib.auth.views import redirect_to_login
 from django.contrib.messages import error as error_message
-from django.core.exceptions import PermissionDenied
+from django.shortcuts import render
+from django.views.defaults import ERROR_403_TEMPLATE_NAME
 
 from .exceptions import ValidationFailed
 from .permissions import NoPermission
@@ -65,9 +66,11 @@ def handle_no_permission(self):
         # on login and error pages
         error_message(request, message)
 
-        # For authenticated users, raise exception.
+        # For authenticated users, render the 403 forbidden template.
         if request.user.is_authenticated:
-            raise PermissionDenied(message)
+            # Do not `raise PermissionDenied(message)` here so that
+            # view decorators such as `xframe_options_exempt` apply.
+            return render(request, ERROR_403_TEMPLATE_NAME, status=403)
 
         # Redirect not authenticated users to login
         return redirect_to_login(request.get_full_path(),
diff --git a/lti_tool/views.py b/lti_tool/views.py
--- a/lti_tool/views.py
+++ b/lti_tool/views.py
@@ -128,6 +128,11 @@ def parse_lti_session_params(self):
             )
             self.message_launch_data = self.message_launch.get_launch_data()
 
+    def get_common_objects(self):
+        super().get_common_objects()
+        self.disable_staff_nav = True
+        self.note("disable_staff_nav")
+
 
 class LtiInstanceView(LtiSessionMixin, InstanceView):
 


Test patch:
Repository: apluslms/a-plus

Issue:
Link to newest course instance does not appear on every page in old course instances
There is supposed to be a link to the newest visible course instance on old course instance pages. However, this link does not always appear at the top of the page.


Discussion:

Resolving patch:
diff --git a/course/models.py b/course/models.py
--- a/course/models.py
+++ b/course/models.py
@@ -16,6 +16,7 @@
 from django.db.models import F, Q
 from django.db.models.signals import post_save
 from django.utils import timezone
+from django.utils.functional import cached_property
 from django.utils.text import format_lazy
 from django.utils.translation import gettext_lazy as _
 from django_colortag.models import ColorTag
@@ -776,6 +777,20 @@ def course_staff_and_students(self):
             enrollment__course_instance=self
         )
 
+    @cached_property
+    def later_instance(self):
+        later_instance = None
+        if self.is_past:
+            try:
+                later_instance = (
+                    CourseInstance.objects
+                        .filter(course=self.course, ending_time__gte=timezone.now())
+                        .latest('starting_time')
+                )
+            except CourseInstance.DoesNotExist:
+                pass
+        return later_instance
+
     def save(self, *args, **kwargs):
         """
         Saves the model.
diff --git a/course/templatetags/course.py b/course/templatetags/course.py
--- a/course/templatetags/course.py
+++ b/course/templatetags/course.py
@@ -65,6 +65,11 @@ def is_visible(entry):
     return CachedContent.is_visible(entry)
 
 
+@register.filter
+def is_visible_to(entry, user):
+    return entry.is_visible_to(user)
+
+
 @register.filter
 def is_listed(entry):
     return CachedContent.is_listed(entry)
diff --git a/course/views.py b/course/views.py
--- a/course/views.py
+++ b/course/views.py
@@ -169,8 +169,6 @@ def get(self, request, *args, **kwargs):
         elif lti_msg:
             messages.info(request, html.escape(lti_msg))
 
-        later_instance = None
-
         # PENDING student has enrolled, but not yet responded to the enrollment questionnaire.
         if request.user.is_authenticated:
             enrollment = self.user_course_data
@@ -182,19 +180,6 @@ def get(self, request, *args, **kwargs):
                 if exercise:
                     return self.redirect(exercise.get_absolute_url())
 
-        if self.instance.is_past:
-            try:
-                later_instance = (
-                    CourseInstance.objects
-                        .get_visible(request.user)
-                        .filter(course=self.course, ending_time__gte=timezone.now())
-                        .latest('starting_time')
-                )
-            except CourseInstance.DoesNotExist:
-                pass
-
-        kwargs['later_instance'] = later_instance
-
         return super().get(request, *args, **kwargs)
 
 class Enroll(EnrollableViewMixin, BaseRedirectMixin, BaseTemplateView):


Test patch:
Repository: apluslms/a-plus

Issue:
Hide exercises in the maintenance mode from the course table of contents
Exercises in maintenance mode (LearningObject.status maintenance) should be hidden from course table of contents.

Exercises with the status hidden are already hidden in the table of contents. The maintenance mode could be handled in the code in the same way.



Discussion:

Resolving patch:
diff --git a/exercise/cache/hierarchy.py b/exercise/cache/hierarchy.py
--- a/exercise/cache/hierarchy.py
+++ b/exercise/cache/hierarchy.py
@@ -302,7 +302,10 @@ def is_listed(cls, entry):
             return (
                 entry.get('category_status') != LearningObjectCategory.STATUS.HIDDEN
                 and entry.get('module_status') != CourseModule.STATUS.UNLISTED
-                and entry['status'] != LearningObject.STATUS.UNLISTED
+                and not entry['status'] in (
+                    LearningObject.STATUS.UNLISTED,
+                    LearningObject.STATUS.MAINTENANCE,
+                )
             )
         if t == 'module':
             return entry['status'] != CourseModule.STATUS.UNLISTED


Test patch:
Repository: apluslms/a-plus

Issue:
Empty date fields when adding a course instance results in 500 internal server error 
Saving a new course instance with the required date fields left empty causes an internal server error. Seems like the field validation doesn't check for empty fields.
Empty date fields when adding a course instance results in 500 internal server error 
Saving a new course instance with the required date fields left empty causes an internal server error. Seems like the field validation doesn't check for empty fields.


Discussion:
The crash occurred in the Django admin page for creating new CourseInstances. The code in `CourseInstance.clean()` clearly tries to use the mandatory starting and ending times before checking if the values are `None`. The less than comparison with the `None` value causes a crash.

https://github.com/apluslms/a-plus/blob/7e4bd40f0df6653b51953ab5bf0100ed681f0f51/course/models.py#L704
The crash occurred in the Django admin page for creating new CourseInstances. The code in `CourseInstance.clean()` clearly tries to use the mandatory starting and ending times before checking if the values are `None`. The less than comparison with the `None` value causes a crash.

https://github.com/apluslms/a-plus/blob/7e4bd40f0df6653b51953ab5bf0100ed681f0f51/course/models.py#L704

Resolving patch:
diff --git a/course/models.py b/course/models.py
--- a/course/models.py
+++ b/course/models.py
@@ -707,7 +707,7 @@ def clean(self):
             errors['instance_name'] = format_lazy(_('COURSE_INSTANCE_ERROR_INSTANCE_NAME -- {}'), self.instance_name)
         if self.url in RESERVED:
             errors['url'] = format_lazy(_('COURSE_INSTANCE_ERROR_URL -- {}'), self.url)
-        if self.ending_time <= self.starting_time:
+        if self.ending_time and self.starting_time and self.ending_time <= self.starting_time:
             errors['ending_time'] = _('COURSE_INSTANCE_ERROR_ENDING_TIME_BEFORE_STARTING')
         if self.lifesupport_time and self.lifesupport_time < self.ending_time:
             errors['lifesupport_time'] = _('COURSE_INSTANCE_ERROR_LIFESUPPORT_TIME_BEFORE_ENDING')


Test patch: