internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: learningequality/kolibri

Issue:
Exporting CSV file that contains Unicode text fails on Python 3 on Windows
### Observed behavior
User reports of CSV file export failing prompted me to add a test for this in a Windows environment in https://github.com/learningequality/kolibri/pull/7899.

When Unicode text should be inserted into the exported CSV, such as a Unicode username or full_name, then an error occurs on windows.

```
Error trying to write csv file: 'charmap' codec can't encode characters in position 29-35: character maps to <undefined>
```

### Expected behavior
CSV Export should work on all platforms.

This is fixed by this commit: https://github.com/learningequality/kolibri/pull/7899/commits/c088d08b3e6c4a0f4194a17062457cbd7ae34816

### User-facing consequences
Windows users are unable to export CSV files.



Discussion:

Resolving patch:
diff --git a/kolibri/core/logger/csv_export.py b/kolibri/core/logger/csv_export.py
--- a/kolibri/core/logger/csv_export.py
+++ b/kolibri/core/logger/csv_export.py
@@ -154,7 +154,7 @@ def csv_file_generator(facility, log_type, filepath, overwrite=False):
     if sys.version_info[0] < 3:
         csv_file = io.open(filepath, "wb")
     else:
-        csv_file = io.open(filepath, "w", newline="")
+        csv_file = io.open(filepath, "w", newline="", encoding="utf-8")
 
     with csv_file as f:
         writer = csv.DictWriter(f, header_labels)


Test patch:
Repository: learningequality/kolibri

Issue:
Create temporary Learn Plugin options flag using options.ini mechanisms to disable custom channel navigation by default



Discussion:
Is this just to establish the mechanism so that one can import from the options the value of some flag so that we can disable it in dev? If not - what is the temporary nature of this?

Resolving patch:
diff --git a/kolibri/plugins/learn/kolibri_plugin.py b/kolibri/plugins/learn/kolibri_plugin.py
--- a/kolibri/plugins/learn/kolibri_plugin.py
+++ b/kolibri/plugins/learn/kolibri_plugin.py
@@ -16,11 +16,13 @@
 from kolibri.core.webpack import hooks as webpack_hooks
 from kolibri.plugins import KolibriPluginBase
 from kolibri.plugins.hooks import register_hook
+from kolibri.utils import conf
 
 
 class Learn(KolibriPluginBase):
     untranslated_view_urls = "api_urls"
     translated_view_urls = "urls"
+    kolibri_options = "options"
 
 
 @register_hook
@@ -51,6 +53,9 @@ def plugin_data(self):
         return {
             "allowGuestAccess": get_device_setting("allow_guest_access"),
             "allowLearnerUnassignedResourceAccess": allow_learner_unassigned_resource_access(),
+            "enableCustomChannelNav": conf.OPTIONS["Learn"][
+                "KOLIBRI_ENABLE_CUSTOM_CHANNEL_NAV"
+            ],
         }
 
 
diff --git a/kolibri/plugins/learn/options.py b/kolibri/plugins/learn/options.py
new file mode 100644
--- /dev/null
+++ b/kolibri/plugins/learn/options.py
@@ -0,0 +1,9 @@
+option_spec = {
+    "Learn": {
+        "KOLIBRI_ENABLE_CUSTOM_CHANNEL_NAV": {
+            "type": "boolean",
+            "default": False,
+            "envvars": ("KOLIBRI_ENABLE_CUSTOM_CHANNEL_NAV",),
+        },
+    },
+}


Test patch:
Repository: learningequality/kolibri

Issue:
Context not transferred from Crowdin
<!--
Instructions:
 * Fill out the sections below, replace …'s with information about your issue
 * Use the 'preview' function above this text box to verify formatting before submitting
-->

### Observed behavior
<!--
Description of the behavior that was observed, including screenshots or other references when applicable
-->

Initially noted by @radinamatic 

This [Vue SFC](https://github.com/learningequality/kolibri/blob/release-v0.14.x/kolibri/plugins/learn/assets/src/views/classes/ClassAssignmentsPage.vue#L68) did not have the [context for its translation](https://crowdin.com/translate/kolibri/3798/en-es?filter=basic&value=0#275962) transferred into the code.

This issue should be fixed in the context of 0.15. This work may ultimately be blocked by some necessary work in  https://github.com/learningequality/kolibri/issues/7709 

### Expected behavior
<!--
Description of what behavior was expected but did not occur
-->

`yarn transfercontext` should transfer all context from Crowdin into the components associated with the translation strings.

### User-facing consequences
<!--
Implications and real-world consequences for learners, coaches, admins, and other users of the application
-->

Internal facing only - but context will not be available in the code, which can result in losing context when uploading the next round of translations. 


### Context
<!--
Tell us about your environment, including:
 * Kolibri version
 * Operating system
 * Browser
-->

Kolibri 0.15 


Discussion:
Is that supported in the format? https://docs.weblate.org/en/latest/formats.html?highlight=vue#translation-types-capabilities doesn't have it for JSON. Why use Crowdin though?
Apologies for the long delay @comradekingu

We use Crowdin as it provides our translators with an excellent interface for performing translations, seeing the context in which their translations are viewed and for our team to audit everything as we need.

We interface with Crowdin via their API and synchronize strings that provide context between the two while we perform our translations.

Resolving patch:
diff --git a/packages/kolibri-tools/lib/i18n/fonts.py b/packages/kolibri-tools/lib/i18n/fonts.py
--- a/packages/kolibri-tools/lib/i18n/fonts.py
+++ b/packages/kolibri-tools/lib/i18n/fonts.py
@@ -35,6 +35,8 @@
         os.path.dirname(__file__),
         os.pardir,
         os.pardir,
+        os.pardir,
+        os.pardir,
         "kolibri",
         "core",
         "static",


Test patch:
Repository: learningequality/kolibri

Issue:
Running 'generateuserdata' command re-enrolls users to facility after first run
To reproduce, do something like:

```
# on a clean and empty .kolibri/ folder
kolibri manage generateuserdata # generates 2 classes by default
kolibri manage generateuserdata --classes=3 # will add one more class
```

Get this in the stack trace: `django.db.utils.IntegrityError: UNIQUE constraint failed: kolibriauth_facilityuser.username, kolibriauth_facilityuser.facility_id`

It seems that even with the new random sample, there are at least some users in the first two classes being added to the third. To test this theory, I added '123' to the usernames from the CSV file and the problem went away.


Discussion:
To fix this, we should do `get_or_create`.

Resolving patch:
diff --git a/kolibri/core/logger/utils/user_data.py b/kolibri/core/logger/utils/user_data.py
--- a/kolibri/core/logger/utils/user_data.py
+++ b/kolibri/core/logger/utils/user_data.py
@@ -10,6 +10,7 @@
 from django.db.models import Min
 from django.db.models import Sum
 from django.db.models.query import Q
+from django.db.utils import IntegrityError
 from django.utils import timezone
 from le_utils.constants import content_kinds
 
@@ -160,16 +161,21 @@ def get_or_create_classroom_users(**options):
             birth_year = str(current_year - int(base_data["Age"]))
             # randomly assign gender
             gender = random.choice(demographics.choices)[0]
-            user = FacilityUser.objects.create(
-                facility=facility,
-                full_name=name,
-                username=base_data["Username"],
-                gender=gender,
-                birth_year=birth_year,
-            )
-            # Set a dummy password so that if we want to login as this learner later, we can.
-            user.set_password("password")
-            user.save()
+            try:
+                user = FacilityUser.objects.create(
+                    facility=facility,
+                    full_name=name,
+                    username=base_data["Username"],
+                    gender=gender,
+                    birth_year=birth_year,
+                )
+                # Set a dummy password so that if we want to login as this learner later, we can.
+                user.set_password("password")
+                user.save()
+            except IntegrityError:
+                user = FacilityUser.objects.get(
+                    facility=facility, username=base_data["Username"]
+                )
 
             # Add the user to the current classroom
             classroom.add_member(user)


Test patch:
Repository: learningequality/kolibri

Issue:
Add PROJECT Deployment Option and return with telemetry data
## Observed behavior
We currently have no good way to track installers built for custom projects in our telemetry data.

## Expected behavior
Add an option to our options.py under `Deployment` as `PROJECT` to allow us to set default values for this in custom installers.

We should then report this data back in our telemetry ping backs as a top level field.


Discussion:

Resolving patch:
diff --git a/kolibri/core/analytics/utils.py b/kolibri/core/analytics/utils.py
--- a/kolibri/core/analytics/utils.py
+++ b/kolibri/core/analytics/utils.py
@@ -432,6 +432,7 @@ def perform_ping(started, server=DEFAULT_SERVER_URL):
         "instance_id": instance.id,
         "version": kolibri.__version__,
         "mode": conf.OPTIONS["Deployment"]["RUN_MODE"],
+        "project": conf.OPTIONS["Deployment"]["PROJECT"],
         "platform": instance.platform,
         "sysversion": instance.sysversion,
         "database_id": instance.database.id,
diff --git a/kolibri/utils/options.py b/kolibri/utils/options.py
--- a/kolibri/utils/options.py
+++ b/kolibri/utils/options.py
@@ -469,6 +469,14 @@ def url_prefix(value):
                 In case a SoUD connects to this server, the SoUD should use this interval to resync every user.
             """,
         },
+        "PROJECT": {
+            "type": "string",
+            "skip_blank": True,
+            "description": """
+                The custom identifier for a project. This is used to identify the project in the telemetry
+                data that is returned to our telemetry server.
+            """,
+        },
     },
     "Python": {
         "PICKLE_PROTOCOL": {


Test patch:
Repository: learningequality/kolibri

Issue:
Describe proper workflow for uploading and updating Perseus translations files
## Observed behavior
Extracting Perseus translatable strings is a murk-ish _affaire_, as the desired workflow is not completely obvious. We should document it properly.

## Errors and logs
…

## Expected behavior
…

## User-facing consequences
…

## Steps to reproduce
…

## Context
…



Discussion:

Resolving patch:
diff --git a/kolibri/plugins/perseus_viewer/__init__.py b/kolibri/plugins/perseus_viewer/__init__.py
new file mode 100644
diff --git a/kolibri/plugins/perseus_viewer/kolibri_plugin.py b/kolibri/plugins/perseus_viewer/kolibri_plugin.py
new file mode 100644
--- /dev/null
+++ b/kolibri/plugins/perseus_viewer/kolibri_plugin.py
@@ -0,0 +1,19 @@
+from __future__ import absolute_import
+from __future__ import print_function
+from __future__ import unicode_literals
+
+from le_utils.constants import format_presets
+
+from kolibri.core.content import hooks as content_hooks
+from kolibri.plugins import KolibriPluginBase
+from kolibri.plugins.hooks import register_hook
+
+
+class ExercisePerseusRenderPlugin(KolibriPluginBase):
+    pass
+
+
+@register_hook
+class ExercisePerseusRenderAsset(content_hooks.ContentRendererHook):
+    bundle_id = "main"
+    presets = (format_presets.EXERCISE,)
diff --git a/kolibri/utils/build_config/default_plugins.py b/kolibri/utils/build_config/default_plugins.py
--- a/kolibri/utils/build_config/default_plugins.py
+++ b/kolibri/utils/build_config/default_plugins.py
@@ -8,8 +8,8 @@
     "kolibri.plugins.learn",
     "kolibri.plugins.media_player",
     "kolibri.plugins.pdf_viewer",
+    "kolibri.plugins.perseus_viewer",
     "kolibri.plugins.setup_wizard",
     "kolibri.plugins.slideshow_viewer",
     "kolibri.plugins.user",
-    "kolibri_exercise_perseus_plugin",
 ]


Test patch:
Repository: learningequality/kolibri

Issue:
Add upgrade task to delete the static folder upon upgrade to 0.15
In 0.15 we removed the need to have a static folder in KOLIBRI_HOME.
Let's clean up an existing static folder to be mindful of user storage.


Discussion:

Resolving patch:
diff --git a/kolibri/core/device/upgrade.py b/kolibri/core/device/upgrade.py
new file mode 100644
--- /dev/null
+++ b/kolibri/core/device/upgrade.py
@@ -0,0 +1,15 @@
+"""
+A file to contain specific logic to handle version upgrades in Kolibri.
+"""
+from shutil import rmtree
+
+from django.conf import settings
+
+from kolibri.core.upgrade import version_upgrade
+
+
+# Before 0.15 we copied static files to the KOLIBRI_HOME directory.
+# After 0.15 we read them directly from their source directories.
+@version_upgrade(old_version="<0.15.0")
+def clear_static_dir():
+    rmtree(settings.STATIC_ROOT, ignore_errors=True)


Test patch:
Repository: learningequality/kolibri

Issue:
Brand new Setup Wizard infinitely redirects
<!--
Instructions:
 * Fill out the sections below, replace …'s with information about your issue
 * Use the 'preview' function above this text box to verify formatting before submitting
-->

## Observed behavior
<!--
Description of the behavior that was observed, including screenshots or other references when applicable
-->

On 0.15 branch, with a fresh KOLIBRI_HOME folder, going to the server causes infinite redirects and does not allow the user to proceed with the Setup Wizard (it doesn't load at all - tries to request the page like 10 times and then gives up).

## Context
<!--
Tell us about your environment, including:
 * Kolibri version
 * Operating system
 * Browser
-->

Kolibri: release-v0.15.x branch
OS: Fedora
Browser: Chrome



Discussion:
@radinamatic have you seen this on testing of current release-v0.15.x (without https://github.com/learningequality/kolibri/pull/8185)?
Can't say that I have @rtibbles 🤔 

I just started the server on my dev VM from the `release-v0.15.x` branch without the `.kolibri` folder, and the setup wizard loads without issues.

![LEDev2104 (start)  Running  - Oracle VM VirtualBox_019](https://user-images.githubusercontent.com/1457929/129224775-39305f7d-664c-4a15-b737-85490c0efeca.png)

Both @pcenov and I have been doing periodic critical workflows tests on the assets built from the 0.15 branch, and we never had issues with the setup wizard.
@nucleogenesis anything specific about your dev setup that might be making this appear for you?

Resolving patch:
diff --git a/kolibri/core/device/middleware.py b/kolibri/core/device/middleware.py
--- a/kolibri/core/device/middleware.py
+++ b/kolibri/core/device/middleware.py
@@ -89,7 +89,11 @@ def __init__(self, get_response):
         self.get_response = get_response
 
     def process_exception(self, request, exception):
-        if isinstance(exception, DeviceNotProvisioned) and SetupHook.provision_url():
+        if (
+            isinstance(exception, DeviceNotProvisioned)
+            and SetupHook.provision_url()
+            and not request.path.startswith(SetupHook.provision_url())
+        ):
             return redirect(SetupHook.provision_url())
         return None
 
diff --git a/kolibri/core/kolibri_plugin.py b/kolibri/core/kolibri_plugin.py
--- a/kolibri/core/kolibri_plugin.py
+++ b/kolibri/core/kolibri_plugin.py
@@ -128,7 +128,9 @@ def plugin_data(self):
             "languageGlobals": self.language_globals(),
             "oidcProviderEnabled": OIDCProviderHook.is_enabled(),
             "kolibriTheme": ThemeHook.get_theme(),
-            "isSubsetOfUsersDevice": get_device_setting("subset_of_users_device"),
+            "isSubsetOfUsersDevice": get_device_setting(
+                "subset_of_users_device", False
+            ),
         }
 
     def language_globals(self):


Test patch:
Repository: learningequality/kolibri

Issue:
Unsupported browser template does not use theme styling
### Observed behavior
The unsupported browser template uses static css in the template - mostly this is fine, but the colours used in there are Kolibri default theme colours, not those drawn from the currently loaded theme plugin.

### Expected behavior
Should use the themes defined by the currently active theme hook.

### User-facing consequences
Custom themes might look odd for unsupported browsers.



Discussion:

Resolving patch:
diff --git a/kolibri/core/views.py b/kolibri/core/views.py
--- a/kolibri/core/views.py
+++ b/kolibri/core/views.py
@@ -29,6 +29,10 @@
 from kolibri.core.device.utils import device_provisioned
 from kolibri.core.hooks import LogoutRedirectHook
 from kolibri.core.hooks import RoleBasedRedirectHook
+from kolibri.core.theme_hook import BRAND_COLORS
+from kolibri.core.theme_hook import COLOR_V400
+from kolibri.core.theme_hook import PRIMARY
+from kolibri.core.theme_hook import ThemeHook
 
 
 # Modified from django.views.i18n
@@ -166,6 +170,16 @@ def get(self, request):
 class UnsupportedBrowserView(TemplateView):
     template_name = "kolibri/unsupported_browser.html"
 
+    def get_context_data(self, **kwargs):
+        context = super(UnsupportedBrowserView, self).get_context_data(**kwargs)
+        context["brand_primary_v400"] = (
+            ThemeHook.get_theme()
+            .get(BRAND_COLORS, {})
+            .get(PRIMARY, {})
+            .get(COLOR_V400, "purple")
+        )
+        return context
+
 
 class StatusCheckView(View):
     def get(self, request):


Test patch:
Repository: learningequality/kolibri

Issue:
password field is always shown during user import

## Observed behavior

The password field is always shown:

![image](https://user-images.githubusercontent.com/2367265/132043887-11bc34e9-b6d4-4558-b947-d09c1f26b4dd.png)

## Expected behavior

User should only be prompted to enter a password when necessary

## User-facing consequences

Unclear how to import learners from a device that has password-free login enabled. Leaving it blank seems to work, but this is a bit confusing.


## Context

latest develop




Discussion:

Resolving patch:
diff --git a/kolibri/core/auth/serializers.py b/kolibri/core/auth/serializers.py
--- a/kolibri/core/auth/serializers.py
+++ b/kolibri/core/auth/serializers.py
@@ -108,9 +108,14 @@ class Meta:
 
 
 class PublicFacilitySerializer(serializers.ModelSerializer):
+    learner_can_login_with_no_password = serializers.SerializerMethodField()
+
+    def get_learner_can_login_with_no_password(self, instance):
+        return instance.dataset.learner_can_login_with_no_password
+
     class Meta:
         model = Facility
-        fields = ("id", "dataset", "name")
+        fields = ("id", "dataset", "name", "learner_can_login_with_no_password")
 
 
 class PublicFacilityUserSerializer(serializers.ModelSerializer):


Test patch:
Repository: learningequality/kolibri

Issue:
Learner - Cannot change username and full name
## Observed behavior
The fields for editing a learner's username and full name are disabled and I cannot change them.

## Expected behavior
It should be possible for a learner to edit their username and full name if the options are enabled in Facility settings

## Steps to reproduce the issue
1. Install the following [build.](https://github.com/learningequality/kolibri/releases/tag/v0.15.0-beta2)
2. Create a facility with enabled options for 'Allow learners to edit their username' and 'Allow learners to edit their full name' in Facility settings
3. Create a Learner user
4. Sign in with the Learner, go to Profile and select the Edit button.
5. Attempt to edit the Full name and Username fields.

## Additional information
![2021-11-15_15-21-13](https://user-images.githubusercontent.com/79847249/141820419-c2511e0e-0ff2-475a-a500-9fdedfafa76a.png)

Logs: 
[logs.zip](https://github.com/learningequality/kolibri/files/7540298/logs.zip)

## Usage Details
 - OS: Windows 10
 - Browser: Chrome


Discussion:
@radinamatic 

Resolving patch:
diff --git a/kolibri/core/auth/middleware.py b/kolibri/core/auth/middleware.py
--- a/kolibri/core/auth/middleware.py
+++ b/kolibri/core/auth/middleware.py
@@ -6,6 +6,7 @@
 from django.contrib.sessions.middleware import SessionMiddleware
 from django.core.cache import cache
 from django.core.exceptions import ImproperlyConfigured
+from django.db.models.signals import post_save
 from django.utils.functional import SimpleLazyObject
 
 
@@ -41,11 +42,14 @@ def get_anonymous_user_model():
         )
 
 
+USER_SESSION_CACHE_KEY = "USER_BY_SESSION_CACHE_{}"
+
+
 def _get_user(request):
     if not hasattr(request, "_cached_user"):
         try:
             user_id = _get_user_session_key(request)
-            USER_CACHE_KEY = "USER_BY_SESSION_CACHE_{}".format(user_id)
+            USER_CACHE_KEY = USER_SESSION_CACHE_KEY.format(user_id)
             user = cache.get(USER_CACHE_KEY)
             if not user:
                 user = get_user(request)
@@ -60,6 +64,14 @@ def _get_user(request):
     return request._cached_user
 
 
+def clear_user_cache(sender, instance, created, **kwargs):
+    if not created:
+        cache.delete(USER_SESSION_CACHE_KEY.format(instance.id))
+
+
+post_save.connect(clear_user_cache, sender=settings.AUTH_USER_MODEL)
+
+
 class CustomAuthenticationMiddleware(AuthenticationMiddleware):
     """
     Adaptation of Django's ``account.middleware.AuthenticationMiddleware``


Test patch:
Repository: learningequality/kolibri

Issue:
The Select all button in Create new quiz is not functioning (when selecting from bookmarked items)
## Observed behavior
The Select all button in Create new quiz is not functioning.

## Expected behavior
The Select all option should be fully functional and identical to the same option when selecting from non-bookmarked channels.

## Steps to reproduce the issue
1. Install the following [build.](https://github.com/learningequality/kolibri/releases/tag/v0.15.0-beta2)
2. As a Coach with bookmarked items go to Class>Plan
3. Click New Quiz>Create new quiz and click the Bookmarks link
4. While at Create new quiz check the Select all checkbox

## Additional information

![2021-11-17_15-35-50](https://user-images.githubusercontent.com/79847249/142243492-e2a9e369-fea0-4568-be5a-1312a3969b2d.png)

## Usage Details
 - OS: Windows 10
 - Browser: Chrome


Discussion:
@radinamatic 

Resolving patch:
diff --git a/kolibri/core/content/api.py b/kolibri/core/content/api.py
--- a/kolibri/core/content/api.py
+++ b/kolibri/core/content/api.py
@@ -10,6 +10,7 @@
 from django.db.models import Exists
 from django.db.models import OuterRef
 from django.db.models import Q
+from django.db.models import Subquery
 from django.db.models import Sum
 from django.db.models.aggregates import Count
 from django.http import Http404
@@ -1199,6 +1200,40 @@ def list(self, request, **kwargs):
         )
 
 
+class BookmarkFilter(FilterSet):
+    available = BooleanFilter(
+        method="filter_available",
+    )
+    kind = CharFilter(
+        method="filter_kind",
+    )
+
+    class Meta:
+        model = Bookmark
+        fields = ("kind",)
+
+    def filter_kind(self, queryset, name, value):
+        queryset = queryset.annotate(
+            kind=Subquery(
+                models.ContentNode.objects.filter(
+                    id=OuterRef("contentnode_id"),
+                ).values_list("kind", flat=True)[:1]
+            )
+        )
+
+        return queryset.filter(kind=value)
+
+    def filter_available(self, queryset, name, value):
+        queryset = queryset.annotate(
+            available=Subquery(
+                models.ContentNode.objects.filter(
+                    available=True,
+                    id=OuterRef("contentnode_id"),
+                ).values_list("available", flat=True)[:1]
+            )
+        )
+
+
 class ContentNodeBookmarksViewset(
     BaseContentNodeMixin, BaseValuesViewset, ListModelMixin
 ):
@@ -1207,7 +1242,7 @@ class ContentNodeBookmarksViewset(
         KolibriAuthPermissionsFilter,
         DjangoFilterBackend,
     )
-    filter_class = None
+    filter_class = BookmarkFilter
     pagination_class = ValuesViewsetLimitOffsetPagination
 
     def get_queryset(self):


Test patch: