Repository: learningequality/kolibri

Issue:
Errors while interacting with resources in a lesson on Learn-only device

## Observed behavior
Replicable on 0.15.0b3 installed on both Windows 7 & 10 with learn-only device on Android 9.0 phone. Errors appear in seemingly random fashion on different resources at different moments. Learner device appears to be syncing correctly with the server, and progress (when achieved) is registered on Coach page.

1 | 2 | 3
--- | --- | ---
![DevTools - localhost:8080-es-es-learn---topics-c-d753aaf3ac794cf59b0169df72964dbe?lessonId=b44f7ef78626b7041476861b64f1c7f1 classId=1b21fb43599f223b4885e95b61aa2ef3 last=LESSON_PLAYLIST_002](https://user-images.githubusercontent.com/1457929/145158636-00e2e472-028d-471a-8c64-1bf2ef039414.png) | ![DevTools - localhost:8080-pt-mz-learn---home_001](https://user-images.githubusercontent.com/1457929/145158660-b5b360d9-b46c-4280-9128-292ed7969670.png) | ![DevTools - localhost:8080-fr-fr-learn-_001](https://user-images.githubusercontent.com/1457929/145158678-3c095b86-9f8b-45a9-b13b-5c859dccef11.png)


## Errors and logs

```
{
  "data": "<h1>Server Error (500)</h1>",
  "status": 500,
  "statusText": "Internal Server Error",
  "headers": {
    "content-length": "27",
    "content-type": "text/html",
    "date": "Wed, 08 Dec 2021 01:20:57 GMT",
    "server": "Cheroot/unknown",
    "vary": "Cookie",
    "x-frame-options": "SAMEORIGIN"
  },
  "config": {
    "url": "/api/logger/trackprogress/c05a4300939687321e2195d8972941f7/",
    "method": "put",
    "data": "{\"time_spent_delta\":123.604}",
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json;charset=utf-8",
      "X-Requested-With": "XMLHttpRequest",
      "X-CSRFToken": "WKza1d9ZwO52LYZNL8Ve0aBq1GsoKKio2ISgQZA2cHI4mNhLu0Ux5LlCG3ZfpCpd"
    },
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "kolibri_csrftoken",
    "xsrfHeaderName": "X-CSRFToken",
    "maxContentLength": -1
  },
  "request": {}
}
```

[db-and-logs.zip](https://github.com/learningequality/kolibri/files/7673793/db-and-logs.zip)

[full home folder](https://drive.google.com/file/d/1qgC6Ovfm91RzdgoFA4TEJNuWeZi-NS5L/view?usp=sharing) (Windows 7, 370MB)

## Expected behavior
...

## User-facing consequences
...

## Steps to reproduce
…

## Context
 * Kolibri version: 0.15.0b3
 * Operating system:  Windows 7 & 10 with learn-only device on Android 9.0 phone
 * Browser: Firefox

cc @marcellamaki @rtibbles 


Discussion:
Possibly related to https://github.com/learningequality/kolibri/issues/8848...?
@radinamatic looking at the error trace it seems to be the same that was fixed in #8858 
Can you try to reproduce it using https://buildkite.com/learningequality/kolibri-windows/builds/2721 ?
It's weird because that debug info is showing a 500, but I don't see any in the `kolibri.txt`. There can be a slight issue sometimes where the API error page appears for non-api related errors that occur during handling.
Nope, still extant when installing the [most recent assets](https://buildkite.com/learningequality/kolibri-python-package/builds/4505) from the 0.15 branch just now. Same Windows/Android devices, same error again 😞 

1 | 2
--- | ---
![DevTools - localhost:8080-en-learn---topics-c-fa98fe1787ac494dbe882bbe0944fbea?lessonId=8503220b0ea56e24bc79d12db2a17e91 classId=f5d83f321d06e190a45e567bce1facb4 last=LESSON_PLAYLIST_001](https://user-images.githubusercontent.com/1457929/145261115-b4080be6-e577-451d-aa8f-be6d6f93f0a6.png) | ![DevTools - localhost:8080-en-learn---topics-c-03be2208f89c4c1fb9fc775a26ef1a60?lessonId=8503220b0ea56e24bc79d12db2a17e91 classId=f5d83f321d06e190a45e567bce1facb4 last=LESSON_PLAYLIST_001](https://user-images.githubusercontent.com/1457929/145261133-06fc8311-a3aa-40b4-9d53-8441fb83f2d6.png)




> Nope, still extant when installing the [most recent assets](https://buildkite.com/learningequality/kolibri-python-package/builds/4505) from the 0.15 branch just now. Same Windows/Android devices, same error again disappointed
> 

@radinamatic is the windows test done in the same vm you shared with me last week?
Any help you can provide to reproduce it is very welcome


We exchanged Ubuntu VMs last week, for `kolibri-server`, right? 
I can upload the full appliance for this Windows VM, so you can try it on your side...
Can you try getting the logs again? Definitely a 500 error there, so would be good to see what the logs say on that.
DB and logs for this latest install:
[db-and-logs.zip](https://github.com/learningequality/kolibri/files/7678119/db-and-logs.zip)

For the first screenshot I bet it the same that was fixed in #8858
For the rest, as @rtibbles said, it's a 500 error. In the `debug.txt `file I see a lot of 
```
  File "c:\python36\lib\logging\__init__.py", line 997, in emit
    stream.write(self.terminator)
OSError: [WinError 6] The handle is invalid
```

errors when `logging` is used, I haven't seen these errors before, but I don't use Windows, some quick googling says it could be due to a wrong `stdin` configuration
> We exchanged Ubuntu VMs last week, for `kolibri-server`, right? I can upload the full appliance for this Windows VM, so you can try it on your side...

ah, yes, it was the Ubuntu VM. It would be great if you can share the windows vm too, so we can replicate in your same environment
Hm, got a new one for you, same setup:

![DevTools - localhost:8080-en-learn---topics-c-a1f54495517144ed9cbd4635e7e2c03c?lessonId=17ea5db4461bb5dd4590c74d0d702677 classId=f5d83f321d06e190a45e567bce1facb4 last=LESSON_PLAYLIST_001](https://user-images.githubusercontent.com/1457929/145265812-2b68a687-8efa-4d46-91f7-d77a93095c30.png)

Yeah, there's definitely an issue with logging on windows.
This seems to be caused by a bug in the DatabaseLocked handling middleware I had implemented:
```
TypeError at /api/logger/trackprogress/08c9d98e8407f237005b22fbabe42837/
__init__() got an unexpected keyword argument 'headers'

Request Method: PUT
Request URL: http://127.0.0.1:8000/api/logger/trackprogress/08c9d98e8407f237005b22fbabe42837/
Django Version: 1.11.29
Python Executable: /home/richard/.virtualenvs/kolibripy3/bin/python
Python Version: 3.6.14
Python Path: ['/home/richard/github/kolibri/kolibri/dist', '/home/richard/.virtualenvs/kolibripy3/bin', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages', '/home/richard/github/kolibri']
Server time: Thu, 9 Dec 2021 12:20:30 -0800
Installed Applications:
['kolibri.core',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django_filters',
 'kolibri.core.auth.apps.KolibriAuthConfig',
 'kolibri.core.bookmarks',
 'kolibri.core.content',
 'kolibri.core.logger',
 'kolibri.core.notifications.apps.KolibriNotificationsConfig',
 'kolibri.core.tasks.apps.KolibriTasksConfig',
 'kolibri.core.deviceadmin',
 'kolibri.core.webpack',
 'kolibri.core.exams',
 'kolibri.core.device',
 'kolibri.core.discovery',
 'kolibri.core.lessons',
 'kolibri.core.analytics',
 'rest_framework',
 'django_js_reverse',
 'jsonfield',
 'morango',
 <AppConfig: kolibri.plugins.setup_wizard>,
 <AppConfig: kolibri.plugins.device>,
 <AppConfig: kolibri.plugins.user_auth>,
 <AppConfig: kolibri.plugins.facility>,
 <AppConfig: kolibri.plugins.learn>,
 <AppConfig: kolibri.plugins.slideshow_viewer>,
 <AppConfig: kolibri.plugins.perseus_viewer>,
 <AppConfig: kolibri.plugins.pdf_viewer>,
 <AppConfig: kolibri.plugins.media_player>,
 <AppConfig: kolibri.plugins.default_theme>,
 <AppConfig: kolibri.plugins.coach>,
 <AppConfig: kolibri.plugins.epub_viewer>,
 <AppConfig: kolibri.plugins.html5_viewer>,
 <AppConfig: kolibri.plugins.user_profile>,
 'drf_yasg']
Installed Middleware:
['kolibri.core.analytics.middleware.cherrypy_access_log_middleware',
 'kolibri.core.device.middleware.ProvisioningErrorHandler',
 'kolibri.core.device.middleware.DatabaseBusyErrorHandler',
 'django.middleware.cache.UpdateCacheMiddleware',
 'kolibri.core.analytics.middleware.MetricsMiddleware',
 'kolibri.core.auth.middleware.KolibriSessionMiddleware',
 'kolibri.core.device.middleware.KolibriLocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'kolibri.core.auth.middleware.CustomAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'django.middleware.cache.FetchFromCacheMiddleware',
 'kolibri.core.webpack.middleware.WebpackErrorHandler',
 'kolibri.core.auth.middleware.XhrPreventLoginPromptMiddleware']


Traceback:  

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py" in execute
  328.         return Database.Cursor.execute(self, query, params)
  
    
      The above exception (database is locked) was the direct cause of the following exception:
    
  

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/views/decorators/csrf.py" in wrapped_view
  58.         return view_func(*args, **kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/rest_framework/viewsets.py" in view
  116.             return self.dispatch(request, *args, **kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
  495.             response = self.handle_exception(exc)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/rest_framework/views.py" in handle_exception
  455.             self.raise_uncaught_exception(exc)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
  492.             response = handler(request, *args, **kwargs)

File "/home/richard/github/kolibri/kolibri/core/logger/api.py" in update
  816.                 pk, request.user, end_timestamp, validated_data

File "/home/richard/github/kolibri/kolibri/core/logger/api.py" in _update_session
  759.         sessionlog.save(update_fields=update_fields)

File "/home/richard/github/kolibri/kolibri/core/logger/models.py" in save
  130.         super(ContentSessionLog, self).save(*args, **kwargs)

File "/home/richard/github/kolibri/kolibri/core/auth/models.py" in save
  285.         super(AbstractFacilityDataModel, self).save(*args, **kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/morango/models/core.py" in save
  733.         super(SyncableModel, self).save(*args, **kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/morango/models/fields/uuids.py" in save
  125.         super(UUIDModelMixin, self).save(*args, **kwargs)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/base.py" in save
  808.                        force_update=force_update, update_fields=update_fields)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/base.py" in save_base
  838.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/base.py" in _save_table
  905.                                       forced_update)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/base.py" in _do_update
  955.         return filtered._update(values) > 0

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/query.py" in _update
  667.         return query.get_compiler(self.db).execute_sql(CURSOR)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
  1204.         cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
  899.             raise original_exception

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
  889.             cursor.execute(sql, params)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
  79.             return super(CursorDebugWrapper, self).execute(sql, params)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/utils.py" in __exit__
  94.                 six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/utils/six.py" in reraise
  685.             raise value.with_traceback(tb)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py" in execute
  328.         return Database.Cursor.execute(self, query, params)
  
    
      During handling of the above exception (database is locked), another exception occurred:
    
  

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/core/handlers/base.py" in process_exception_by_middleware
  227.             response = middleware_method(request, exception)

File "/home/richard/github/kolibri/kolibri/core/device/middleware.py" in process_exception
  134.             headers={"Retry-After": 10},

File "/home/richard/.virtualenvs/kolibripy3/lib/python3.6/site-packages/django/http/response.py" in __init__
  301.         super(HttpResponse, self).__init__(*args, **kwargs)

Exception Type: TypeError at /api/logger/trackprogress/08c9d98e8407f237005b22fbabe42837/
Exception Value: __init__() got an unexpected keyword argument 'headers'
```

Resolving patch:
diff --git a/kolibri/core/device/middleware.py b/kolibri/core/device/middleware.py
--- a/kolibri/core/device/middleware.py
+++ b/kolibri/core/device/middleware.py
@@ -128,11 +128,12 @@ def process_exception(self, request, exception):
             return None
         # Return a 503 response with a Retry-After of 10 seconds. In future we may be able to customize this value
         # based on what is currently happening on the server.
-        return HttpResponse(
+        response = HttpResponse(
             "Database is not available for write operations",
             status=503,
-            headers={"Retry-After": 10},
         )
+        response["Retry-After"] = 10
+        return response
 
     def __call__(self, request):
         return self.get_response(request)


Test patch: