Repository: learningequality/kolibri
Issue: Cannot start Kolibri after upgrading Kolibri 0.14.7 to Kolibri 0.15.0 beta2

## Observed behavior
I'm getting a database error when I attempt to start Kolibri after an upgrade from the 0.14.7 version to 0.15.0 beta2

## Expected behavior
It should be possible to upgrade the 0.14 version of Kolibri to the 0.15 version

## Steps to reproduce the issue
1. Install the a 0.14.7 version of Kolibri from [here](https://learningequality.org/r/kolibri-windows-setup-latest).
2. Setup a facility, create classes, users, lesson etc and perform some user interactions such as completing a lesson or a quiz.
3. Upgrade to the 0.15 version by Installing the following [build.](https://github.com/learningequality/kolibri/releases/tag/v0.15.0-beta2).
4. Attempt to start the Kolibri server.

## Additional information

Console info when I execute the `kolibri start --foreground` command in Ubuntu:

```
le@osboxes:~$ kolibri start --foreground
INFO     Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO     Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
/usr/lib/python3/dist-packages/kolibri/dist/rest_framework/utils/serializer_helpers.py:107: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if value is None or value is '':
INFO     Running Kolibri with the following settings: kolibri.deployment.default.settings.base
INFO     Backed up database to: /home/le/.kolibri/backups/db-v0.14.7_2021-11-25_16-17-29.dump
INFO     Version was 0.14.7, new version: 0.15.0b2
INFO     Running update routines for new version...
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Operations to perform:
  Apply all migrations: admin, analytics, auth, bookmarks, content, contenttypes, device, discovery, exams, kolibriauth, lessons, logger, morango, notifications, sessions
Running migrations:
  No migrations to apply.
Installed 2 object(s) from 1 fixture(s)
Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kolibri/utils/cli.py", line 193, in invoke
    initialize(**get_initialize_params())
  File "/usr/lib/python3/dist-packages/kolibri/utils/main.py", line 268, in initialize
    update(version, kolibri.__version__)
  File "/usr/lib/python3/dist-packages/kolibri/utils/main.py", line 326, in update
    run_upgrades(old_version, new_version)
  File "/usr/lib/python3/dist-packages/kolibri/core/upgrade.py", line 156, in run_upgrades
    version_upgrade()
  File "/usr/lib/python3/dist-packages/kolibri/core/upgrade.py", line 51, in __call__
    return self.upgrade()
  File "/usr/lib/python3/dist-packages/kolibri/core/content/upgrade.py", line 306, in metadata_annotation_update
    get_all_contentnode_label_metadata()
  File "/usr/lib/python3/dist-packages/kolibri/core/content/utils/search.py", line 122, in get_all_contentnode_label_metadata
    return get_available_metadata_labels(
  File "/usr/lib/python3/dist-packages/kolibri/core/content/utils/search.py", line 86, in get_available_metadata_labels
    content_cache_key = ContentCacheKey.get_cache_key()
  File "/usr/lib/python3/dist-packages/kolibri/core/device/models.py", line 159, in get_cache_key
    cache.set(CONTENT_CACHE_KEY_CACHE_KEY, key, 5000)
  File "/usr/lib/python3/dist-packages/kolibri/dist/diskcache/djangocache.py", line 147, in set
    return self._cache.set(key, value, timeout, read, tag, retry)
  File "/usr/lib/python3/dist-packages/kolibri/dist/diskcache/fanout.py", line 97, in set
    return shard.set(key, value, expire, read, tag, retry)
  File "/usr/lib/python3/dist-packages/kolibri/dist/diskcache/core.py", line 836, in set
    self._row_update(rowid, now, columns)
  File "/usr/lib/python3/dist-packages/kolibri/dist/diskcache/core.py", line 860, in _row_update
    sql('UPDATE Cache SET'
sqlite3.DatabaseError: database disk image is malformed
```

Complete Windows 10 and Ubuntu folders with DBs and logs: https://drive.google.com/drive/folders/1ZWYLxZ24lghoBfYbyCU1b71YaMw5eJub?usp=sharing

## Usage Details
 - OS: Windows 10, Ubuntu 20
 - Browser: Chrome

Changed files:
kolibri/core/tasks/upgrade.py, kolibri/deployment/default/cache.py, kolibri/utils/server.py, kolibri/utils/main.py

diff --git a/kolibri/core/tasks/upgrade.py b/kolibri/core/tasks/upgrade.py
index 705038f363..9d9a3ba9fc 100644
--- a/kolibri/core/tasks/upgrade.py
+++ b/kolibri/core/tasks/upgrade.py
@@ -4,6 +4,7 @@ A file to contain specific logic to handle version upgrades in Kolibri.
 import logging
 
 from kolibri.core.tasks.main import job_storage
+from kolibri.core.tasks.main import scheduler
 from kolibri.core.upgrade import version_upgrade
 
 logger = logging.getLogger(__name__)
@@ -20,3 +21,4 @@ def drop_old_iceqube_tables():
     and let iceqube reinitialize the tables from scratch.
     """
     job_storage.recreate_tables()
+    scheduler.recreate_tables()
diff --git a/kolibri/deployment/default/cache.py b/kolibri/deployment/default/cache.py
index 6304536cb3..06b77e7ff1 100644
--- a/kolibri/deployment/default/cache.py
+++ b/kolibri/deployment/default/cache.py
@@ -1,10 +1,6 @@
 import copy
 import os
-import shutil
 import sys
-from sqlite3 import DatabaseError
-
-from diskcache import Cache
 
 from kolibri.utils.conf import KOLIBRI_HOME
 from kolibri.utils.conf import OPTIONS
@@ -16,23 +12,6 @@ pickle_protocol = OPTIONS["Python"]["PICKLE_PROTOCOL"]
 
 diskcache_location = os.path.join(KOLIBRI_HOME, "process_cache")
 
-
-def recreate_diskcache():
-    if cache_options["CACHE_BACKEND"] != "redis":
-        try:
-            diskcache_cache = Cache(
-                diskcache_location, disk_pickle_protocol=pickle_protocol
-            )
-        except DatabaseError:
-            shutil.rmtree(diskcache_location, ignore_errors=True)
-            os.mkdir(diskcache_location)
-            diskcache_cache = Cache(
-                diskcache_location, disk_pickle_protocol=pickle_protocol
-            )
-        diskcache_cache.clear()
-        diskcache_cache.close()
-
-
 # Default to LocMemCache, as it has the simplest configuration
 default_cache = {
     "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
@@ -48,6 +27,7 @@ default_cache = {
 process_cache = {
     "BACKEND": "diskcache.DjangoCache",
     "LOCATION": diskcache_location,
+    "TIMEOUT": cache_options["CACHE_TIMEOUT"],
     "SHARDS": CACHE_SHARDS,
     "OPTIONS": {
         "MAX_ENTRIES": cache_options["CACHE_MAX_ENTRIES"],
diff --git a/kolibri/utils/main.py b/kolibri/utils/main.py
index 6d839441de..308b60b11b 100644
--- a/kolibri/utils/main.py
+++ b/kolibri/utils/main.py
@@ -9,6 +9,7 @@ import sys
 from sqlite3 import DatabaseError as SQLite3DatabaseError
 
 import django
+from diskcache.fanout import FanoutCache
 from django.conf import settings
 from django.core.exceptions import ValidationError
 from django.core.management import call_command
@@ -23,6 +24,7 @@ from kolibri.core.deviceadmin.utils import get_backup_files
 from kolibri.core.tasks.main import import_tasks_module_from_django_apps
 from kolibri.core.upgrade import matches_version
 from kolibri.core.upgrade import run_upgrades
+from kolibri.core.utils.cache import process_cache
 from kolibri.deployment.default.sqlite_db_names import ADDITIONAL_SQLITE_DATABASES
 from kolibri.plugins.utils import autoremove_unavailable_plugins
 from kolibri.plugins.utils import check_plugin_config_file_location
@@ -207,6 +209,21 @@ def _upgrades_before_django_setup(updated, version):
             _copy_preseeded_db(db_name)
 
 
+def _post_django_initialization():
+    if OPTIONS["Cache"]["CACHE_BACKEND"] != "redis":
+        try:
+            process_cache.cull()
+        except SQLite3DatabaseError:
+            shutil.rmtree(process_cache.directory, ignore_errors=True)
+            os.mkdir(process_cache.directory)
+            process_cache._cache = FanoutCache(
+                process_cache.directory,
+                settings.CACHES["process_cache"]["SHARDS"],
+                settings.CACHES["process_cache"]["TIMEOUT"],
+                **settings.CACHES["process_cache"]["OPTIONS"]
+            )
+
+
 def _upgrades_after_django_setup(updated, version):
     # If device is not provisioned, attempt automatic provisioning
     if not device_provisioned() and OPTIONS["Paths"]["AUTOMATIC_PROVISION_FILE"]:
@@ -254,6 +271,8 @@ def initialize(
 
     _setup_django()
 
+    _post_django_initialization()
+
     if updated and not skip_update:
         conditional_backup(kolibri.__version__, version)
 
diff --git a/kolibri/utils/server.py b/kolibri/utils/server.py
index 5a7618a8de..ec1a8db573 100644
--- a/kolibri/utils/server.py
+++ b/kolibri/utils/server.py
@@ -238,11 +238,6 @@ class ServicesPlugin(SimplePlugin):
         self.bus = bus
         self.workers = None
 
-    def ENTER(self):
-        from kolibri.deployment.default.cache import recreate_diskcache
-
-        recreate_diskcache()
-
     def START(self):
         from kolibri.core.tasks.main import initialize_workers
         from kolibri.core.tasks.main import scheduler