Repository: learningequality/kolibri

Issue:
Halt Kolibri startup if Redis is unavailable when configured
<!--
Instructions:
 * Fill out the sections below, replace …'s with information about your issue
 * Use the 'preview' function above this text box to verify formatting before submitting
-->
This needs reproduced on 0.16 before proceeding. There is likely benefit to adding a dedicated connection test though.

## Observed behavior
<!--
Description of the behavior that was observed, including screenshots or other references when applicable
-->
If Kolibri is configured to use Redis as its cache backend, when it's unavailable we should properly halt Kolibri startup. This has been observed in BCK environments where the Redis service may not be ready when Kolibri starts, specifically when running the `kolibri services` command. The result is that Kolibri reaches a state where it's stuck still running, outputting messages like those below. Note: this was observed on 0.15, so some improvements may have 

## Errors and logs

```
Got an exception running <bound method Worker.check_jobs of <kolibri.core.tasks.worker.Worker object at 0x7fd6f475d2b0>>: cannot schedule new futures after interpreter shutdown
```

## Expected behavior
<!--
Description of what behavior was expected but did not occur
-->
In an [existing location](https://github.com/learningequality/kolibri/blob/2ff89f2df90f0b9db3757c38b93b5ea12cbc5ac7/kolibri/core/apps.py#L105) where we test Redis settings, we should raise a proper exception if the connection to Redis fails. Since we already have a dedicated check for Redis settings, adding this should be a small but dedicated improvement to stop Kolibri startup when this occurs.

## User-facing consequences
<!--
Implications and real-world consequences for learners, coaches, admins, and other users of the application
-->
Kolibri could get stuck in a running state

## Steps to reproduce
<!--
Precise steps that someone else can follow in order to see this behavior
-->
Run `kolibri services` with it configured to use Redis as its cache backend, but without redis running.

## Context
<!--
Tell us about your environment, including:
 * Kolibri version
 * Operating system
 * Browser
-->
0.15


Discussion:

Resolving patch:
diff --git a/kolibri/core/apps.py b/kolibri/core/apps.py
--- a/kolibri/core/apps.py
+++ b/kolibri/core/apps.py
@@ -10,7 +10,9 @@
 from django.db.utils import DatabaseError
 from django_filters.filters import UUIDFilter
 from django_filters.rest_framework.filterset import FilterSet
+from six import raise_from
 
+from kolibri.core.errors import RedisConnectionError
 from kolibri.core.sqlite.pragmas import CONNECTION_PRAGMAS
 from kolibri.core.sqlite.pragmas import START_PRAGMAS
 from kolibri.core.sqlite.utils import repair_sqlite_db
@@ -108,12 +110,15 @@ def activate_pragmas_on_start():
             cursor.execute(START_PRAGMAS)
             connection.close()
 
-    @staticmethod
-    def check_redis_settings():
+    @staticmethod  # noqa C901
+    def check_redis_settings():  # noqa C901
         """
         Check that Redis settings are sensible, and use the lower level Redis client to make updates
         if we are configured to do so, and if we should, otherwise make some logging noise.
         """
+
+        from redis.exceptions import ConnectionError
+
         if OPTIONS["Cache"]["CACHE_BACKEND"] != "redis":
             return
         config_maxmemory = OPTIONS["Cache"]["CACHE_REDIS_MAXMEMORY"]
@@ -161,6 +166,14 @@ def check_redis_settings():
                     "Problematic Redis settings detected, please see Redis configuration "
                     "documentation for details: https://redis.io/topics/config"
                 )
+
+        except ConnectionError as e:
+            logger.warning("Unable to connect to Redis: {}".format(str(e)))
+
+            raise_from(
+                RedisConnectionError("Unable to connect to Redis: {}".format(str(e))), e
+            )
+
         except Exception as e:
             logger.warning("Unable to check Redis settings")
             logger.warning(e)
diff --git a/kolibri/core/errors.py b/kolibri/core/errors.py
--- a/kolibri/core/errors.py
+++ b/kolibri/core/errors.py
@@ -17,3 +17,7 @@ class KolibriUpgradeError(KolibriError):
     """
 
     pass
+
+
+class RedisConnectionError(Exception):
+    pass


Test patch: