Repository: learningequality/kolibri

Issue:
<High>The videos are not loading on a server updated from 0.4.5 to 0.5.0beta8.
## Summary
The videos are not loading and we ar getting the error "Kolibri is unable to render this content"
Note:
This sever is being used at the Learning centre here in Pune. We will be testing/using the latest and stable pex file. We would be raising issues and and when we find it.
Also I reimported one of the channels and fund the issue not resolved.  

**Following are the Channels in the server and videos don't load on all theses channels:**
8fc515793aad4e52b490b03c48ad21de
797ff4dfe6fd473e91b68cfb8ec3a44e
095baa553e1547c19be273e96f591ff9
3f6e54734f45445bb3e597a408e95202
197934f144305350b5820c7c4dd8e194

## System information
 - Version: Kolibri 0.5.0beta8
 - Operating system: Ubuntu 14.04LTS
 - Browser: Chome

## How to reproduce
1. Upgrade the server from 0.4.5 to 0.5.0beta8
2. The videos that were working will not load anymore.

## Screenshots
![screenshot from 2017-08-22 18_14_04](https://user-images.githubusercontent.com/13453618/29575340-c96dfda6-8781-11e7-9c5e-fd7ec52f4907.png)
![screenshot from 2017-08-22 21_27_00](https://user-images.githubusercontent.com/13453618/29575342-c9c93216-8781-11e7-8832-9fc75e421823.png)


## Real-life consequences (anything community should be aware of, for instance how it affects your deployment)
For Nalanda's context the updation from 0.4.5 to any future releases will be a biggest challenge.


Discussion:
Hi @harishbalachandran a little surprised that this is happening, as we had explicitly included a fix to prevent this from happening, but we can investigate further.

Can you detail exactly how you did the update process?
We always run the platform using the command sudo ./Kolibri-version.pex start. 

In this case I deleted the older version 0.4.5 and used the latest pex file and ran sudo ./Kolibri-0.5.0beta8.pex start 
OK, we'll test and try to replicate. Thanks!
Here is the kolibri_settings.json file after replicating this issue:

`{
  "AUTO_SEARCH_PLUGINS": [], 
  "FIRST_RUN": false, 
  "INSTALLED_APPS": [
    "kolibri.plugins.management", 
    "kolibri.plugins.learn", 
    "kolibri.plugins.document_pdf_render", 
    "kolibri.plugins.html5_app_renderer", 
    "kolibri.plugins.setup_wizard", 
    "kolibri.plugins.coach", 
    "kolibri.plugins.user", 
    "kolibri_exercise_perseus_plugin"
  ]
}`
@benjaoming after investigation by @lyw07, it looks like while we are getting rid of the old video and audio renderer plugins, we are not adding the new content renderer plugin to upgrading installations.

Would you have time to make a quick PR to tackle this on 0.5.x?

Resolving patch:
diff --git a/kolibri/utils/cli.py b/kolibri/utils/cli.py
--- a/kolibri/utils/cli.py
+++ b/kolibri/utils/cli.py
@@ -39,7 +39,7 @@
 # ...we need to (re)move it /benjaoming
 # Force python2 to interpret every string as unicode.
 if sys.version[0] == '2':
-    reload(sys)
+    reload(sys)  # noqa
     sys.setdefaultencoding('utf8')
 
 USAGE = """
@@ -131,17 +131,28 @@ def initialize(debug=False):
     :param: debug: Tells initialization to setup logging etc.
     """
 
-    # TODO: We'll move this to a more deliberate location whereby we can
-    # ensure that some parts of kolibri can run without the whole django stack
-    django.setup()
+    if not os.path.isfile(VERSION_FILE):
+        django.setup()
 
-    setup_logging(debug=debug)
+        setup_logging(debug=debug)
 
-    if not os.path.isfile(VERSION_FILE):
         _first_run()
     else:
+        # Do this here so that we can fix any issues with our configuration file before
+        # we attempt to setup django.
+        from kolibri.utils.conf import autoremove_unavailable_plugins, enable_default_plugins
+        autoremove_unavailable_plugins()
+
         version = open(VERSION_FILE, "r").read()
-        if kolibri.__version__ != version.strip():
+        change_version = kolibri.__version__ != version.strip()
+        if change_version:
+            enable_default_plugins()
+
+        django.setup()
+
+        setup_logging(debug=debug)
+
+        if change_version:
             logger.info(
                 "Version was {old}, new version: {new}".format(
                     old=version,
@@ -200,6 +211,10 @@ def update():
 
     logger.info("Running update routines for new version...")
 
+    # Need to do this here, before we run any Django management commands that
+    # import settings. Otherwise the updated configuration will not be used
+    # during this runtime.
+
     call_command("collectstatic", interactive=False)
 
     from kolibri.core.settings import SKIP_AUTO_DATABASE_MIGRATION
diff --git a/kolibri/utils/compat.py b/kolibri/utils/compat.py
--- a/kolibri/utils/compat.py
+++ b/kolibri/utils/compat.py
@@ -14,11 +14,7 @@ def module_exists(module_path):
     if sys.version_info >= (3, 4):
         from importlib.util import find_spec
         try:
-            if "." in module_path:
-                find_spec(module_path)
-                return True
-            else:
-                return find_spec(module_path) is not None
+            return find_spec(module_path) is not None
         except ImportError:
             return False
     elif sys.version_info < (3,):
diff --git a/kolibri/utils/conf.py b/kolibri/utils/conf.py
--- a/kolibri/utils/conf.py
+++ b/kolibri/utils/conf.py
@@ -20,6 +20,7 @@
 import json
 import logging
 import os
+
 from kolibri.utils.compat import module_exists
 
 logger = logging.getLogger(__name__)
@@ -38,8 +39,7 @@
 #: Set defaults before updating the dict
 config = {}
 
-#: Everything in this list is added to django.conf.settings.INSTALLED_APPS
-config['INSTALLED_APPS'] = [
+DEFAULT_PLUGINS = [
     # Note from Devon -
     # Temporarily adding these here to get things working for most devs.
     # It's not clear to me where the correct place to add them is.
@@ -54,6 +54,9 @@
     "kolibri_exercise_perseus_plugin"
 ]
 
+#: Everything in this list is added to django.conf.settings.INSTALLED_APPS
+config['INSTALLED_APPS'] = DEFAULT_PLUGINS
+
 #: Well-known plugin names that are automatically searched for and enabled on
 #: first-run.
 config['AUTO_SEARCH_PLUGINS'] = []
@@ -96,7 +99,8 @@ def autoremove_unavailable_plugins():
     """
     global config
     changed = False
-    for module_path in config['INSTALLED_APPS']:
+    # Iterate over a copy of the list so that it is not modified during the loop
+    for module_path in config['INSTALLED_APPS'][:]:
         if not module_exists(module_path):
             config['INSTALLED_APPS'].remove(module_path)
             logger.error(
@@ -106,9 +110,30 @@ def autoremove_unavailable_plugins():
                 ).format(mod=module_path)
             )
             changed = True
-
     if changed:
         save()
 
 
-autoremove_unavailable_plugins()
+def enable_default_plugins():
+    """
+    Enable new plugins that have been added between versions
+    This will have the undesired side effect of reactivating
+    default plugins that have been explicitly disabled by a user.
+    However, until we add disabled plugins to a blacklist, this is
+    unavoidable.
+    """
+    global config
+    changed = False
+    for module_path in DEFAULT_PLUGINS:
+        if module_path not in config['INSTALLED_APPS']:
+            config['INSTALLED_APPS'].append(module_path)
+            logger.warning(
+                (
+                    "Default plugin {mod} not found in configuration. To re-disable it, run:\n"
+                    "   $ kolibri plugin {mod} disable"
+                ).format(mod=module_path)
+            )
+            changed = True
+
+    if changed:
+        save()


Test patch: