Repository: learningequality/kolibri

Issue:
add 'disabled' list of plugins alongside 'enabled' in kolibri_settings.json

### Observed behavior

Currently we have no way of distinguishing disabled plugins from newly created plugins.

### Expected behavior

* disabled plugins should remain disabled across upgrades
* newly created default plugins should automatically become enabled on upgrade

### User-facing consequences
<!--
Implications and real-world consequences for learners, coaches, admins, and other users of the application
-->

…

### Errors and logs
<!--
Relevant logs from:
 * the command line
 * ~/.kolibri/logs/kolibri.txt
 * the browser console

Please wrap errors in triple backticks for clean formatting like this:
```
01:10 info: something happened
01:12 error: something bad happened
```
-->

…

### Steps to reproduce
<!--
Precise steps that someone else can follow in order to see this behavior
-->

…

### Context
<!--
Tell us about your environment, including:
 * Kolibri version
 * Operating system
 * Browser
-->

…


Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/learningequality/kolibri/pull/5637?src=pr&el=h1) Report
> Merging [#5637](https://codecov.io/gh/learningequality/kolibri/pull/5637?src=pr&el=desc) into [release-v0.12.y](https://codecov.io/gh/learningequality/kolibri/commit/80fc694d7ab40cba571bc671a578c230718943ea?src=pr&el=desc) will **decrease** coverage by `0.11%`.
> The diff coverage is `74.01%`.

| [Impacted Files](https://codecov.io/gh/learningequality/kolibri/pull/5637?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [kolibri/deployment/default/settings/base.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS9kZXBsb3ltZW50L2RlZmF1bHQvc2V0dGluZ3MvYmFzZS5weQ==) | `75.3% <ø> (ø)` | :arrow_up: |
| [kolibri/plugins/base.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS9wbHVnaW5zL2Jhc2UucHk=) | `85.41% <50%> (ø)` | :arrow_up: |
| [kolibri/utils/conf.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS91dGlscy9jb25mLnB5) | `75.86% <70%> (-9.14%)` | :arrow_down: |
| [kolibri/plugins/utils.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS9wbHVnaW5zL3V0aWxzLnB5) | `77.55% <76.08%> (-22.45%)` | :arrow_down: |
| [kolibri/utils/cli.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS91dGlscy9jbGkucHk=) | `65.56% <88.23%> (-3.45%)` | :arrow_down: |
| [kolibri/core/logger/utils/handler.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS9jb3JlL2xvZ2dlci91dGlscy9oYW5kbGVyLnB5) | `70.21% <0%> (-21.28%)` | :arrow_down: |
| [kolibri/utils/version.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS91dGlscy92ZXJzaW9uLnB5) | `79.41% <0%> (-5.89%)` | :arrow_down: |
| [kolibri/utils/compat.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS91dGlscy9jb21wYXQucHk=) | `94.28% <0%> (-5.72%)` | :arrow_down: |
| [kolibri/core/discovery/utils/filesystem/posix.py](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree#diff-a29saWJyaS9jb3JlL2Rpc2NvdmVyeS91dGlscy9maWxlc3lzdGVtL3Bvc2l4LnB5) | `57.84% <0%> (-4.91%)` | :arrow_down: |
| ... and [1 more](https://codecov.io/gh/learningequality/kolibri/pull/5637/diff?src=pr&el=tree-more) | |

indirectlylit: pushing to 0.12.5 because this is mainly for internal dev experince

Patch:
diff --git a/kolibri/deployment/default/settings/base.py b/kolibri/deployment/default/settings/base.py
index 30d876985e..72b4f926e2 100644
--- a/kolibri/deployment/default/settings/base.py
+++ b/kolibri/deployment/default/settings/base.py
@@ -74,7 +74,7 @@ INSTALLED_APPS = [
     "django_js_reverse",
     "jsonfield",
     "morango",
-] + conf.config["INSTALLED_APPS"]
+] + conf.config.ACTIVE_PLUGINS
 
 # Add in the external plugins' locale paths. Our frontend messages depends
 # specifically on the value of LOCALE_PATHS to find its catalog files.
diff --git a/kolibri/plugins/base.py b/kolibri/plugins/base.py
index 26fecbeb3a..e7fcf0f5ba 100644
--- a/kolibri/plugins/base.py
+++ b/kolibri/plugins/base.py
@@ -60,8 +60,8 @@ class KolibriPluginBase(object):
         """Call this from your enable() method to have the plugin automatically
         added to Kolibri configuration"""
         module_path = cls._module_path()
-        if module_path not in config["INSTALLED_APPS"]:
-            config["INSTALLED_APPS"].append(module_path)
+        if module_path not in config.ACTIVE_PLUGINS:
+            config.enable_plugin(module_path)
         else:
             logger.warning("{} already enabled".format(module_path))
 
@@ -70,8 +70,8 @@ class KolibriPluginBase(object):
         """Call this from your enable() method to have the plugin automatically
         added to Kolibri configuration"""
         module_path = cls._module_path()
-        if module_path in config["INSTALLED_APPS"]:
-            config["INSTALLED_APPS"].remove(module_path)
+        if module_path in config.ACTIVE_PLUGINS:
+            config.disable_plugin(module_path)
         else:
             logger.warning("{} already disabled".format(module_path))
 
diff --git a/kolibri/plugins/utils.py b/kolibri/plugins/utils.py
index eb511b583d..f6e415fee2 100644
--- a/kolibri/plugins/utils.py
+++ b/kolibri/plugins/utils.py
@@ -1,5 +1,30 @@
+import importlib
+import logging
+
+from django.core.exceptions import AppRegistryNotReady
 from django.core.urlresolvers import reverse
 
+from kolibri.plugins.base import KolibriPluginBase
+from kolibri.utils.conf import config
+
+logger = logging.getLogger(__name__)
+
+
+class PluginDoesNotExist(Exception):
+    """
+    This exception is local to the CLI environment in case actions are performed
+    on a plugin that cannot be loaded.
+    """
+
+
+class PluginBaseLoadsApp(Exception):
+    """
+    An exception raised in case a kolibri_plugin.py results in loading of the
+    Django app stack.
+    """
+
+    pass
+
 
 def plugin_url(plugin_class, url_name):
     return reverse(
@@ -7,3 +32,88 @@ def plugin_url(plugin_class, url_name):
             namespace=plugin_class().url_namespace(), url_name=url_name
         )
     )
+
+
+def _is_plugin(obj):
+    return (
+        # Check that the object is an instance of type
+        # i.e. that it is a class definition, not an
+        # instantiated class.
+        # Failing to do this will result in the call to
+        # issubclass below blowing up.
+        isinstance(obj, type)
+        and obj is not KolibriPluginBase
+        and issubclass(obj, KolibriPluginBase)
+    )
+
+
+def get_kolibri_plugin(plugin_name):
+    """
+    Try to load kolibri_plugin from given plugin module identifier
+
+    :returns: A list of classes inheriting from KolibriPluginBase
+    """
+
+    plugin_classes = []
+
+    try:
+        # Exceptions are expected to be thrown from here.
+        plugin_module = importlib.import_module(plugin_name + ".kolibri_plugin")
+        # If no exception is thrown, use this to populate our plugin classes.
+        for obj in plugin_module.__dict__.values():
+            if _is_plugin(obj):
+                plugin_classes.append(obj)
+    except ImportError as e:
+        # Python 2: message, Python 3: msg
+        exc_message = getattr(e, "message", getattr(e, "msg", None))
+        if exc_message.startswith("No module named"):
+            msg = (
+                "Plugin '{}' does not seem to exist. Is it on the PYTHONPATH?"
+            ).format(plugin_name)
+            raise PluginDoesNotExist(msg)
+        else:
+            raise
+    except AppRegistryNotReady:
+        msg = (
+            "Plugin '{}' loads the Django app registry, which it isn't "
+            "allowed to do while enabling or disabling itself."
+        ).format(plugin_name)
+        raise PluginBaseLoadsApp(msg)
+
+    if not plugin_classes:
+        # There's no clear use case for a plugin without a KolibriPluginBase
+        # inheritor, for now just throw a warning
+        logger.warning(
+            "Plugin '{}' has no KolibriPluginBase defined".format(plugin_name)
+        )
+
+    return plugin_classes
+
+
+def enable_plugin(plugin_name):
+    plugin_classes = get_kolibri_plugin(plugin_name)
+    for klass in plugin_classes:
+        klass.enable()
+
+
+def disable_plugin(plugin_name):
+    try:
+        plugin_classes = get_kolibri_plugin(plugin_name)
+        for klass in plugin_classes:
+            klass.disable()
+    except PluginDoesNotExist as e:
+        logger.error(str(e))
+        logger.warning(
+            "Removing '{}' from configuration in a naive way.".format(plugin_name)
+        )
+        if plugin_name in config["INSTALLED_APPS"]:
+            config["INSTALLED_APPS"].remove(plugin_name)
+            logger.info("Removed '{}' from INSTALLED_APPS".format(plugin_name))
+        else:
+            logger.warning(
+                (
+                    "Could not find any matches for {} in INSTALLED_APPS".format(
+                        plugin_name
+                    )
+                )
+            )
diff --git a/kolibri/utils/cli.py b/kolibri/utils/cli.py
index a99fdea82a..d1596a6cb6 100644
--- a/kolibri/utils/cli.py
+++ b/kolibri/utils/cli.py
@@ -2,7 +2,6 @@ from __future__ import absolute_import
 from __future__ import print_function
 from __future__ import unicode_literals
 
-import importlib
 import logging
 import os
 import signal
@@ -10,7 +9,6 @@ import sys
 from sqlite3 import DatabaseError as SQLite3DatabaseError
 
 import django
-from django.core.exceptions import AppRegistryNotReady
 from django.core.management import call_command
 from django.db.utils import DatabaseError
 from docopt import docopt
@@ -29,7 +27,11 @@ from .sanity_checks import check_other_kolibri_running  # noqa
 from .sanity_checks import check_log_file_location  # noqa
 from .system import become_daemon  # noqa
 from kolibri.core.deviceadmin.utils import IncompatibleDatabase  # noqa
-from kolibri.utils import conf  # noqa
+from kolibri.plugins.utils import disable_plugin  # noqa
+from kolibri.plugins.utils import enable_plugin  # noqa
+from kolibri.utils.conf import config  # noqa
+from kolibri.utils.conf import KOLIBRI_HOME  # noqa
+from kolibri.utils.conf import OPTIONS  # noqa
 
 
 USAGE = """
@@ -108,29 +110,11 @@ Auto-generated usage instructions from ``kolibri -h``::
 logger = logging.getLogger(__name__)
 
 
-class PluginDoesNotExist(Exception):
-    """
-    This exception is local to the CLI environment in case actions are performed
-    on a plugin that cannot be loaded.
-    """
-
-
-class PluginBaseLoadsApp(Exception):
-    """
-    An exception raised in case a kolibri_plugin.py results in loading of the
-    Django app stack.
-    """
-
-    pass
-
-
 def version_file():
     """
     During test runtime, this path may differ because KOLIBRI_HOME is
     regenerated
     """
-    from .conf import KOLIBRI_HOME
-
     return os.path.join(KOLIBRI_HOME, ".data_version")
 
 
@@ -164,9 +148,8 @@ def initialize(debug=False, skip_update=False):
     else:
         # Do this here so that we can fix any issues with our configuration file before
         # we attempt to set up django.
-        from .conf import autoremove_unavailable_plugins, enable_default_plugins
 
-        autoremove_unavailable_plugins()
+        config.autoremove_unavailable_plugins()
 
         version = open(version_file(), "r").read()
         version = version.strip() if version else ""
@@ -175,7 +158,7 @@ def initialize(debug=False, skip_update=False):
             # Reset the enabled plugins to the defaults
             # This needs to be run before dbbackup because
             # dbbackup relies on settings.INSTALLED_APPS
-            enable_default_plugins()
+            config.enable_default_plugins()
 
         if should_back_up(kolibri.__version__, version):
             # Version changed, make a backup no matter what.
@@ -238,9 +221,7 @@ def _first_run():
     if not SKIP_AUTO_DATABASE_MIGRATION:
         _migrate_databases()
 
-    from .conf import enable_default_plugins
-
-    enable_default_plugins()
+    config.enable_default_plugins()
 
     logger.info("Automatically enabling applications.")
 
@@ -288,7 +269,7 @@ def start(port=None, daemon=True):
     :param: port: Port number (default: 8080)
     :param: daemon: Fork to background process (default: True)
     """
-    run_cherrypy = conf.OPTIONS["Server"]["CHERRYPY_START"]
+    run_cherrypy = OPTIONS["Server"]["CHERRYPY_START"]
 
     # In case some tests run start() function only
     if not isinstance(port, int):
@@ -346,7 +327,7 @@ def stop():
         pid, __, __ = server.get_status()
         server.stop(pid=pid)
         stopped = True
-        if conf.OPTIONS["Server"]["CHERRYPY_START"]:
+        if OPTIONS["Server"]["CHERRYPY_START"]:
             logger.info("Kolibri server has successfully been stopped.")
         else:
             logger.info("Kolibri background services have successfully been stopped.")
@@ -458,7 +439,7 @@ def setup_logging(debug=False):
     fail when debug=False, but if it's true it can fail?
     """
     try:
-        from django.conf.settings import LOGGING
+        from django.settings import LOGGING
     except ImportError:
         from kolibri.deployment.default.settings.base import LOGGING
     if debug:
@@ -488,94 +469,20 @@ def manage(cmd, args=[]):
     execute_from_command_line(argv=argv)
 
 
-def _is_plugin(obj):
-    from kolibri.plugins.base import KolibriPluginBase  # NOQA
-
-    return (
-        isinstance(obj, type)
-        and obj is not KolibriPluginBase
-        and issubclass(obj, KolibriPluginBase)
-    )
-
-
-def get_kolibri_plugin(plugin_name):
-    """
-    Try to load kolibri_plugin from given plugin module identifier
-
-    :returns: A list of classes inheriting from KolibriPluginBase
-    """
-
-    plugin_classes = []
-
-    try:
-        plugin_module = importlib.import_module(plugin_name + ".kolibri_plugin")
-        for obj in plugin_module.__dict__.values():
-            if _is_plugin(obj):
-                plugin_classes.append(obj)
-    except ImportError as e:
-        # Python 2: message, Python 3: msg
-        exc_message = getattr(e, "message", getattr(e, "msg", None))
-        if exc_message.startswith("No module named"):
-            msg = (
-                "Plugin '{}' does not seem to exist. Is it on the PYTHONPATH?"
-            ).format(plugin_name)
-            raise PluginDoesNotExist(msg)
-        else:
-            raise
-    except AppRegistryNotReady:
-        msg = (
-            "Plugin '{}' loads the Django app registry, which it isn't "
-            "allowed to do while enabling or disabling itself."
-        ).format(plugin_name)
-        raise PluginBaseLoadsApp(msg)
-
-    if not plugin_classes:
-        # There's no clear use case for a plugin without a KolibriPluginBase
-        # inheritor, for now just throw a warning
-        logger.warning(
-            "Plugin '{}' has no KolibriPluginBase defined".format(plugin_name)
-        )
-
-    return plugin_classes
-
-
 def plugin(plugin_name, **kwargs):
     """
     Receives a plugin identifier and tries to load its main class. Calls class
     functions.
     """
-    from kolibri.utils import conf
-
     if kwargs.get("enable", False):
         logger.info("Enabling Kolibri plugin {}.".format(plugin_name))
-        plugin_classes = get_kolibri_plugin(plugin_name)
-        for klass in plugin_classes:
-            klass.enable()
+        enable_plugin(plugin_name)
 
     if kwargs.get("disable", False):
-        try:
-            logger.info("Disabling Kolibri plugin {}.".format(plugin_name))
-            plugin_classes = get_kolibri_plugin(plugin_name)
-            for klass in plugin_classes:
-                klass.disable()
-        except PluginDoesNotExist as e:
-            logger.error(str(e))
-            logger.warning(
-                "Removing '{}' from configuration in a naive way.".format(plugin_name)
-            )
-            if plugin_name in conf.config["INSTALLED_APPS"]:
-                conf.config["INSTALLED_APPS"].remove(plugin_name)
-                logger.info("Removed '{}' from INSTALLED_APPS".format(plugin_name))
-            else:
-                logger.warning(
-                    (
-                        "Could not find any matches for {} in INSTALLED_APPS".format(
-                            plugin_name
-                        )
-                    )
-                )
+        logger.info("Disabling Kolibri plugin {}.".format(plugin_name))
+        disable_plugin(plugin_name)
 
-    conf.save()
+    config.save()
 
 
 def set_default_language(lang):
@@ -584,7 +491,6 @@ def set_default_language(lang):
     instance of Kolibri needs to be restarted in order for this change to work.
     """
 
-    from kolibri.utils import conf
     from django.conf import settings
 
     valid_languages = [l[0] for l in settings.LANGUAGES]
@@ -595,8 +501,8 @@ def set_default_language(lang):
                 lang
             )
         )
-        conf.config["LANGUAGE_CODE"] = lang
-        conf.save()
+        config["LANGUAGE_CODE"] = lang
+        config.save()
     else:
         msg = "Invalid language code {langcode}. Must be one of: {validlangs}".format(
             langcode=lang, validlangs=valid_languages
diff --git a/kolibri/utils/conf.py b/kolibri/utils/conf.py
index 4c62412675..01356a292c 100644
--- a/kolibri/utils/conf.py
+++ b/kolibri/utils/conf.py
@@ -52,8 +52,6 @@ LOG_ROOT = os.path.join(KOLIBRI_HOME, "logs")
 if not os.path.exists(LOG_ROOT):
     os.mkdir(LOG_ROOT)
 
-#: Set defaults before updating the dict
-config = {}
 
 try:
     # The default list for this is populated from build_tools/default_plugins.txt
@@ -80,92 +78,128 @@ except ImportError:
         "kolibri.plugins.default_theme",
     ]
 
-#: Everything in this list is added to django.conf.settings.INSTALLED_APPS
-config["INSTALLED_APPS"] = DEFAULT_PLUGINS
-
-#: Well-known plugin names that are automatically searched for and enabled on
-#: first-run.
-config["AUTO_SEARCH_PLUGINS"] = []
-
-#: If a config file does not exist, we assume it's the first run
-config["FIRST_RUN"] = True
-
 conf_file = os.path.join(KOLIBRI_HOME, "kolibri_settings.json")
 
 
-def update(new_values):
-    """
-    Updates current configuration with ``new_values``. Does not save to file.
-    """
-    config.update(new_values)
-
-
-def save(first_run=False):
-    """Saves the current state of the configuration"""
-    config["FIRST_RUN"] = first_run
-    # use default OS encoding
-    with open(conf_file, "w") as kolibri_conf_file:
-        json.dump(config, kolibri_conf_file, indent=2, sort_keys=True)
-
-
-if not os.path.isfile(conf_file):
-    logger.info("Initialize kolibri_settings.json..")
-    save(True)
-else:
-    # Open up the config file and overwrite defaults
-    # use default OS encoding
-    with open(conf_file, "r") as kolibri_conf_file:
-        config.update(json.load(kolibri_conf_file))
-
-
-def autoremove_unavailable_plugins():
-    """
-    Sanitize INSTALLED_APPS - something that should be done separately for all
-    build in plugins, but we should not auto-remove plugins that are actually
-    configured by the user or some other kind of hard dependency that should
-    make execution stop if not loadable.
-    """
-    global config
-    changed = False
-    # Iterate over a copy of the list so that it is not modified during the loop
-    for module_path in config["INSTALLED_APPS"][:]:
-        if not module_exists(module_path):
-            config["INSTALLED_APPS"].remove(module_path)
-            logger.error(
-                (
-                    "Plugin {mod} not found and disabled. To re-enable it, run:\n"
-                    "   $ kolibri plugin {mod} enable"
-                ).format(mod=module_path)
-            )
-            changed = True
-    if changed:
-        save()
-
-
-def enable_default_plugins():
-    """
-    Enable new plugins that have been added between versions
-    This will have the undesired side effect of reactivating
-    default plugins that have been explicitly disabled by a user.
-    However, until we add disabled plugins to a blacklist, this is
-    unavoidable.
-    """
-    global config
-    changed = False
-    for module_path in DEFAULT_PLUGINS:
-        if module_path not in config["INSTALLED_APPS"]:
-            config["INSTALLED_APPS"].append(module_path)
-            logger.warning(
-                (
-                    "Default plugin {mod} not found in configuration. To re-disable it, run:\n"
-                    "   $ kolibri plugin {mod} disable"
-                ).format(mod=module_path)
-            )
-            changed = True
-
-    if changed:
-        save()
+# These values are encoded on the config dict as sets
+# so they need to be treated specially for serialization
+# and deserialization to/from JSON
+SET_KEYS = ["INSTALLED_APPS", "DISABLED_APPS"]
+
+
+class ConfigDict(dict):
+    def __init__(self):
+        # If the settings file does not exist or does not contain
+        # valid JSON then create it
+        self.update(
+            {
+                #: Everything in this list is added to django.conf.settings.INSTALLED_APPS
+                # except disabled ones below
+                "INSTALLED_APPS": DEFAULT_PLUGINS,
+                #: Everything in this list is removed from the list above
+                "DISABLED_APPS": [],
+            }
+        )
+        if os.path.isfile(conf_file):
+            try:
+                # Open up the config file and load settings
+                # use default OS encoding
+                with open(conf_file, "r") as kolibri_conf_file:
+                    self.update(json.load(kolibri_conf_file))
+                return
+            except json.JSONDecodeError:
+                logger.warn(
+                    "Attempted to load kolibri_settings.json but encountered a file that could not be decoded as valid JSON."
+                )
+        logger.info("Initialize kolibri_settings.json..")
+        self.save()
+
+    @property
+    def ACTIVE_PLUGINS(self):
+        return list(self["INSTALLED_APPS"] - self["DISABLED_APPS"])
+
+    def update(self, new_values):
+        """
+        Updates current configuration with ``new_values``. Does not save to file.
+        """
+        values_copy = new_values.copy()
+        for key in SET_KEYS:
+            if key in values_copy:
+                values_copy[key] = set(values_copy[key])
+        super(ConfigDict, self).update(values_copy)
+
+    def save(self):
+        # use default OS encoding
+        config_copy = self.copy()
+        for key in SET_KEYS:
+            if key in config_copy:
+                config_copy[key] = list(config_copy[key])
+        with open(conf_file, "w") as kolibri_conf_file:
+            json.dump(config_copy, kolibri_conf_file, indent=2, sort_keys=True)
+
+    def autoremove_unavailable_plugins(self):
+        """
+        Sanitize INSTALLED_APPS - something that should be done separately for all
+        build in plugins, but we should not auto-remove plugins that are actually
+        configured by the user or some other kind of hard dependency that should
+        make execution stop if not loadable.
+        """
+        changed = False
+        # Iterate over a copy of the set so that it is not modified during the loop
+        for module_path in self["INSTALLED_APPS"].copy():
+            if not module_exists(module_path):
+                self["INSTALLED_APPS"].remove(module_path)
+                logger.error(
+                    (
+                        "Plugin {mod} not found and disabled. To re-enable it, run:\n"
+                        "   $ kolibri plugin {mod} enable"
+                    ).format(mod=module_path)
+                )
+                changed = True
+        if changed:
+            self.save()
+
+    def enable_default_plugins(self):
+        """
+        Enable new plugins that have been added between versions
+        This will have the undesired side effect of reactivating
+        default plugins that have been explicitly disabled by a user,
+        in versions prior to the implementation of a plugin blacklist.
+        """
+        changed = False
+        for module_path in DEFAULT_PLUGINS:
+            if module_path not in self["INSTALLED_APPS"]:
+                self["INSTALLED_APPS"].add(module_path)
+                # Can be migrated to upgrade only logic
+                if module_path not in self["DISABLED_APPS"]:
+                    logger.warning(
+                        (
+                            "Default plugin {mod} not found in configuration. To re-disable it, run:\n"
+                            "   $ kolibri plugin {mod} disable"
+                        ).format(mod=module_path)
+                    )
+                changed = True
+
+        if changed:
+            self.save()
+
+    def enable_plugin(self, module_path):
+        self["INSTALLED_APPS"].add(module_path)
+        try:
+            self["DISABLED_APPS"].remove(module_path)
+        except KeyError:
+            pass
+
+    def disable_plugin(self, module_path):
+        self["DISABLED_APPS"].add(module_path)
+        try:
+            self["INSTALLED_APPS"].remove(module_path)
+        except KeyError:
+            pass
+
 
+#: Set defaults before updating the dict
+config = ConfigDict()
 
 # read the config file options in here so they can be accessed from a standard location
 OPTIONS = read_options_file(KOLIBRI_HOME)


Test patch:
diff --git a/kolibri/utils/tests/test_cli.py b/kolibri/utils/tests/test_cli.py
index 316196e5bc..a041712c37 100755
--- a/kolibri/utils/tests/test_cli.py
+++ b/kolibri/utils/tests/test_cli.py
@@ -77,8 +77,8 @@ def conf():
 
     old_config = copy.deepcopy(conf.config)
     yield conf
-    conf.update(old_config)
-    conf.save()
+    conf.config.update(old_config)
+    conf.config.save()
 
 
 def test_bogus_plugin_autoremove(conf):
@@ -86,9 +86,9 @@ def test_bogus_plugin_autoremove(conf):
     Checks that a plugin is auto-removed when it cannot be imported
     """
     plugin_name = "giraffe.horse"
-    conf.config["INSTALLED_APPS"].append(plugin_name)
-    conf.save()
-    conf.autoremove_unavailable_plugins()
+    conf.config["INSTALLED_APPS"].add(plugin_name)
+    conf.config.save()
+    conf.config.autoremove_unavailable_plugins()
     assert plugin_name not in conf.config["INSTALLED_APPS"]
 
 
@@ -97,16 +97,18 @@ def test_bogus_plugin_autoremove_no_path(conf):
     Checks that a plugin without a dotted path is also auto-removed
     """
     plugin_name = "giraffehorse"
-    conf.config["INSTALLED_APPS"].append(plugin_name)
-    conf.save()
-    conf.autoremove_unavailable_plugins()
+    conf.config["INSTALLED_APPS"].add(plugin_name)
+    conf.config.save()
+    conf.config.autoremove_unavailable_plugins()
     assert plugin_name not in conf.config["INSTALLED_APPS"]
 
 
 def test_bogus_plugin_disable(conf):
-    installed_apps_before = conf.config["INSTALLED_APPS"][:]
+    installed_apps_before = conf.config["INSTALLED_APPS"].copy()
+    disabled_apps_before = conf.config["DISABLED_APPS"].copy()
     cli.plugin("i_do_not_exist", disable=True)
     assert installed_apps_before == conf.config["INSTALLED_APPS"]
+    assert disabled_apps_before == conf.config["DISABLED_APPS"]
 
 
 def test_plugin_cannot_be_imported_disable(conf):
@@ -114,31 +116,38 @@ def test_plugin_cannot_be_imported_disable(conf):
     A plugin may be in conf.config['INSTALLED_APPS'] but broken or uninstalled
     """
     plugin_name = "giraffe.horse"
-    conf.config["INSTALLED_APPS"].append(plugin_name)
-    conf.save()
+    conf.config["INSTALLED_APPS"].add(plugin_name)
+    conf.config.save()
     cli.plugin(plugin_name, disable=True)
     assert plugin_name not in conf.config["INSTALLED_APPS"]
+    # We also don't want to endlessly add cruft to the disabled apps
+    assert plugin_name not in conf.config["DISABLED_APPS"]
 
 
 def test_real_plugin_disable(conf):
-    installed_apps_before = conf.config["INSTALLED_APPS"][:]
+    installed_apps_before = conf.config["INSTALLED_APPS"].copy()
     test_plugin = "kolibri.plugins.media_player"
     assert test_plugin in installed_apps_before
     # Because RIP example plugin
     cli.plugin(test_plugin, disable=True)
+    assert test_plugin not in conf.config.ACTIVE_PLUGINS
     assert test_plugin not in conf.config["INSTALLED_APPS"]
+    assert test_plugin in conf.config["DISABLED_APPS"]
 
 
 def test_real_plugin_disable_twice(conf):
-    installed_apps_before = conf.config["INSTALLED_APPS"][:]
+    installed_apps_before = conf.config["INSTALLED_APPS"].copy()
     test_plugin = "kolibri.plugins.media_player"
     assert test_plugin in installed_apps_before
-    # Because RIP example plugin
     cli.plugin(test_plugin, disable=True)
+    assert test_plugin not in conf.config.ACTIVE_PLUGINS
     assert test_plugin not in conf.config["INSTALLED_APPS"]
-    installed_apps_before = conf.config["INSTALLED_APPS"][:]
+    assert test_plugin in conf.config["DISABLED_APPS"]
+    installed_apps_before = conf.config["INSTALLED_APPS"].copy()
     cli.plugin(test_plugin, disable=True)
+    assert test_plugin not in conf.config.ACTIVE_PLUGINS
     assert test_plugin not in conf.config["INSTALLED_APPS"]
+    assert test_plugin in conf.config["DISABLED_APPS"]
 
 
 def test_plugin_with_no_plugin_class(conf):
@@ -147,7 +156,7 @@ def test_plugin_with_no_plugin_class(conf):
     a warning and nothing is enabled or changed in the configuration.
     """
     # For fun, we pass in a system library
-    installed_apps_before = conf.config["INSTALLED_APPS"][:]
+    installed_apps_before = conf.config["INSTALLED_APPS"].copy()
     cli.plugin("os.path")
     assert installed_apps_before == conf.config["INSTALLED_APPS"]