Repository: learningequality/kolibri

Issue:
File-Based Logging During Upgrade Ignores Kolibri Setting 
<!--
Instructions:
 * Fill out the sections below, replace …'s with information about your issue
 * Use the 'preview' function above this text box to verify formatting before submitting
-->

## Observed behavior
<!--
Description of the behavior that was observed, including screenshots or other references when applicable
-->

Despite the `KOLIBRI_NO_FILE_BASED_LOGGING` envar being set, Kolibri tries to write to logs during upgrade.

## Errors and logs
<!--
Relevant logs from:
 * the command line
 * ~/.kolibri/logs/kolibri.txt
 * the browser console

Please wrap errors in triple backticks for clean formatting like this:
```
01:10 info: something happened
01:12 error: something bad happened
```
-->

```
Traceback (most recent call last):
  File "/usr/local/bin/kolibri", line 5, in <module>
    from kolibri.utils.cli import main
  File "/usr/local/lib/python3.6/site-packages/kolibri/utils/cli.py", line 17, in <module>
    from kolibri.plugins import config
  File "/usr/local/lib/python3.6/site-packages/kolibri/plugins/__init__.py", line 16, in <module>
    from kolibri.utils.conf import KOLIBRI_HOME
  File "/usr/local/lib/python3.6/site-packages/kolibri/utils/conf.py", line 47, in <module>
    os.mkdir(LOG_ROOT)
OSError: [Errno 30] Read-only file system: '/home/kolibri/.kolibri/logs'
```

## Expected behavior
<!--
Description of what behavior was expected but did not occur
-->

Would assume that it Kolibri does not try to log to file when the `NO_FILE_BASED_LOGGING` setting is set to `True`

## Steps to reproduce
<!--
Precise steps that someone else can follow in order to see this behavior
-->

1. Set the `KOLIBRI_NO_FILE_BASED_LOGGING` envar to true
2. Run any command that triggers upgrade. `kolibri manage migrate` in my case

## Context
<!--
Tell us about your environment, including:
 * Kolibri version
 * Operating system
 * Browser
-->

Was discovered during CD, in a Linux container of Ubuntu LTS 18.



Discussion:
Discovered by myself, mentioned to @rtibbles 
Hi @rtibbles !
I looked into this issue a bit. The environment variable `KOLIBRI_NO_FILE_BASED_LOGGING` works as expected when explicitly typecasted to a boolean value in the code. 
The thing is, if the _'logs'_ folder doesn't already exist, the conf.py files tries to create it here:
https://github.com/learningequality/kolibri/blob/0f182a3ab253bc53c5ed8927e75f74f3e3b182f7/kolibri/utils/conf.py#L45-L47
Now, if the folder location as pointed by `KOLIBRI_HOME` is read-only, we'll always get the error mentioned in the issue. So, we just have to make sure that if the variable is set to True, we don't create the _'logs'_ folder, if it isn't already present.
Please correct me if I am wrong with my inference. 

Also, I'd like to work on this issue. Could you please assign it to me? I'd love to hear your suggestions on how to tackle it.
Excellent sleuthing @KshitijThareja! That does seem like the cause here. And I think you are absolutely correct, that we should only try to create the `LOG_ROOT` directory in the case that it does not exist and we do not have the `KOLIBRI_NO_FILE_BASED_LOGGING` flag set!

I have assigned you, looking forward to the pull request. I have retargeted the issue to the 0.16.0 milestone, as this would be a useful fix to have sooner rather than later and seems low risk. Please branch off release-v0.16.x and target there for your pull request, thank you!
Thanks @rtibbles! Will try to put up a PR soon for this 👍🏻

Resolving patch:
diff --git a/kolibri/utils/conf.py b/kolibri/utils/conf.py
--- a/kolibri/utils/conf.py
+++ b/kolibri/utils/conf.py
@@ -29,6 +29,7 @@
 #: Absolute path of the main user data directory.
 #: Will be created automatically if it doesn't exist.
 KOLIBRI_HOME = os.path.abspath(os.path.expanduser(os.environ["KOLIBRI_HOME"]))
+NO_FILE_BASED_LOGGING = os.environ.get("KOLIBRI_NO_FILE_BASED_LOGGING", False)
 
 # Creating KOLIBRI_HOME atm. has to happen here as for instance utils.cli is not
 # called through py.test. This file is the first basic entry point of
@@ -43,7 +44,7 @@
 
 # Create a folder named logs inside KOLIBRI_HOME to store all the log files.
 LOG_ROOT = os.path.join(KOLIBRI_HOME, "logs")
-if not os.path.exists(LOG_ROOT):
+if not os.path.exists(LOG_ROOT) and not NO_FILE_BASED_LOGGING:
     os.mkdir(LOG_ROOT)
 
 
diff --git a/kolibri/utils/logger.py b/kolibri/utils/logger.py
--- a/kolibri/utils/logger.py
+++ b/kolibri/utils/logger.py
@@ -205,6 +205,43 @@ def get_default_logging_config(LOG_ROOT, debug=False, debug_database=False):
     DEFAULT_LEVEL = "INFO" if not debug else "DEBUG"
     DATABASE_LEVEL = "INFO" if not debug_database else "DEBUG"
 
+    handlers = {
+        "console-error": {
+            "level": "ERROR",
+            "class": "kolibri.utils.logger.EncodingStreamHandler",
+            "formatter": "color",
+            "stream": "ext://sys.stderr",
+        },
+        "console": {
+            "level": DEFAULT_LEVEL,
+            "filters": ["no_exceptions"],
+            "class": "kolibri.utils.logger.EncodingStreamHandler",
+            "formatter": "color",
+            "stream": "ext://sys.stdout",
+        },
+    }
+
+    if not NO_FILE_BASED_LOGGING:
+        handlers["file"] = {
+            "level": "INFO",
+            "filters": [],
+            "class": "kolibri.utils.logger.KolibriTimedRotatingFileHandler",
+            "filename": os.path.join(LOG_ROOT, "kolibri.txt"),
+            "formatter": "simple_date",
+            "when": "midnight",
+            "backupCount": 30,
+            "encoding": "utf-8",
+        }
+
+        handlers["file_debug"] = {
+            "level": "DEBUG",
+            "filters": ["require_debug_true"],
+            "class": "logging.FileHandler",
+            "filename": os.path.join(LOG_ROOT, "debug.txt"),
+            "formatter": "simple_date",
+            "encoding": "utf-8",
+        }
+
     return {
         "version": 1,
         "disable_existing_loggers": False,
@@ -223,39 +260,7 @@ def get_default_logging_config(LOG_ROOT, debug=False, debug_database=False):
                 "log_colors": LOG_COLORS,
             },
         },
-        "handlers": {
-            "console-error": {
-                "level": "ERROR",
-                "class": "kolibri.utils.logger.EncodingStreamHandler",
-                "formatter": "color",
-                "stream": "ext://sys.stderr",
-            },
-            "console": {
-                "level": DEFAULT_LEVEL,
-                "filters": ["no_exceptions"],
-                "class": "kolibri.utils.logger.EncodingStreamHandler",
-                "formatter": "color",
-                "stream": "ext://sys.stdout",
-            },
-            "file": {
-                "level": "INFO",
-                "filters": [],
-                "class": "kolibri.utils.logger.KolibriTimedRotatingFileHandler",
-                "filename": os.path.join(LOG_ROOT, "kolibri.txt"),
-                "formatter": "simple_date",
-                "when": "midnight",
-                "backupCount": 30,
-                "encoding": "utf-8",
-            },
-            "file_debug": {
-                "level": "DEBUG",
-                "filters": ["require_debug_true"],
-                "class": "logging.FileHandler",
-                "filename": os.path.join(LOG_ROOT, "debug.txt"),
-                "formatter": "simple_date",
-                "encoding": "utf-8",
-            },
-        },
+        "handlers": handlers,
         "loggers": {
             "": {
                 "handlers": DEFAULT_HANDLERS,


Test patch: