Repository: rotki/rotki

Issue:
Allow users to provide a custom config to the dockerized app
## Problem Definition

I just updated my rotki Docker instance to the latest, and now I'm unable to access the UI at all (`ERR_CONNECTION_TIMED_OUT`)

## Logs

I can't show debug logs as I can't get into the UI.
`$HOME/.rotki/logs/rotki.log`:
```
[08/05/2021 19:38:46 UTC] INFO rotkehlchen.api.server: Rotki API server is running at: 0.0.0.0:4242
[08/05/2021 19:39:15 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:39:15 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.002098
[08/05/2021 19:39:45 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:39:45 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001365
[08/05/2021 19:40:16 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:40:16 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001411
[08/05/2021 19:40:46 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:40:46 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001371
[08/05/2021 19:41:16 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:41:16 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001422
[08/05/2021 19:41:47 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:41:47 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001365
[08/05/2021 19:42:17 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:42:17 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001661
[08/05/2021 19:42:48 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:42:48 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001390
[08/05/2021 19:43:18 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:43:18 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001448
[08/05/2021 19:43:48 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:43:48 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001344
[08/05/2021 19:44:19 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:44:19 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001342
[08/05/2021 19:44:49 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:44:49 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001375
[08/05/2021 19:45:20 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:45:20 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001341
[08/05/2021 19:45:50 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:45:50 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001357
[08/05/2021 19:46:20 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:46:20 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001534
[08/05/2021 19:46:51 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:46:51 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.002069
[08/05/2021 19:47:21 UTC] DEBUG rotkehlchen.api.rest: Request successful response={"result": {}, "message": ""}, status_code=200
[08/05/2021 19:47:21 UTC] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/users HTTP/1.0" 200 236 0.001344
```

This simply repeats over and over, and I can't get into the UI. Is it possible to handle `rotki_config.json` via the Docker image to provide more information?

Interesting to note that the interval is 30s...

### System Description
amd64 Linux host

Rotki version: 1.16.2


Discussion:
@m-roberts do you see anything at all when you attempt to open the web page? At least it seems that something is hitting the backend.

If you see at least part of the page can you try closing the tab and opening a new one? Or doing this in combination with restarting the container? 
@kelsos I have rebooted the machine, restarted the container, opened new tabs, etc. I see nothing other than an attempt to connect that times out.
It would be great if there was a way to easily change the configuration of the running app from `docker run` - that would allow me to easily change to debug, plus all other additional running features!
@kelsos - my apologies. This was a VERY sneaky issue with my server!

However, I would like to repurpose this Issue for my suggestion on improving config ease-of-use for us containerizers! 😄
No worries. Yes let's keep the issue to expose the config 
Specification ideas (from #3792):

* Environment variable parameters for `docker run`.
* Pass in a config file via a `/config` volume mount? (e.g. `/config/rotki_config.json`)
It shouldn't be hard to do any of both

Resolving patch:
diff --git a/packaging/docker/entrypoint.py b/packaging/docker/entrypoint.py
new file mode 100755
--- /dev/null
+++ b/packaging/docker/entrypoint.py
@@ -0,0 +1,146 @@
+#!/usr/bin/python3
+import json
+import logging
+import os
+import subprocess
+import time
+from pathlib import Path
+from typing import Dict, Optional, Any, List
+
+logger = logging.getLogger('monitor')
+logging.basicConfig(level=logging.DEBUG)
+
+DEFAULT_LOG_LEVEL = 'critical'
+
+
+def load_config_from_file() -> Optional[Dict[str, Any]]:
+    config_file = Path('/config/rotki_config.json')
+    if not config_file.exists():
+        logger.info('no config file provided')
+        return None
+
+    with open(config_file) as file:
+        try:
+            data = json.load(file)
+            return data
+        except json.JSONDecodeError as e:
+            logger.error(e)
+            return None
+
+
+def load_config_from_env() -> Dict[str, Any]:
+    loglevel = os.environ.get('LOGLEVEL')
+    logfromothermodules = os.environ.get('LOGFROMOTHERMODDULES')
+    sleep_secs = os.environ.get('SLEEP_SECS')
+    max_size_in_mb_all_logs = os.environ.get('MAX_SIZE_IN_MB_ALL_LOGS')
+    max_logfiles_num = os.environ.get('MAX_LOGFILES_NUM')
+
+    return {
+        'loglevel': loglevel,
+        'logfromothermodules': logfromothermodules,
+        'sleep_secs': sleep_secs,
+        'max_logfiles_num': max_logfiles_num,
+        'max_size_in_mb_all_logs': max_size_in_mb_all_logs,
+    }
+
+
+def load_config() -> List[str]:
+    env_config = load_config_from_env()
+    file_config = load_config_from_file()
+
+    logger.info('loading config from env')
+
+    loglevel = env_config.get('loglevel')
+    log_from_other_modules = env_config.get('logfromothermodules')
+    sleep_secs = env_config.get('sleep_secs')
+    max_logfiles_num = env_config.get('max_logfiles_num')
+    max_size_in_mb_all_logs = env_config.get('max_size_in_mb_all_logs')
+
+    if file_config is not None:
+        logger.info('loading config from file')
+
+        if file_config.get('loglevel') is not None:
+            loglevel = file_config.get('loglevel')
+
+        if file_config.get('logfromothermodules') is not None:
+            log_from_other_modules = file_config.get('logfromothermodules')
+
+        if file_config.get('sleep-secs') is not None:
+            sleep_secs = file_config.get('sleep-secs')
+
+        if file_config.get('max_logfiles_num') is not None:
+            max_logfiles_num = file_config.get('max_logfiles_num')
+
+        if file_config.get('max_size_in_mb_all_logs') is not None:
+            max_size_in_mb_all_logs = file_config.get('max_size_in_mb_all_logs')
+
+    args = [
+        '--data-dir',
+        '/data',
+        '--logfile',
+        '/logs/rotki.log',
+        '--loglevel',
+        loglevel if loglevel is not None else DEFAULT_LOG_LEVEL,
+    ]
+
+    if log_from_other_modules is True:
+        args.append('--logfromothermodules')
+
+    if sleep_secs is not None:
+        args.append('--sleep-secs')
+        args.append(str(sleep_secs))
+
+    if max_logfiles_num is not None:
+        args.append('--max-logfiles-num')
+        args.append(str(max_logfiles_num))
+
+    if max_size_in_mb_all_logs is not None:
+        args.append('--max-size-in-mb-all-logs')
+        args.append(str(max_size_in_mb_all_logs))
+
+    return args
+
+
+base_args = [
+    '/usr/sbin/rotki',
+    '--rest-api-port',
+    '4242',
+    '--websockets-api-port',
+    '4243',
+    '--api-cors',
+    'http://localhost:*/*,app://.',
+    '--api-host',
+    '0.0.0.0',
+]
+
+config_args = load_config()
+cmd = base_args + config_args
+
+logger.info('starting rotki backend')
+
+rotki = subprocess.Popen(cmd)
+
+if rotki.returncode == 1:
+    logger.error('Failed to start rotki')
+    exit(1)
+
+logger.info('starting nginx')
+
+nginx = subprocess.Popen('nginx -g "daemon off;"', shell=True)
+
+if nginx.returncode == 1:
+    logger.error('Failed to start nginx')
+    exit(1)
+
+while True:
+    time.sleep(60)
+
+    if rotki.poll() is not None:
+        logger.error('rotki has terminated exiting')
+        exit(1)
+
+    if nginx.poll() is not None:
+        logger.error('nginx was not running')
+        exit(1)
+
+    logger.info('OK: processes still running')


Test patch: