Repository: freqtrade/freqtrade

Issue:
 Docker image making logfiles in user_data
docker image places error logfiles in user_data by default. (apparently it should not be doing that)
Maybe cud have it put them in a log dir?


docker-compose.yml

command: >
      trade
      --logfile /freqtrade/user_data/freqtrade.log

can maybe be changed to 
      --logfile /freqtrade/user_data/log/freqtrade.log



Discussion:
I personally agree. 👍 
@xmatthias ?
in theory yes - but how does the logging module react if you log to a folder which doesn't exist? will it autocreate it ?  (this is not a docker-issue - as the whole user_data folder is mounted to the container.

Changing this seems easy enough from a user perspective - but thinking about it from an update perspective however it may require you to either create that directory manually - or rerun `create-userdir` to actually go the logs  if logging does not handle this automatically.


just tested ... that's my point  ...

```
Traceback (most recent call last):
  File "/freqtrade/freqtrade/main.py", line 36, in main
    return_code = args['func'](args)
  File "/freqtrade/freqtrade/commands/trade_commands.py", line 19, in start_trading
    worker = Worker(args)
  File "/freqtrade/freqtrade/worker.py", line 34, in __init__
    self._init(False)
  File "/freqtrade/freqtrade/worker.py", line 50, in _init
    self._config = Configuration(self._args, None).get_config()
  File "/freqtrade/freqtrade/configuration/configuration.py", line 41, in get_config
    self.config = self.load_config()
  File "/freqtrade/freqtrade/configuration/configuration.py", line 99, in load_config
    self._process_logging_options(config)
  File "/freqtrade/freqtrade/configuration/configuration.py", line 131, in _process_logging_options
    setup_logging(config)
  File "/freqtrade/freqtrade/loggers.py", line 71, in setup_logging
    log_handlers.append(RotatingFileHandler(logfile,
  File "/home/xmatt/.pyenv/versions/3.8.1/lib/python3.8/logging/handlers.py", line 148, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
  File "/home/xmatt/.pyenv/versions/3.8.1/lib/python3.8/logging/handlers.py", line 55, in __init__
    logging.FileHandler.__init__(self, filename, mode, encoding, delay)
  File "/home/xmatt/.pyenv/versions/3.8.1/lib/python3.8/logging/__init__.py", line 1143, in __init__
    StreamHandler.__init__(self, self._open())
  File "/home/xmatt/.pyenv/versions/3.8.1/lib/python3.8/logging/__init__.py", line 1172, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/freqtrade/user_data/logs/logfile.txt'

```
logs/ did not exist - and "logging" did not create it.
I'm also not a big fan of creating it intentionally - assume i mistyped and instead of putting it to `./user_data/logs` - it would (try) to create `/user_data/logs` (in the system root) - which will very clearly fail.

> it would (try) to create /user_data/logs (in the system root) - which will very clearly fail.

and that's fine (should not be with an ugly traceback as above, but with a clean error message) -- you'll notice that you misstyped it.


Honestly, i don't see the problem in this.

Users will need to modify the docker-compose file anyway - you will not run the Samplestrategy in prodction (i hope) ... so while modifying this, everyone can modify the log location as he desires.

I don't see a problem to log to user_data - apart from an eventual `config.json` and a few directories - it's empty.

One note: we will definitely NOT use /var/log - as that will fail on windows (docker).

do we have windows docker images?
Docker is cross-platform - you can run the regular linux images on windows as well.

Now Docker on windows CAN be switched to support "windows containers" - but the default is to not use windows containers (and i've never heared of anyone using this so far as it's pretty new ...).


Apart from that - mounting /var/log is a huge permission pain - so you will not want to go there.
> Docker is cross-platform - you can run the regular linux images on windows as well.

I mean, docker images can be windows images and linux images ("windows containers" as you named them below)

> Apart from that - mounting /var/log is a huge permission pain - so you will not want to go there.

No-no, I mean writing logs into /var/log in the docker container, not into the host system /var/log, which is to be mounted and, yes, can cause pain.
> No-no, I mean writing logs into /var/log in the docker container, not into the host system /var/log, which is to be mounted and, yes, can cause pain.

with what goal? 
I don't think that has anything to do with this issue - all it means is that it'll be another volume that has to be mounted.

Resolving patch:
diff --git a/freqtrade/configuration/directory_operations.py b/freqtrade/configuration/directory_operations.py
--- a/freqtrade/configuration/directory_operations.py
+++ b/freqtrade/configuration/directory_operations.py
@@ -33,8 +33,8 @@ def create_userdata_dir(directory: str, create_dir: bool = False) -> Path:
     :param create_dir: Create directory if it does not exist.
     :return: Path object containing the directory
     """
-    sub_dirs = ["backtest_results", "data", "hyperopts", "hyperopt_results", "notebooks",
-                "plot", "strategies", ]
+    sub_dirs = ["backtest_results", "data", "hyperopts", "hyperopt_results", "logs",
+                "notebooks", "plot", "strategies", ]
     folder = Path(directory)
     if not folder.is_dir():
         if create_dir:


Test patch:
diff --git a/tests/test_directory_operations.py b/tests/test_directory_operations.py
--- a/tests/test_directory_operations.py
+++ b/tests/test_directory_operations.py
@@ -25,7 +25,7 @@ def test_create_userdata_dir(mocker, default_conf, caplog) -> None:
     md = mocker.patch.object(Path, 'mkdir', MagicMock())
 
     x = create_userdata_dir('/tmp/bar', create_dir=True)
-    assert md.call_count == 8
+    assert md.call_count == 9
     assert md.call_args[1]['parents'] is False
     assert log_has(f'Created user-data directory: {Path("/tmp/bar")}', caplog)
     assert isinstance(x, Path)