Repository: freqtrade/freqtrade

Issue:
wrong starting datetime for download-data
```
$ freqtrade download-data --exchange ftx -t 1d -p ETH/USD --days 100
...
2020-03-05 12:38:23,910 - freqtrade.exchange.exchange - INFO - downloaded ETH/USD with length 101.

$ freqtrade download-data --exchange ftx -t 1h -p ETH/USD --days 100
...
2020-03-05 12:38:37,412 - freqtrade.exchange.exchange - INFO - downloaded ETH/USD with length 2411.
```

-- here 101 is okay (seems to be): 100 days plus 1 current day, which is then thrown away as incomplete (?)
-- 2411 is wrong, should be 2401 or 2400. Plus, 11 hours here relates to the local time on the machine (as you can see, local time is 12:38, in UTC+3)



Discussion:
why?
It's downloading data for the last 100 days up to the current moment - which is 100 days (2400) + 11 hours.

downloading data from 100 days ago starting at 11:00 (which is what you're requesting here) does not make any sense at all - what would you do with half a day worth of data ... ? 

to me, this is not a bug, but it works as expected.
> why?
> It's downloading data for the last 100 days up to the current moment - which is 100 days (2400) + 11 hours.

I expected it to be last 100 full 24-h intervals (as data load for last 5 hours assumes 5 last full 1h-intervals and not 5 hrs + 38 minutes, if it happens that we start downloading at 38th minute of the current hour...)
at least, local time shift is used. UTC time is 09:38:37,412, while 11 hours are added to 100 days... Very confusing.

the parameter is called `--days X` ... it calculates X days back from the current day - and downloads full days.

Using the current time of the day would be strange - you would get data from 100 days ago - starting at 17:15 ... but what's the point in not having these additional 17 hours... ? if you're that tight on disk-space you probably got other problems to worry about first.
it's not about disk space, current behavior is odd, as for me: it's "X days back", i.e. starting since the time X 24-h intervals ago, as "5 hrs ago" means 5 hrs ago, not 5 hrs and 38 min ago... Why "X full days" should be used if we download for shorter timeframe, for 1h or 1m? 


Resolving patch:
diff --git a/freqtrade/commands/arguments.py b/freqtrade/commands/arguments.py
--- a/freqtrade/commands/arguments.py
+++ b/freqtrade/commands/arguments.py
@@ -56,7 +56,7 @@
 
 ARGS_LIST_DATA = ["exchange", "dataformat_ohlcv", "pairs"]
 
-ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "download_trades", "exchange",
+ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "timerange", "download_trades", "exchange",
                       "timeframes", "erase", "dataformat_ohlcv", "dataformat_trades"]
 
 ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit",
diff --git a/freqtrade/commands/data_commands.py b/freqtrade/commands/data_commands.py
--- a/freqtrade/commands/data_commands.py
+++ b/freqtrade/commands/data_commands.py
@@ -25,11 +25,17 @@ def start_download_data(args: Dict[str, Any]) -> None:
     """
     config = setup_utils_configuration(args, RunMode.UTIL_EXCHANGE)
 
+    if 'days' in config and 'timerange' in config:
+        raise OperationalException("--days and --timerange are mutually exclusive. "
+                                   "You can only specify one or the other.")
     timerange = TimeRange()
     if 'days' in config:
         time_since = arrow.utcnow().shift(days=-config['days']).strftime("%Y%m%d")
         timerange = TimeRange.parse_timerange(f'{time_since}-')
 
+    if 'timerange' in config:
+        timerange = timerange.parse_timerange(config['timerange'])
+
     if 'pairs' not in config:
         raise OperationalException(
             "Downloading data requires a list of pairs. "


Test patch:
diff --git a/tests/commands/test_commands.py b/tests/commands/test_commands.py
--- a/tests/commands/test_commands.py
+++ b/tests/commands/test_commands.py
@@ -2,6 +2,7 @@
 from pathlib import Path
 from unittest.mock import MagicMock, PropertyMock
 
+import arrow
 import pytest
 
 from freqtrade.commands import (start_convert_data, start_create_userdir,
@@ -552,6 +553,50 @@ def test_download_data_keyboardInterrupt(mocker, caplog, markets):
     assert dl_mock.call_count == 1
 
 
+def test_download_data_timerange(mocker, caplog, markets):
+    dl_mock = mocker.patch('freqtrade.commands.data_commands.refresh_backtest_ohlcv_data',
+                           MagicMock(return_value=["ETH/BTC", "XRP/BTC"]))
+    patch_exchange(mocker)
+    mocker.patch(
+        'freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets)
+    )
+    args = [
+        "download-data",
+        "--exchange", "binance",
+        "--pairs", "ETH/BTC", "XRP/BTC",
+        "--days", "20",
+        "--timerange", "20200101-"
+    ]
+    with pytest.raises(OperationalException,
+                       match=r"--days and --timerange are mutually.*"):
+        start_download_data(get_args(args))
+    assert dl_mock.call_count == 0
+
+    args = [
+        "download-data",
+        "--exchange", "binance",
+        "--pairs", "ETH/BTC", "XRP/BTC",
+        "--days", "20",
+    ]
+    start_download_data(get_args(args))
+    assert dl_mock.call_count == 1
+    # 20days ago
+    days_ago = arrow.get(arrow.utcnow().shift(days=-20).date()).timestamp
+    assert dl_mock.call_args_list[0][1]['timerange'].startts == days_ago
+
+    dl_mock.reset_mock()
+    args = [
+        "download-data",
+        "--exchange", "binance",
+        "--pairs", "ETH/BTC", "XRP/BTC",
+        "--timerange", "20200101-"
+    ]
+    start_download_data(get_args(args))
+    assert dl_mock.call_count == 1
+
+    assert dl_mock.call_args_list[0][1]['timerange'].startts == arrow.Arrow(2020, 1, 1).timestamp
+
+
 def test_download_data_no_markets(mocker, caplog):
     dl_mock = mocker.patch('freqtrade.commands.data_commands.refresh_backtest_ohlcv_data',
                            MagicMock(return_value=["ETH/BTC", "XRP/BTC"]))