Repository: freqtrade/freqtrade

Issue:
Possibly slightly wrong informative pair merging
When merging longer timeframe to a shorter one, the timedelta is added to the date:
https://github.com/freqtrade/freqtrade/blob/f320cb0d7a12e064018967cf049dd6719fff8ddb/freqtrade/strategy/strategy_helper.py#L35

then it merges to the shorter timeframe with these dates..
https://github.com/freqtrade/freqtrade/blob/f320cb0d7a12e064018967cf049dd6719fff8ddb/freqtrade/strategy/strategy_helper.py#L42-L43

So lets say we are merging a `1h` timeframe to a `5m` timeframe and there is a signal `True` at `00:00` on the `1h` timeseries. 
With this merge, the signal will appear in the `5m` timeseries also at `00:00`. However the `00:00` candle for the `5m` timeframe is received at `00:05`, that is five minutes later than the time you actually received the `1h` candle, which should have been received at the candle with date `11:55`. 
So after merging, the values should be shifted backward (`shift(-1)`)..or the merging dates should be reduced by one unit of timedelta of the shorter timeframe..
```python
informative['date_merge'] = informative["date"] + pd.to_timedelta(minutes_inf, 'm') - pd.to_timedelta(minutes, 'm')
```


Discussion:
hi @orehunt,

all candle dates are candle open dates (to keep in mind for the rest below, both for myself and for future readers)

I'll use 15m below, as this is easier to visualize (but the concept remains the same for 5m and 15m candles)
i've intentionally left all timestamps, also the one that's not shown to users usually (`date_merge_1h`).

![2020-12-16-195613_1219x286_scrot](https://user-images.githubusercontent.com/5024695/102393524-c70dfa80-3fd8-11eb-84af-d4dfcb0a83ec.png)

If i understand you correctly, then what you're saying is that the 15m candle at 23:45 should be merged with the 1h candle of 23:00 instead of with the candle of 22:00 (which is currently the case).

what you'd see as correct would basically look as follows?
(notice that the 23:45 candle is merged with the 23:00 candle)

![2020-12-16-201415_1216x294_scrot](https://user-images.githubusercontent.com/5024695/102395350-48668c80-3fdb-11eb-83bf-af04a47fb4da.png)

@xmatthias Yes, that is correct, the `23:45/15m` candle is received at the same time of the `23:00/1h` candle, therefore should be merged on those dates. 

To be precise, currently, it does not merge `22:00/1h` to `23:45/15m`, it merges to `23:00/15m` and forward fills to `23:45/15m`

Resolving patch:
diff --git a/freqtrade/strategy/strategy_helper.py b/freqtrade/strategy/strategy_helper.py
--- a/freqtrade/strategy/strategy_helper.py
+++ b/freqtrade/strategy/strategy_helper.py
@@ -24,15 +24,24 @@ def merge_informative_pair(dataframe: pd.DataFrame, informative: pd.DataFrame,
     :param timeframe: Timeframe of the original pair sample.
     :param timeframe_inf: Timeframe of the informative pair sample.
     :param ffill: Forwardfill missing values - optional but usually required
+    :return: Merged dataframe
+    :raise: ValueError if the secondary timeframe is shorter than the dataframe timeframe
     """
 
     minutes_inf = timeframe_to_minutes(timeframe_inf)
     minutes = timeframe_to_minutes(timeframe)
-    if minutes >= minutes_inf:
+    if minutes == minutes_inf:
         # No need to forwardshift if the timeframes are identical
         informative['date_merge'] = informative["date"]
+    elif minutes < minutes_inf:
+        # Subtract "small" timeframe so merging is not delayed by 1 small candle
+        # Detailed explanation in https://github.com/freqtrade/freqtrade/issues/4073
+        informative['date_merge'] = (
+            informative["date"] + pd.to_timedelta(minutes_inf, 'm') - pd.to_timedelta(minutes, 'm')
+            )
     else:
-        informative['date_merge'] = informative["date"] + pd.to_timedelta(minutes_inf, 'm')
+        raise ValueError("Tried to merge a faster timeframe to a slower timeframe."
+                         "This would create new rows, and can throw off your regular indicators.")
 
     # Rename columns to be unique
     informative.columns = [f"{col}_{timeframe_inf}" for col in informative.columns]


Test patch:
diff --git a/tests/strategy/test_strategy_helpers.py b/tests/strategy/test_strategy_helpers.py
--- a/tests/strategy/test_strategy_helpers.py
+++ b/tests/strategy/test_strategy_helpers.py
@@ -1,5 +1,6 @@
 import numpy as np
 import pandas as pd
+import pytest
 
 from freqtrade.strategy import merge_informative_pair, timeframe_to_minutes
 
@@ -47,17 +48,17 @@ def test_merge_informative_pair():
     assert 'volume_1h' in result.columns
     assert result['volume'].equals(data['volume'])
 
-    # First 4 rows are empty
+    # First 3 rows are empty
     assert result.iloc[0]['date_1h'] is pd.NaT
     assert result.iloc[1]['date_1h'] is pd.NaT
     assert result.iloc[2]['date_1h'] is pd.NaT
-    assert result.iloc[3]['date_1h'] is pd.NaT
     # Next 4 rows contain the starting date (0:00)
+    assert result.iloc[3]['date_1h'] == result.iloc[0]['date']
     assert result.iloc[4]['date_1h'] == result.iloc[0]['date']
     assert result.iloc[5]['date_1h'] == result.iloc[0]['date']
     assert result.iloc[6]['date_1h'] == result.iloc[0]['date']
-    assert result.iloc[7]['date_1h'] == result.iloc[0]['date']
     # Next 4 rows contain the next Hourly date original date row 4
+    assert result.iloc[7]['date_1h'] == result.iloc[4]['date']
     assert result.iloc[8]['date_1h'] == result.iloc[4]['date']
 
 
@@ -86,3 +87,11 @@ def test_merge_informative_pair_same():
 
     # Dates match 1:1
     assert result['date_15m'].equals(result['date'])
+
+
+def test_merge_informative_pair_lower():
+    data = generate_test_data('1h', 40)
+    informative = generate_test_data('15m', 40)
+
+    with pytest.raises(ValueError, match=r"Tried to merge a faster timeframe .*"):
+        merge_informative_pair(data, informative, '1h', '15m', ffill=True)