Repository: freqtrade/freqtrade

Issue:
Exit after n minutes/n candles and roi = 0
Hi there,

1/ For testing purpose of a strategy i would like to do something like (test what happens if i close my trade after 30 minutes)

minimal_roi = {'30':-1}
stop_loss = -1

![image](https://user-images.githubusercontent.com/2643583/65308194-d1e59f80-db89-11e9-8ced-8aeb62f72bfc.png)

Freqtrade is not happy with that.

2/ this type of problem appear too when using 

minimal_roi = {'0': 0.02, '30': 0}
stoploss = -0.02

![image](https://user-images.githubusercontent.com/2643583/65308693-e5ddd100-db8a-11e9-9550-2be69f357bd9.png)

14% of the trades have 0% roi but it is not possible because when you close the trade 30 minutes with profit > 0 to must be take into consideration.

I think in live he don't have this type of problem ?

Have a nice day.






Exit after n minutes/n candles and roi = 0
Hi there,

1/ For testing purpose of a strategy i would like to do something like (test what happens if i close my trade after 30 minutes)

minimal_roi = {'30':-1}
stop_loss = -1

![image](https://user-images.githubusercontent.com/2643583/65308194-d1e59f80-db89-11e9-8ced-8aeb62f72bfc.png)

Freqtrade is not happy with that.

2/ this type of problem appear too when using 

minimal_roi = {'0': 0.02, '30': 0}
stoploss = -0.02

![image](https://user-images.githubusercontent.com/2643583/65308693-e5ddd100-db8a-11e9-9550-2be69f357bd9.png)

14% of the trades have 0% roi but it is not possible because when you close the trade 30 minutes with profit > 0 to must be take into consideration.

I think in live he don't have this type of problem ?

Have a nice day.








Discussion:
I think  "sell-reason reporting" is the real problem, since your sell-reason is always ROI after 30 minutes (since you're abusing it with negative values). 

I think you're misinterpreting sell_reason.
This column tells you why the bot sold (because of your ROI configuration) - not necessarily if it's positive or negative. You can also have trailing-stop as reason, which can be positive or negative, but the reason will still be trailing-stoploss - no matter if it's + or - 5%.

1)
your first configuration seems to work, but produces a huge loss after 30 minutes due to "roi". 
Ignore the sell-reason column for now and load the backtest-results file to pandas as shown in the documentation (i assume you're doing that here anyway, according to the screenshots...)

Assume you load the trades, then override sell_reason with `backtest_df.loc[backtest_df['profitperc'] < 0, 'sell_reason'] = "negative_roi"`, and then do a `value_counts()` grouping by that column again (which will produce the same output than backtesting, but with fixed sell-reasons.

This should produce "some" positive results (maybe not - then your buy-signal is not suitable for 30min trades)...

2)
I'm not certain about how to interpret your second results... you're counting over "profitperc" - so it's telling me that 13% of your trades were sold with 0 profit. Why should this not be possible?





Thinking about 2 again, i know the reason for this too:
backtesting always uses the "roi" value as sell-value. The reason for this is simple - assume your ROI setting is 2% - but "high" was 5% above buy price. 
in Dry/live runs, your bot would have sold at 2%, and not 5% - so using high is wrong, same with close (which could be 4% - but the bot sells as soon as 2% roi is hit).

Now in dry/live runs, you would not see 2% but 2.01% or so (depending on the sell ordertype, how fast the market moves etc) ... but simulating this reliably in backtesting is impossible, since we lack the information for it.

hope this helps explaining this.
Hi,

the problematic i have is solve after n bar/n minutes no matter loss/profit. i would like to manage this with 

```
minimal_roi = {'30':-1}
stop_loss = -1
```

After doing that he stop the trade at 30 minutes but take the roi value : -1 as the profit (-99.9% per trade) and not the close-open value to calculate profit. 

![image](https://user-images.githubusercontent.com/2643583/65606138-9f351000-dfaa-11e9-9061-b1e72fa016d2.png)


2/ Here it is the same problem but with more realistic use case  i want to try closing the trade after 30 minutes :
54% make 2% profit before 30 minutes -> ok
31% touch the stop loss not depending time duration -> ok
14% of the trades are sold after 30 minutes if roi >= 0.
         -> ok if trade open since 30 minutes and current  loss is 1% -> at 40 minutes going up and roi 0 reach !
         -> nok if trade is winning at 30 minutes (current profit at bar close is 1.5%) but report as 0% in backtest.

```
 minimal_roi = {'0': 0.02, '30': 0}
 stoploss = -0.02
```

> ![image](https://user-images.githubusercontent.com/2643583/65308693-e5ddd100-db8a-11e9-9550-2be69f357bd9.png)

Many tks.

Have a nice day.


I think  "sell-reason reporting" is the real problem, since your sell-reason is always ROI after 30 minutes (since you're abusing it with negative values). 

I think you're misinterpreting sell_reason.
This column tells you why the bot sold (because of your ROI configuration) - not necessarily if it's positive or negative. You can also have trailing-stop as reason, which can be positive or negative, but the reason will still be trailing-stoploss - no matter if it's + or - 5%.

1)
your first configuration seems to work, but produces a huge loss after 30 minutes due to "roi". 
Ignore the sell-reason column for now and load the backtest-results file to pandas as shown in the documentation (i assume you're doing that here anyway, according to the screenshots...)

Assume you load the trades, then override sell_reason with `backtest_df.loc[backtest_df['profitperc'] < 0, 'sell_reason'] = "negative_roi"`, and then do a `value_counts()` grouping by that column again (which will produce the same output than backtesting, but with fixed sell-reasons.

This should produce "some" positive results (maybe not - then your buy-signal is not suitable for 30min trades)...

2)
I'm not certain about how to interpret your second results... you're counting over "profitperc" - so it's telling me that 13% of your trades were sold with 0 profit. Why should this not be possible?





Thinking about 2 again, i know the reason for this too:
backtesting always uses the "roi" value as sell-value. The reason for this is simple - assume your ROI setting is 2% - but "high" was 5% above buy price. 
in Dry/live runs, your bot would have sold at 2%, and not 5% - so using high is wrong, same with close (which could be 4% - but the bot sells as soon as 2% roi is hit).

Now in dry/live runs, you would not see 2% but 2.01% or so (depending on the sell ordertype, how fast the market moves etc) ... but simulating this reliably in backtesting is impossible, since we lack the information for it.

hope this helps explaining this.
Hi,

the problematic i have is solve after n bar/n minutes no matter loss/profit. i would like to manage this with 

```
minimal_roi = {'30':-1}
stop_loss = -1
```

After doing that he stop the trade at 30 minutes but take the roi value : -1 as the profit (-99.9% per trade) and not the close-open value to calculate profit. 

![image](https://user-images.githubusercontent.com/2643583/65606138-9f351000-dfaa-11e9-9061-b1e72fa016d2.png)


2/ Here it is the same problem but with more realistic use case  i want to try closing the trade after 30 minutes :
54% make 2% profit before 30 minutes -> ok
31% touch the stop loss not depending time duration -> ok
14% of the trades are sold after 30 minutes if roi >= 0.
         -> ok if trade open since 30 minutes and current  loss is 1% -> at 40 minutes going up and roi 0 reach !
         -> nok if trade is winning at 30 minutes (current profit at bar close is 1.5%) but report as 0% in backtest.

```
 minimal_roi = {'0': 0.02, '30': 0}
 stoploss = -0.02
```

> ![image](https://user-images.githubusercontent.com/2643583/65308693-e5ddd100-db8a-11e9-9550-2be69f357bd9.png)

Many tks.

Have a nice day.



Resolving patch:
diff --git a/freqtrade/optimize/backtesting.py b/freqtrade/optimize/backtesting.py
--- a/freqtrade/optimize/backtesting.py
+++ b/freqtrade/optimize/backtesting.py
@@ -267,6 +267,11 @@ def _get_sell_trade_entry(
                         # - (Expected abs profit + open_rate + open_fee) / (fee_close -1)
                         closerate = - (trade.open_rate * roi + trade.open_rate *
                                        (1 + trade.fee_open)) / (trade.fee_close - 1)
+
+                        # Use the maximum between closerate and low as we
+                        # cannot sell outside of a candle.
+                        # Applies when using {"xx": -1} as roi to force sells after xx minutes
+                        closerate = max(closerate, sell_row.low)
                     else:
                         # This should not be reached...
                         closerate = sell_row.open


Test patch:
diff --git a/tests/edge/test_edge.py b/tests/edge/test_edge.py
--- a/tests/edge/test_edge.py
+++ b/tests/edge/test_edge.py
@@ -79,7 +79,7 @@ def _time_on_candle(number):
     # D  O     H     L     C     V    B  S
     [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
     [1, 5000, 5025, 4975, 4987, 6172, 0, 1]],  # enter trade (signal on last candle)
-    stop_loss=-0.99, roi=float('inf'), profit_perc=0.00,
+    stop_loss=-0.99, roi={"0": float('inf')}, profit_perc=0.00,
     trades=[]
 )
 
@@ -94,7 +94,7 @@ def _time_on_candle(number):
     [5, 5000, 5025, 4975, 4987, 6172, 0, 1],  # no action
     [6, 5000, 5025, 4975, 4987, 6172, 0, 0],  # should sell
 ],
-    stop_loss=-0.99, roi=float('inf'), profit_perc=0.00,
+    stop_loss=-0.99, roi={"0": float('inf')}, profit_perc=0.00,
     trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=2),
             BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=4, close_tick=6)]
 )
@@ -106,7 +106,7 @@ def _time_on_candle(number):
     [1, 5000, 5025, 4600, 4987, 6172, 0, 0],  # enter trade, stoploss hit
     [2, 5000, 5025, 4975, 4987, 6172, 0, 0],
 ],
-    stop_loss=-0.01, roi=float('inf'), profit_perc=-0.01,
+    stop_loss=-0.01, roi={"0": float('inf')}, profit_perc=-0.01,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
 )
 
@@ -117,7 +117,7 @@ def _time_on_candle(number):
     [1, 5000, 5025, 4800, 4987, 6172, 0, 0],  # enter trade, stoploss hit
     [2, 5000, 5025, 4975, 4987, 6172, 0, 0],
 ],
-    stop_loss=-0.03, roi=float('inf'), profit_perc=-0.03,
+    stop_loss=-0.03, roi={"0": float('inf')}, profit_perc=-0.03,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
 )
 
@@ -128,7 +128,7 @@ def _time_on_candle(number):
     [1, 5000, 5025, 4800, 4987, 6172, 0, 1],  # enter trade, stoploss hit, sell signal
     [2, 5000, 5025, 4975, 4987, 6172, 0, 0],
 ],
-    stop_loss=-0.03, roi=float('inf'), profit_perc=-0.03,
+    stop_loss=-0.03, roi={"0": float('inf')}, profit_perc=-0.03,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
 )
 
diff --git a/tests/optimize/__init__.py b/tests/optimize/__init__.py
--- a/tests/optimize/__init__.py
+++ b/tests/optimize/__init__.py
@@ -1,4 +1,4 @@
-from typing import NamedTuple, List
+from typing import Dict, List, NamedTuple
 
 import arrow
 from pandas import DataFrame
@@ -25,7 +25,7 @@ class BTContainer(NamedTuple):
     """
     data: List[float]
     stop_loss: float
-    roi: float
+    roi: Dict[str, float]
     trades: List[BTrade]
     profit_perc: float
     trailing_stop: bool = False
diff --git a/tests/optimize/test_backtest_detail.py b/tests/optimize/test_backtest_detail.py
--- a/tests/optimize/test_backtest_detail.py
+++ b/tests/optimize/test_backtest_detail.py
@@ -22,7 +22,7 @@
     [3, 5010, 5000, 4980, 5010, 6172, 0, 1],
     [4, 5010, 4987, 4977, 4995, 6172, 0, 0],
     [5, 4995, 4995, 4995, 4950, 6172, 0, 0]],
-    stop_loss=-0.01, roi=1, profit_perc=0.002, use_sell_signal=True,
+    stop_loss=-0.01, roi={"0": 1}, profit_perc=0.002, use_sell_signal=True,
     trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=4)]
 )
 
@@ -36,7 +36,7 @@
     [3, 4975, 5000, 4980, 4977, 6172, 0, 0],
     [4, 4977, 4987, 4977, 4995, 6172, 0, 0],
     [5, 4995, 4995, 4995, 4950, 6172, 0, 0]],
-    stop_loss=-0.01, roi=1, profit_perc=-0.01,
+    stop_loss=-0.01, roi={"0": 1}, profit_perc=-0.01,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)]
 )
 
@@ -51,7 +51,7 @@
     [3, 4975, 5000, 4800, 4962, 6172, 0, 0],  # exit with stoploss hit
     [4, 4962, 4987, 4937, 4950, 6172, 0, 0],
     [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
-    stop_loss=-0.03, roi=1, profit_perc=-0.03,
+    stop_loss=-0.03, roi={"0": 1}, profit_perc=-0.03,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=3)]
 )
 
@@ -71,7 +71,7 @@
     [4, 4975, 5000, 4950, 4962, 6172, 0, 0],  # enter trade 2 (signal on last candle)
     [5, 4962, 4987, 4000, 4000, 6172, 0, 0],  # exit with stoploss hit
     [6, 4950, 4975, 4975, 4950, 6172, 0, 0]],
-    stop_loss=-0.02, roi=1, profit_perc=-0.04,
+    stop_loss=-0.02, roi={"0": 1}, profit_perc=-0.04,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2),
             BTrade(sell_reason=SellType.STOP_LOSS, open_tick=4, close_tick=5)]
 )
@@ -88,7 +88,7 @@
     [3, 4975, 5000, 4950, 4962, 6172, 0, 0],
     [4, 4962, 4987, 4937, 4950, 6172, 0, 0],
     [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
-    stop_loss=-0.02, roi=0.06, profit_perc=-0.02,
+    stop_loss=-0.02, roi={"0": 0.06}, profit_perc=-0.02,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)]
 )
 
@@ -102,7 +102,7 @@
     [3, 4975, 6000, 4975, 6000, 6172, 0, 0],  # ROI
     [4, 4962, 4987, 4972, 4950, 6172, 0, 0],
     [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
-    stop_loss=-0.01, roi=0.03, profit_perc=0.03,
+    stop_loss=-0.01, roi={"0": 0.03}, profit_perc=0.03,
     trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=3)]
 )
 
@@ -116,7 +116,7 @@
     [3, 4975, 5000, 4950, 4962, 6172, 0, 0],
     [4, 4962, 4987, 4972, 4950, 6172, 0, 0],
     [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
-    stop_loss=-0.02, roi=0.05, profit_perc=-0.02,
+    stop_loss=-0.02, roi={"0": 0.05}, profit_perc=-0.02,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)]
 )
 
@@ -130,7 +130,7 @@
     [3, 4975, 5000, 4950, 4962, 6172, 0, 0],
     [4, 4962, 4987, 4972, 4950, 6172, 0, 0],
     [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
-    stop_loss=-0.02, roi=0.03, profit_perc=0.03,
+    stop_loss=-0.02, roi={"0": 0.03}, profit_perc=0.03,
     trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=2)]
 )
 
@@ -144,7 +144,7 @@
     [2, 5000, 5250, 4750, 4850, 6172, 0, 0],
     [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.10, profit_perc=-0.055, trailing_stop=True,
+    stop_loss=-0.10, roi={"0": 0.10}, profit_perc=-0.055, trailing_stop=True,
     trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
 )
 
@@ -158,7 +158,7 @@
     [2, 5000, 5050, 4950, 5000, 6172, 0, 0],
     [3, 5000, 5200, 4550, 4850, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.10, profit_perc=-0.064, trailing_stop=True,
+    stop_loss=-0.10, roi={"0": 0.10}, profit_perc=-0.064, trailing_stop=True,
     trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
 )
 
@@ -172,7 +172,7 @@
     [2, 5100, 5251, 5100, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.10, profit_perc=-0.1, trailing_stop=True,
+    stop_loss=-0.10, roi={"0": 0.10}, profit_perc=-0.1, trailing_stop=True,
     trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.10,
     trailing_stop_positive=0.03,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=4)]
@@ -188,7 +188,7 @@
     [2, 5100, 5251, 5100, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.10, profit_perc=0.019, trailing_stop=True,
+    stop_loss=-0.10, roi={"0": 0.10}, profit_perc=0.019, trailing_stop=True,
     trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.05,
     trailing_stop_positive=0.03,
     trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
@@ -204,7 +204,7 @@
     [2, 5100, 5251, 4650, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.10, profit_perc=0.019, trailing_stop=True,
+    stop_loss=-0.10, roi={"0": 0.10}, profit_perc=0.019, trailing_stop=True,
     trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.05,
     trailing_stop_positive=0.03,
     trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=2)]
@@ -219,7 +219,7 @@
     [2, 5100, 5251, 4850, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4850, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4850, 4750, 6172, 0, 0]],
-    stop_loss=-0.10, roi=0.01, profit_perc=0.01,
+    stop_loss=-0.10, roi={"0": 0.01}, profit_perc=0.01,
     trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=1)]
 )
 
@@ -232,7 +232,7 @@
     [2, 5100, 5251, 4850, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4850, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.05, roi=0.10, profit_perc=-0.05,
+    stop_loss=-0.05, roi={"0": 0.10}, profit_perc=-0.05,
     trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
 )
 
@@ -246,11 +246,26 @@
     [2, 5100, 5251, 4650, 5100, 6172, 0, 0],
     [3, 4850, 5050, 4850, 4750, 6172, 0, 0],
     [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
-    stop_loss=-0.05, roi=0.01, profit_perc=-0.04,
+    stop_loss=-0.05, roi={"0": 0.01}, profit_perc=-0.04,
     trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=1),
             BTrade(sell_reason=SellType.STOP_LOSS, open_tick=2, close_tick=2)]
 )
 
+# Test 16: Buy, hold for 65 mins, then forcesell using roi=-1
+# Causes negative profit even though sell-reason is ROI.
+# stop-loss: 10%, ROI: 10% (should not apply), -100% after 65 minutes (limits trade duration)
+tc16 = BTContainer(data=[
+    # D  O     H     L     C     V    B  S
+    [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
+    [1, 5000, 5025, 4975, 4987, 6172, 0, 0],
+    [2, 4987, 5300, 4950, 5050, 6172, 0, 0],
+    [3, 4975, 5000, 4940, 4962, 6172, 0, 0],  # ForceSell on ROI (roi=-1)
+    [4, 4962, 4987, 4972, 4950, 6172, 0, 0],
+    [5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
+    stop_loss=-0.10, roi={"0": 0.10, "65": -1}, profit_perc=-0.012,
+    trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=3)]
+)
+
 TESTS = [
     tc0,
     tc1,
@@ -268,6 +283,7 @@
     tc13,
     tc14,
     tc15,
+    tc16,
 ]
 
 
@@ -277,7 +293,7 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
     run functional tests
     """
     default_conf["stoploss"] = data.stop_loss
-    default_conf["minimal_roi"] = {"0": data.roi}
+    default_conf["minimal_roi"] = data.roi
     default_conf["ticker_interval"] = tests_ticker_interval
     default_conf["trailing_stop"] = data.trailing_stop
     default_conf["trailing_only_offset_is_reached"] = data.trailing_only_offset_is_reached