Repository: freqtrade/freqtrade

Issue:
Incoherent sell order, negative price in dry run
<!-- 
Have you searched for similar issues before posting it?

If you have discovered a bug in the bot, please [search our issue tracker](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue). 
If it hasn't been reported, please create a new issue.

Please do not use bug reports to request new features.
-->

## Describe your environment

  * Operating system: Docker 20.10.3 linux/amd64
  * Python Version: 3.9.6
  * CCXT version: 1.53.72
  * Freqtrade Version: 2021.7
  
Note: All issues other than enhancement requests will be closed without further comment if the above template is deleted or not filled out.

## Describe the problem:

Hello !
I have several bots running on the same strategy (NFIv7, see attached file below) in live mode, and another on dry mode.

[NFIv7.py.txt](https://github.com/freqtrade/freqtrade/files/6960765/NFIv7.py.txt)


On dry mode, a sell order has been made at a negative rate :

> ❌ Binance: Selling DEGO/BUSD (#1251)
Profit: -141.26% (loss: -2403.59637244 BUSD / -2049.410 EUR)
Sell Reason: signal_profit_u_e_1
Duration: 3:39:59 (220.0 min)
Amount: 153.31500000
Open Rate: 11.08700000
Current Rate: 11.11100000
Close Rate: -4.58400000

In live mode, it seems Ok :

> ✳ Binance: Selling DEGO/BUSD (#1035)
Profit: 0.09% (profit: 0.07120012 BUSD / 0.060 EUR)
Sell Reason: signal_profit_u_e_1
Duration: 3:39:46 (219.8 min)
Amount: 6.87400000
Open Rate: 11.08300000
Current Rate: 11.12700000
Close Rate: 11.11555644


### Relevant code exceptions or logs

```
ing Sell for DEGO/BUSD. Reason: custom_sell
2021-08-09 19:31:03,412 - freqtrade.wallets - INFO - Wallets synced.
2021-08-09 19:31:04,334 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1251, pair=DEGO/BUSD, amount=153.31500000, open_rate=11.08700000, open_since=2021-08-09 15:51:05)
2021-08-09 19:31:04,552 - freqtrade.freqtradebot - INFO - Fee for Trade Trade(id=1251, pair=DEGO/BUSD, amount=153.31500000, open_rate=11.08700000, open_since=2021-08-09 15:51:05) [sell]: -0.70279596 BUSD - rate: 0.001
2021-08-09 19:31:04,553 - freqtrade.persistence.models - INFO - Updating trade (id=1251) ...
2021-08-09 19:31:04,554 - freqtrade.persistence.models - INFO - MARKET_SELL has been fulfilled for Trade(id=1251, pair=DEGO/BUSD, amount=153.31500000, open_rate=11.08700000, open_since=2021-08-09 15:51:05).
2021-08-09 19:31:04,554 - freqtrade.persistence.models - INFO - Marking Trade(id=1251, pair=DEGO/BUSD, amount=153.31500000, open_rate=11.08700000, open_since=closed) as closed as the trade is fulfilled and found no open orders for it.
2021-08-09 19:31:04,742 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': sell_fill, 'trade_id': 1251, 'exchange': 'Binance', 'pair': 'DEGO/BUSD', 'gain': 'loss', 'limit': -4.584, 'order_type': '', 'amount': 153.315, 'open_rate': 11.087, 'close_rate': -4.584, 'current_rate': None, 'profit_amount': -2403.59637244, 'profit_ratio': -1.41263111, 'sell_reason': 'signal_profit_u_e_1', 'open_date': datetime.datetime(2021, 8, 9, 15, 51, 5, 863631), 'close_date': datetime.datetime(2021, 8, 9, 19, 31, 4, 554575), 'stake_currency': 'BUSD', 'fiat_currency': 'EUR'}
2021-08-09 19:31:04,743 - freqtrade.rpc.telegram - INFO - Notification 'sell_fill' not sent.
2021-08-09 19:31:04,748 - freqtrade.wallets - INFO - Wallets synced.
2021-08-09 19:31:04,954 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': sell, 'trade_id': 1251, 'exchange': 'Binance', 'pair': 'DEGO/BUSD', 'gain': 'loss', 'limit': -4.584, 'order_type': 'market', 'amount': 153.315, 'open_rate': 11.087, 'close_rate': -4.584, 'current_rate': 11.111, 'profit_amount': -2403.59637244, 'profit_ratio': -1.41263111, 'sell_reason': 'signal_profit_u_e_1', 'open_date': datetime.datetime(2021, 8, 9, 15, 51, 5, 863631), 'close_date': datetime.datetime(2021, 8, 9, 19, 31, 4, 554575), 'stake_currency': 'BUSD', 'fiat_currency': 'EUR'}
2021-08-09 19:31:06,076 - freqtrade.wallets - INFO - Wallets synced.
2021-08-09 19:31:06,638 - freqtrade.freqtradebot - INFO - Pair DEGO/BUSD is still locked until 2021-08-09 19:35:00 due to Auto lock.
```

Here is a screenshot of the UI:

![Freqtrade UI DEGO trade close](https://user-images.githubusercontent.com/3023499/128849839-29362fc9-3ad4-41fb-bf37-40d4b85b9dab.png)

The logs are not so helpful so I would be glad to provide any more relevant information if needed.



Discussion:
so just to confirm - this did happen in dry-run, not in a live run?

In dry-run, we're now (trying) to simulate fill-behaviour for both market an dlimit orders.
Unfortunately, that's heavily dependent on the orderbook that's available - and if the orderbook is "pretty" empty, it can lead to humongous slippage.
Hi,

This happened to me as well in dry-run mode. Was so strange and was ,,added" to profit.

Thanx
Yes, it happened only in dry mode. In my first post, I posted the "same" transaction that happened on a bot in live mode with the same config (but not the same stake amount :>). More frightened than hurt!

Resolving patch:
diff --git a/freqtrade/exchange/exchange.py b/freqtrade/exchange/exchange.py
--- a/freqtrade/exchange/exchange.py
+++ b/freqtrade/exchange/exchange.py
@@ -618,6 +618,8 @@ def get_dry_market_fill_price(self, pair: str, side: str, amount: float, rate: f
         if self.exchange_has('fetchL2OrderBook'):
             ob = self.fetch_l2_order_book(pair, 20)
             ob_type = 'asks' if side == 'buy' else 'bids'
+            slippage = 0.05
+            max_slippage_val = rate * ((1 + slippage) if side == 'buy' else (1 - slippage))
 
             remaining_amount = amount
             filled_amount = 0
@@ -626,7 +628,9 @@ def get_dry_market_fill_price(self, pair: str, side: str, amount: float, rate: f
                 book_entry_coin_volume = book_entry[1]
                 if remaining_amount > 0:
                     if remaining_amount < book_entry_coin_volume:
+                        # Orderbook at this slot bigger than remaining amount
                         filled_amount += remaining_amount * book_entry_price
+                        break
                     else:
                         filled_amount += book_entry_coin_volume * book_entry_price
                     remaining_amount -= book_entry_coin_volume
@@ -635,7 +639,14 @@ def get_dry_market_fill_price(self, pair: str, side: str, amount: float, rate: f
             else:
                 # If remaining_amount wasn't consumed completely (break was not called)
                 filled_amount += remaining_amount * book_entry_price
-            forecast_avg_filled_price = filled_amount / amount
+            forecast_avg_filled_price = max(filled_amount, 0) / amount
+            # Limit max. slippage to specified value
+            if side == 'buy':
+                forecast_avg_filled_price = min(forecast_avg_filled_price, max_slippage_val)
+
+            else:
+                forecast_avg_filled_price = max(forecast_avg_filled_price, max_slippage_val)
+
             return self.price_to_precision(pair, forecast_avg_filled_price)
 
         return rate


Test patch:
diff --git a/tests/exchange/test_exchange.py b/tests/exchange/test_exchange.py
--- a/tests/exchange/test_exchange.py
+++ b/tests/exchange/test_exchange.py
@@ -984,16 +984,21 @@ def test_create_dry_run_order_limit_fill(default_conf, mocker, side, startprice,
     assert order['fee']
 
 
-@pytest.mark.parametrize("side,amount,endprice", [
-    ("buy", 1, 25.566),
-    ("buy", 100, 25.5672),  # Requires interpolation
-    ("buy", 1000, 25.575),  # More than orderbook return
-    ("sell", 1, 25.563),
-    ("sell", 100, 25.5625),  # Requires interpolation
-    ("sell", 1000, 25.5555),  # More than orderbook return
+@pytest.mark.parametrize("side,rate,amount,endprice", [
+    # spread is 25.263-25.266
+    ("buy", 25.564, 1, 25.566),
+    ("buy", 25.564, 100, 25.5672),  # Requires interpolation
+    ("buy", 25.590, 100, 25.5672),  # Price above spread ... average is lower
+    ("buy", 25.564, 1000, 25.575),  # More than orderbook return
+    ("buy", 24.000, 100000, 25.200),  # Run into max_slippage of 5%
+    ("sell", 25.564, 1, 25.563),
+    ("sell", 25.564, 100, 25.5625),  # Requires interpolation
+    ("sell", 25.510, 100, 25.5625),  # price below spread - average is higher
+    ("sell", 25.564, 1000, 25.5555),  # More than orderbook return
+    ("sell", 27, 10000, 25.65),  # max-slippage 5%
 ])
 @pytest.mark.parametrize("exchange_name", EXCHANGES)
-def test_create_dry_run_order_market_fill(default_conf, mocker, side, amount, endprice,
+def test_create_dry_run_order_market_fill(default_conf, mocker, side, rate, amount, endprice,
                                           exchange_name, order_book_l2_usd):
     default_conf['dry_run'] = True
     exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
@@ -1003,7 +1008,7 @@ def test_create_dry_run_order_market_fill(default_conf, mocker, side, amount, en
                           )
 
     order = exchange.create_dry_run_order(
-        pair='LTC/USDT', ordertype='market', side=side, amount=amount, rate=25.5)
+        pair='LTC/USDT', ordertype='market', side=side, amount=amount, rate=rate)
     assert 'id' in order
     assert f'dry_run_{side}_' in order["id"]
     assert order["side"] == side