Repository: freqtrade/freqtrade

Issue:
Add fill_holes_in_book option to config file so the bot place orders between two orders on the orderbook
Now that we have or will soon have the ability to place orders using the orderbook an not only the top order another useful feature would be the ability for the bot to fill holes on the orderbook, what it means is that if the bot bought B2G at 174.5 USD but there are only two orders close to that price, one is at 174 USD and the other one at 175 USD, if we open a new order for a few cents lower than the one at 175 USD for example at 174.9 USD we will be one position higher on the orderbook and most likely the volume of our order will be smaller than the one at 175 USD meaning that our order will have a higher chance to get filled before the one at 175 USD.


Discussion:
This is a strategy not a core bot configuration. 
The place to manage targetting where to place buys and sells is in a strategy file. 
That should be a core function in my opinion but it can be easily added as a true/false option on the config just to say if we want to use it or not and then on the strategy it can be modified to make it better and improve the decision of where to place the order exactly.
its a buying strategy 
we could put all buy/sell strategy options into config.json with true/false. 

There is no difference between make buy when see a  gaps in order book than buy when crosses sma21. 

the bot should provide the order book as it should be able to provide the SMA
A gap, or step volume, on the order book is just an indicator, like any other 

The analysis of that indicator and buying decision is for a strategy to trigger on.

I would propose an "orderbook step" indicator be written that takes two argument of ceiling from spot and step 

e.g if btc is at 7,000 the ceiling may be $10, the step $0.50, it will provide back order volume for each 0.50$ step upto $10 above spot. 

This will allow  a strategy to determine where is currently best to put an order, ahead of a buy wall as example. If there are 0 orders on a step, this would be an empty step to put a limit order on.
Slightly off topic, but of value.
The community should be aware of icebergs  - large hidden orders on the book.
Most exchanges support paying market rate on a limit order to hide the order from the book. 

Buying into an empty step may well be others have spotted a hidden buy wall and are avoiding it.

I agree with the hidden orders for example Hitbtc allows that, for the rest of what you said I agree partially with you that the strategy should be able to handle all that stuff but if the fill holes thing will be handled by the strategies I recommend that its added to the default strategy and that just like other values like minimal_roi we can override it if we set a value for that on the config file, again, I agree with you that everything related to buy/sell should be handled by the strategy but some values should be easy to edit on the config file for those people who dont want to mess with the strategy directly and just want to change a few values easily.

Resolving patch:
diff --git a/freqtrade/constants.py b/freqtrade/constants.py
--- a/freqtrade/constants.py
+++ b/freqtrade/constants.py
@@ -191,6 +191,9 @@
             },
             'required': ['price_side']
         },
+        'custom_price_max_distance_ratio': {
+           'type': 'number', 'minimum': 0.0
+        },
         'order_types': {
             'type': 'object',
             'properties': {
diff --git a/freqtrade/freqtradebot.py b/freqtrade/freqtradebot.py
--- a/freqtrade/freqtradebot.py
+++ b/freqtrade/freqtradebot.py
@@ -479,7 +479,13 @@ def execute_buy(self, pair: str, stake_amount: float, price: Optional[float] = N
             buy_limit_requested = price
         else:
             # Calculate price
-            buy_limit_requested = self.exchange.get_rate(pair, refresh=True, side="buy")
+            proposed_buy_rate = self.exchange.get_rate(pair, refresh=True, side="buy")
+            custom_entry_price = strategy_safe_wrapper(self.strategy.custom_entry_price,
+                                                       default_retval=proposed_buy_rate)(
+                pair=pair, current_time=datetime.now(timezone.utc),
+                proposed_rate=proposed_buy_rate)
+
+            buy_limit_requested = self.get_valid_price(custom_entry_price, proposed_buy_rate)
 
         if not buy_limit_requested:
             raise PricingError('Could not determine buy price.')
@@ -1076,6 +1082,17 @@ def execute_sell(self, trade: Trade, limit: float, sell_reason: SellCheckTuple)
            and self.strategy.order_types['stoploss_on_exchange']:
             limit = trade.stop_loss
 
+        # set custom_exit_price if available
+        proposed_limit_rate = limit
+        current_profit = trade.calc_profit_ratio(limit)
+        custom_exit_price = strategy_safe_wrapper(self.strategy.custom_exit_price,
+                                                  default_retval=proposed_limit_rate)(
+            pair=trade.pair, trade=trade,
+            current_time=datetime.now(timezone.utc),
+            proposed_rate=proposed_limit_rate, current_profit=current_profit)
+
+        limit = self.get_valid_price(custom_exit_price, proposed_limit_rate)
+
         # First cancelling stoploss on exchange ...
         if self.strategy.order_types.get('stoploss_on_exchange') and trade.stoploss_order_id:
             try:
@@ -1375,3 +1392,26 @@ def fee_detection_from_trades(self, trade: Trade, order: Dict, order_amount: flo
                                               amount=amount, fee_abs=fee_abs)
         else:
             return amount
+
+    def get_valid_price(self, custom_price: float, proposed_price: float) -> float:
+        """
+        Return the valid price.
+        Check if the custom price is of the good type if not return proposed_price
+        :return: valid price for the order
+        """
+        if custom_price:
+            try:
+                valid_custom_price = float(custom_price)
+            except ValueError:
+                valid_custom_price = proposed_price
+        else:
+            valid_custom_price = proposed_price
+
+        cust_p_max_dist_r = self.config.get('custom_price_max_distance_ratio', 0.02)
+        min_custom_price_allowed = proposed_price - (proposed_price * cust_p_max_dist_r)
+        max_custom_price_allowed = proposed_price + (proposed_price * cust_p_max_dist_r)
+
+        # Bracket between min_custom_price_allowed and max_custom_price_allowed
+        return max(
+            min(valid_custom_price, max_custom_price_allowed),
+            min_custom_price_allowed)
diff --git a/freqtrade/strategy/interface.py b/freqtrade/strategy/interface.py
--- a/freqtrade/strategy/interface.py
+++ b/freqtrade/strategy/interface.py
@@ -280,6 +280,43 @@ def custom_stoploss(self, pair: str, trade: Trade, current_time: datetime, curre
         """
         return self.stoploss
 
+    def custom_entry_price(self, pair: str, current_time: datetime, proposed_rate: float,
+                           **kwargs) -> float:
+        """
+        Custom entry price logic, returning the new entry price.
+
+        For full documentation please go to https://www.freqtrade.io/en/latest/strategy-advanced/
+
+        When not implemented by a strategy, returns None, orderbook is used to set entry price
+
+        :param pair: Pair that's currently analyzed
+        :param current_time: datetime object, containing the current datetime
+        :param proposed_rate: Rate, calculated based on pricing settings in ask_strategy.
+        :param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
+        :return float: New entry price value if provided
+        """
+        return proposed_rate
+
+    def custom_exit_price(self, pair: str, trade: Trade,
+                          current_time: datetime, proposed_rate: float,
+                          current_profit: float, **kwargs) -> float:
+        """
+        Custom exit price logic, returning the new exit price.
+
+        For full documentation please go to https://www.freqtrade.io/en/latest/strategy-advanced/
+
+        When not implemented by a strategy, returns None, orderbook is used to set exit price
+
+        :param pair: Pair that's currently analyzed
+        :param trade: trade object.
+        :param current_time: datetime object, containing the current datetime
+        :param proposed_rate: Rate, calculated based on pricing settings in ask_strategy.
+        :param current_profit: Current profit (as ratio), calculated based on current_rate.
+        :param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
+        :return float: New exit price value if provided
+        """
+        return proposed_rate
+
     def custom_sell(self, pair: str, trade: Trade, current_time: datetime, current_rate: float,
                     current_profit: float, **kwargs) -> Optional[Union[str, bool]]:
         """


Test patch:
diff --git a/tests/test_freqtradebot.py b/tests/test_freqtradebot.py
--- a/tests/test_freqtradebot.py
+++ b/tests/test_freqtradebot.py
@@ -904,6 +904,40 @@ def test_execute_buy(mocker, default_conf, fee, limit_buy_order, limit_buy_order
     with pytest.raises(PricingError, match="Could not determine buy price."):
         freqtrade.execute_buy(pair, stake_amount)
 
+    # In case of custom entry price
+    mocker.patch('freqtrade.exchange.Exchange.get_rate', return_value=0.50)
+    limit_buy_order['status'] = 'open'
+    limit_buy_order['id'] = '5566'
+    freqtrade.strategy.custom_entry_price = lambda **kwargs: 0.508
+    assert freqtrade.execute_buy(pair, stake_amount)
+    trade = Trade.query.all()[6]
+    assert trade
+    assert trade.open_rate_requested == 0.508
+
+    # In case of custom entry price set to None
+    limit_buy_order['status'] = 'open'
+    limit_buy_order['id'] = '5567'
+    freqtrade.strategy.custom_entry_price = lambda **kwargs: None
+
+    mocker.patch.multiple(
+        'freqtrade.exchange.Exchange',
+        get_rate=MagicMock(return_value=10),
+    )
+
+    assert freqtrade.execute_buy(pair, stake_amount)
+    trade = Trade.query.all()[7]
+    assert trade
+    assert trade.open_rate_requested == 10
+
+    # In case of custom entry price not float type
+    limit_buy_order['status'] = 'open'
+    limit_buy_order['id'] = '5568'
+    freqtrade.strategy.custom_entry_price = lambda **kwargs: "string price"
+    assert freqtrade.execute_buy(pair, stake_amount)
+    trade = Trade.query.all()[8]
+    assert trade
+    assert trade.open_rate_requested == 10
+
 
 def test_execute_buy_confirm_error(mocker, default_conf, fee, limit_buy_order) -> None:
     freqtrade = get_patched_freqtradebot(mocker, default_conf)
@@ -2715,6 +2749,70 @@ def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, mocker)
     } == last_msg
 
 
+def test_execute_sell_custom_exit_price(default_conf, ticker, fee, ticker_sell_up, mocker) -> None:
+    rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
+    mocker.patch.multiple(
+        'freqtrade.exchange.Exchange',
+        fetch_ticker=ticker,
+        get_fee=fee,
+        _is_dry_limit_order_filled=MagicMock(return_value=False),
+    )
+    patch_whitelist(mocker, default_conf)
+    freqtrade = FreqtradeBot(default_conf)
+    patch_get_signal(freqtrade)
+    freqtrade.strategy.confirm_trade_exit = MagicMock(return_value=False)
+
+    # Create some test data
+    freqtrade.enter_positions()
+    rpc_mock.reset_mock()
+
+    trade = Trade.query.first()
+    assert trade
+    assert freqtrade.strategy.confirm_trade_exit.call_count == 0
+
+    # Increase the price and sell it
+    mocker.patch.multiple(
+        'freqtrade.exchange.Exchange',
+        fetch_ticker=ticker_sell_up
+    )
+
+    freqtrade.strategy.confirm_trade_exit = MagicMock(return_value=True)
+
+    # Set a custom exit price
+    freqtrade.strategy.custom_exit_price = lambda **kwargs: 1.170e-05
+
+    freqtrade.execute_sell(trade=trade, limit=ticker_sell_up()['bid'],
+                           sell_reason=SellCheckTuple(sell_type=SellType.SELL_SIGNAL))
+
+    # Sell price must be different to default bid price
+
+    assert freqtrade.strategy.confirm_trade_exit.call_count == 1
+
+    assert rpc_mock.call_count == 1
+    last_msg = rpc_mock.call_args_list[-1][0][0]
+    assert {
+        'trade_id': 1,
+        'type': RPCMessageType.SELL,
+        'exchange': 'Binance',
+        'pair': 'ETH/BTC',
+        'gain': 'profit',
+        'limit': 1.170e-05,
+        'amount': 91.07468123,
+        'order_type': 'limit',
+        'open_rate': 1.098e-05,
+        'current_rate': 1.173e-05,
+        'profit_amount': 6.041e-05,
+        'profit_ratio': 0.06025919,
+        'stake_currency': 'BTC',
+        'fiat_currency': 'USD',
+        'sell_reason': SellType.SELL_SIGNAL.value,
+        'open_date': ANY,
+        'close_date': ANY,
+        'close_rate': ANY,
+    } == last_msg
+
+
 def test_execute_sell_down_stoploss_on_exchange_dry_run(default_conf, ticker, fee,
                                                         ticker_sell_down, mocker) -> None:
     rpc_mock = patch_RPCManager(mocker)
@@ -4491,3 +4589,43 @@ def reset_open_orders(trade):
 
     freqtrade.refind_lost_order(trades[4])
     assert log_has(f"Error updating {order['id']}.", caplog)
+
+
+def test_get_valid_price(mocker, default_conf) -> None:
+    patch_RPCManager(mocker)
+    patch_exchange(mocker)
+    freqtrade = FreqtradeBot(default_conf)
+    freqtrade.config['custom_price_max_distance_ratio'] = 0.02
+
+    custom_price_string = "10"
+    custom_price_badstring = "10abc"
+    custom_price_float = 10.0
+    custom_price_int = 10
+
+    custom_price_over_max_alwd = 11.0
+    custom_price_under_min_alwd = 9.0
+    proposed_price = 10.1
+
+    valid_price_from_string = freqtrade.get_valid_price(custom_price_string, proposed_price)
+    valid_price_from_badstring = freqtrade.get_valid_price(custom_price_badstring, proposed_price)
+    valid_price_from_int = freqtrade.get_valid_price(custom_price_int, proposed_price)
+    valid_price_from_float = freqtrade.get_valid_price(custom_price_float, proposed_price)
+
+    valid_price_at_max_alwd = freqtrade.get_valid_price(custom_price_over_max_alwd, proposed_price)
+    valid_price_at_min_alwd = freqtrade.get_valid_price(custom_price_under_min_alwd, proposed_price)
+
+    assert isinstance(valid_price_from_string, float)
+    assert isinstance(valid_price_from_badstring, float)
+    assert isinstance(valid_price_from_int, float)
+    assert isinstance(valid_price_from_float, float)
+
+    assert valid_price_from_string == custom_price_float
+    assert valid_price_from_badstring == proposed_price
+    assert valid_price_from_int == custom_price_int
+    assert valid_price_from_float == custom_price_float
+
+    assert valid_price_at_max_alwd < custom_price_over_max_alwd
+    assert valid_price_at_max_alwd > proposed_price
+
+    assert valid_price_at_min_alwd > custom_price_under_min_alwd
+    assert valid_price_at_min_alwd < proposed_price