Repository: freqtrade/freqtrade

Issue:
Penalize: stop trading losing pairs
It would be very helpful if there was a way to stop or pause losing pairs. For example, if a pair caused 2 subsequent losing trades, it might be a good idea to stop trading said pair for a while. 


Discussion:
Just a question, to clarify the idea: assume you 'stopped' a pair. At what conditions should it be returned back to trading? After some timeout or what? Manually by an RPC command?


i think it's a duplicate of #1841. 

However, i'm not convinced that this should be part of the in the bot core logic.

Having this in the strategy will allow "custom" reenable configurations (see below) - which will be very difficult to generalize flexibly enough to fit most users needs.

Possible "following" behaviour just jumping to my mind - in no way complete: 

* completely blacklist this pair - require manual reenabling
* reenable after X candles
* reenable after X other trades
* reenable after indicator Y is > 200
* . . .
* any combination of the above with either OR or AND ...


maybe, the best we can do for this type of problem is to help with documentation / samples, but the "reenable" desires can vary greatly...
What I would love to see is a feature where you can define after how many stop-losses it blacklists the pairs. It will send you a message on Telegram and the user gets to decide what to do with it, either keep it that way or /reload_conf (could be fancier ways, but I'm pretty sure this way is the easiest to implement for you guys. And will just do the job) to set the blacklist to it's initial state. If the user doesn't want this feature because after 2 losses the probability of a gain on that pair is more likely then he simply disables this feature in the config (which should be off by default).

In addition to that I would also like a feature where you can set a fixed stop on the buying after your balance/wallet hits $X.
OR if your balance loses X% from the highest value measured, so basically a trailing_stop_loss. When it does it will notify the user on Telegram and he can manually start the bot again whenever he likes and make the necessary changes.

For me these features would decrease big risks, for example when I'm asleep or not active for a few days (or even hours in this market) and the market suddenly dips so much I lose many pairs in a row and my total balance goes -30%. It would be an amazing feature if the bot stopped buying at -20% of my total balance for example. 
Of course you can say that my strategy is not good then, and that I should have never used it. But that's still something unpredictable, it could happen to the bests strategies.
> i think it's a duplicate of #1841.

Not completely, #1841 as I understand it, is about a custom callback method in the strategy, which is called once at the beginning of every throttling iteration. It can check (custom code) some conditions (like inacceptable total losses, many trades with losses, or the position of the Moon) and return runMode -- stopped -- in case of bad conditions.

This one is on per-pair basis. This can be implemented as a pairlist, which adds the pair to the blacklist for some time in case of N loss trades or in case of losses > M amount...

@abdullahe if you postulate every trade is independent, then there is no reasons to black list a pair it is just probability. Why would you consider losing 2 trades of the same pair is bad. When you play roulette if the number 2 comes twice then you don't put your chips on number 2 ? It is an error because the probability are independents so it is a biais named [Illusory_correlation](https://en.wikipedia.org/wiki/Illusory_correlation)
I disagree with that @gaugau3000, the theory sounds fun and all but in practise I have had many times that the same pairs triggered over 2+ times with a stoploss. Reason for this is a huge bearish market. Some pairs go -80%. If your stop_loss is 5% and your buy signals triggers at that pair (because an indicator shows that it's rising again, but then boom right after you buy it starts collapsing again, and this many times over and over), it could sell many pairs with a huge loss. 

This feature would be here just to decrease risks and big losses.
@gaugau3000 It can be illusory correlation, sure, but in the same time the market condition changes and the strategy may start behaving bad for some pairs. So per-pair emergency check can also have sense in the eyes of the authors of the strategies/users/traders...

I just went to a random pair that I had this issue with for an example. It went in a day up to over 100%. The day after it dipped almost with 80%. On the screen you see before every arrow one or multiple green candles that could trigger buy signals that a bullish market is upcoming. All the red candles at the arrows are dips of around 10-20%! It's arguable whether your strategy should trigger so quick, but these green candles were pretty decent and fast increases.
![dips](https://user-images.githubusercontent.com/24569139/69758121-f39d6d00-115e-11ea-9593-a1b0032d4624.png)

@Theagainmen 
I think you have to manage the retracement straight in your strategy -> your losing trades are correlated because of the retracement drawdown. Why my loss are correlated -> Because of the big pump then dump -> can i handle this in strategy -> yes. This is a limit use case who have a rational reason. When you stop trading after two looses of the same pair you don't know why : is it bad luck or it have a rational reason ? you don't choose and prefer your fear witch can not be considered as a technical indicator.

@hroff-1902 
i think the emergency is most of the time a physiological response not sure this functional have part have sense in freqtrade (when you have multi-bots on correlated market i can understand the emergency part). I think what is difficult in freqtrade about what to develop and what not is we don't have the trading knowledge (no judgement of course) : we are not full time traders.

Have a nice day
@Theagainmen 
By the way you can analyse your trading results at pair level if you have an enough sample that ensure convergence using  [durbin watson statitics](https://www.investopedia.com/terms/d/durbin-watson-statistic.asp)
 you may see that the result of durbin watson is ≈ 2 that means that there is no serial correlation between trades. You can use [durbin_watson_statsmodels_lib](https://www.statsmodels.org/dev/generated/statsmodels.stats.stattools.durbin_watson.html).
> This one is on per-pair basis. This can be implemented as a pairlist, which adds the pair to the blacklist for some time in case of N loss trades or in case of losses > M amount...

Let's be honest - this functionality is there - i added it to support this usecase in the future when adding the "lock until candle end" mechanism - but then forgot about it . . . 

I think this can be done quite easily, within the strategy - following "necessities" a strategy has (different parameters / checks) - so this is a documentation problem in my opinion, since the infrastructure to support it is there.

[partial documentation source](https://www.freqtrade.io/en/latest/strategy-customization/#additional-data-trades)

``` python
from freqtrade.persistence import Trade
from datetime import timedelta, datetime, timezone
# Put the above lines a the top of the strategy file

#Within populate indicators (or populate_buy):
if self.config['runmode'] in ('live', 'dry_run'):
   # fetch trades for the last 2 days
    trades = Trade.get_trades([Trade.pair == metadata['pair'],
                               Trade.open_date > datetime.utcnow() - timedelta(days=2),
                               Trade.is_open == False,
                ]).all()
    # Analyze the conditions you'd like to lock the pair .... will probably be different for every strategy
    sumprofit = sum(trade.close_profit for trade in trades)
    if sumprofit < 0:
        # Lock pair for 2 days
        self.lock_pair(metadata['pair'], until=datetime.now(timezone.utc) + timedelta(days=2))

```

All values are samples only - i'm not sure it makes sense to look at the past 2 days, nor am i certain it makes sense to lock pairs that were negative in the last 2 days.

calling `self.lock_pair()` is the key - this will lock the pair "until" - so everyone can decide how long to lock the pair (or don't).



I want to lock pairs that hit stop loss 2 times in X time period, how would I adapt this in my strategy code? And also make it send a notify message in the Telegram chat?
well - i don't think that's a good filter.
what does 2x stoploss in a day mean? nothing if you had 5 winning trades from the same pair.

you can do it by adjusting the filter and add something like `Trade.sell_reason == 'stoploss'` ...
> I want to lock pairs that hit stop loss 2 times in X time period, how would I adapt this in my strategy code? And also make it send a notify message in the Telegram chat?

The Edge module does something similar (not exactly in these terms): after N loosing trades the pair will receive small win ratio, this changes expectation value, used by Edge ("what does 2x stoploss in a day mean? nothing if you had 5 winning trades from the same pair." -- @xmatthias, you are not quite correct, it changes win rate and expectation value), so the pair will be at the bottom of the edge pairlist.

Actually, Edge also is a kind of pairlist (one part of it), it can be represented as a pairlist in the pairlists list, it was discussed between us, we won't do this due to historical reasons...

Try to configure and use Edge...

> @xmatthias, you are not quite correct, it changes win rate and expectation value)

this issue is not about edge. Using edge is one method - but it's not suitable for every strategy (since it lacks implementations for ROI, trailing stoploss, ... - it's unsuitable for most strategies, if you ask me).

It'll change win-rate and expectation, yes - but not necessarily towards the negative, if you consider 7 trades within 1 day - 5 wining and 2 loosing.
that's different to 2 trades, both loosing.

So i'll stick to my opinion, having 2 loosing trades for one pair is non-conclusive, and requires information about all other pairs too.

> @hroff-1902  At what conditions should it be returned back to trading? After some timeout or what? Manually by an RPC command?

Manually by reloading config maybe? Or giving a penalty time that could be configured in strategy. 

> @gaugau3000 When you play roulette if the number 2 comes twice then you don't put your chips on number 2 ?

I don't think the analogy accurately represents the situation. Strategies cannot be generalized to cover all situations. Usually they are good for a certain set of conditions. If these said conditions change for a pair in your whitelist, it could drain your funds quickly until you notice. As you know, crypto markets are volatile and huge dumps are not rare. I think such feature would benefit the novice users like me.


@abdullahe 

please see https://github.com/freqtrade/freqtrade/issues/2582#issuecomment-559404079 - where i posted a solution which - in my opinion, covers exactly what you describe - locking a pair for X time - thus preventing it from trading.


Resolving patch:
diff --git a/freqtrade/strategy/interface.py b/freqtrade/strategy/interface.py
--- a/freqtrade/strategy/interface.py
+++ b/freqtrade/strategy/interface.py
@@ -168,11 +168,24 @@ def lock_pair(self, pair: str, until: datetime) -> None:
         """
         Locks pair until a given timestamp happens.
         Locked pairs are not analyzed, and are prevented from opening new trades.
+        Locks can only count up (allowing users to lock pairs for a longer period of time).
+        To remove a lock from a pair, use `unlock_pair()`
         :param pair: Pair to lock
         :param until: datetime in UTC until the pair should be blocked from opening new trades.
                 Needs to be timezone aware `datetime.now(timezone.utc)`
         """
-        self._pair_locked_until[pair] = until
+        if pair not in self._pair_locked_until or self._pair_locked_until[pair] < until:
+            self._pair_locked_until[pair] = until
+
+    def unlock_pair(self, pair) -> None:
+        """
+        Unlocks a pair previously locked using lock_pair.
+        Not used by freqtrade itself, but intended to be used if users lock pairs
+        manually from within the strategy, to allow an easy way to unlock pairs.
+        :param pair: Unlock pair to allow trading again
+        """
+        if pair in self._pair_locked_until:
+            del self._pair_locked_until[pair]
 
     def is_pair_locked(self, pair: str) -> bool:
         """


Test patch:
diff --git a/tests/strategy/test_interface.py b/tests/strategy/test_interface.py
--- a/tests/strategy/test_interface.py
+++ b/tests/strategy/test_interface.py
@@ -302,6 +302,19 @@ def test_is_pair_locked(default_conf):
     # ETH/BTC locked for 4 minutes
     assert strategy.is_pair_locked(pair)
 
+    # Test lock does not change
+    lock = strategy._pair_locked_until[pair]
+    strategy.lock_pair(pair, arrow.utcnow().shift(minutes=2).datetime)
+    assert lock == strategy._pair_locked_until[pair]
+
     # XRP/BTC should not be locked now
     pair = 'XRP/BTC'
     assert not strategy.is_pair_locked(pair)
+
+    # Unlocking a pair that's not locked should not raise an error
+    strategy.unlock_pair(pair)
+
+    # Unlock original pair
+    pair = 'ETH/BTC'
+    strategy.unlock_pair(pair)
+    assert not strategy.is_pair_locked(pair)