Repository: freqtrade/freqtrade

Issue:
askVolume and bidVolume in VolumePairList
Hi,

I come back with the discussion with @hroff-1902 we have about askVolume and bidVolume.
If i have well understood it filtered based on the volume of the best bid/ask.

It means like if Bitcoin have a bid with 30 dollar and a shitcoin have a bid at 3000 dollar it will update  the pairlist and put before in the list shitcoin vs BTC ?

Many tks.
askVolume and bidVolume in VolumePairList
Hi,

I come back with the discussion with @hroff-1902 we have about askVolume and bidVolume.
If i have well understood it filtered based on the volume of the best bid/ask.

It means like if Bitcoin have a bid with 30 dollar and a shitcoin have a bid at 3000 dollar it will update  the pairlist and put before in the list shitcoin vs BTC ?

Many tks.


Discussion:
askVolume and bidVolume can also be rolling 24-h values. At least on Binance.

I don't understand where  in freqtrade i can specify the rolling interval (i don't see this params in the doc). Tks.
It comes from exchange. Exchange calculates those rolling values. As quoteVolume -- it's also a rolling 24-h value. askVolume and bidVolume are not related to orderbook volumes. These are 24-h rolling values of bought and sold amounts for the pair.

@hroff-1902, i'm not sure where you got the information that bidVolume/askVolume can be rolling - it's not on binance.

``` python
import ccxt
ct1 = ccxt.binance()
t = ct1.fetch_ticker('IOTA/USDT')
print(t['baseVolume'])
print(t['quoteVolume'])
print(t['askVolume'])
print(t['bidVolume'])

# output:
19025801.69
4166761.633471
7582.39
2784.37
```

Now this 7582.39 did correspond to the orderbook amount at the time of writing.

In the screenshot below it's the lowest value in the orderbook's ask side.

![2020-02-27-071553_389x592_scrot](https://user-images.githubusercontent.com/5024695/75417319-00ce2180-5931-11ea-8120-01fda760fcd8.png)

----

You can also look at the definition by ccxt [here](https://github.com/ccxt/ccxt/wiki/Manual#ticker-structure).

relevant sections in this case:

```
     'bidVolume':     float, // current best bid (buy) amount (may be missing or undefined)
...
    'askVolume':     float, // current best ask (sell) amount (may be missing or undefined)
...
    'baseVolume':    float, // volume of base currency traded for last 24 hours
    'quoteVolume':   float, // volume of quote currency traded for last 24 hours
```

Now the problem with baseVolume is that it's in base-currency - so what does it tell you if you compare 
300 BTC with 10_000 IOTA? 
USDT volume for these tow might correspond to `300 * 10_000 = 3_000_000` and `10_000 * 0.2 = 2_000` USDT (which will correspond to QuoteVolume) - and for that reason, using baseVolume does not make sense (it's never been possibile iirc). 

bidVolume / askVolume on the other hand uses the volume of the orderbook closest to the current spread (not rolling - just the "current" volume.

Now using that in VolumePairlist, which is refreshed every X hours is higly questionable to me - but it's been there as possibility from the beginning...



If i have well understood the unique criteria that have a functional sense for VolumePairList  is quoteVolume has the others are defined on their base currency and cannot not be compared to each others.

Plus event even if bidVolume and askVolume could be transform in a bidQuoteVolume and askQuoteVolume to be sure we can compare markets volume to each other than it drop in this case :

> It means like if Bitcoin have a bid with 30 dollar and a shitcoin have a bid at 3000 dollar it will update the pairlist and put before in the list shitcoin vs BTC ?

witch still don't have sense i think.

So how it can be handle ?

Have a nice day.




ah, yeah, they are from the orderbook, I do not know where I got the opposite info, maybe just invented it from the air, my bad

in this case seems these values are also in the base currency units, no?
you're probably right, and no other setting than `quoteVolume` does make sense in reality, as the others are in base-currency.


let's deprecate this parameter? to have this filter using quoteVolume only?
askVolume and bidVolume can also be rolling 24-h values. At least on Binance.

I don't understand where  in freqtrade i can specify the rolling interval (i don't see this params in the doc). Tks.
It comes from exchange. Exchange calculates those rolling values. As quoteVolume -- it's also a rolling 24-h value. askVolume and bidVolume are not related to orderbook volumes. These are 24-h rolling values of bought and sold amounts for the pair.

@hroff-1902, i'm not sure where you got the information that bidVolume/askVolume can be rolling - it's not on binance.

``` python
import ccxt
ct1 = ccxt.binance()
t = ct1.fetch_ticker('IOTA/USDT')
print(t['baseVolume'])
print(t['quoteVolume'])
print(t['askVolume'])
print(t['bidVolume'])

# output:
19025801.69
4166761.633471
7582.39
2784.37
```

Now this 7582.39 did correspond to the orderbook amount at the time of writing.

In the screenshot below it's the lowest value in the orderbook's ask side.

![2020-02-27-071553_389x592_scrot](https://user-images.githubusercontent.com/5024695/75417319-00ce2180-5931-11ea-8120-01fda760fcd8.png)

----

You can also look at the definition by ccxt [here](https://github.com/ccxt/ccxt/wiki/Manual#ticker-structure).

relevant sections in this case:

```
     'bidVolume':     float, // current best bid (buy) amount (may be missing or undefined)
...
    'askVolume':     float, // current best ask (sell) amount (may be missing or undefined)
...
    'baseVolume':    float, // volume of base currency traded for last 24 hours
    'quoteVolume':   float, // volume of quote currency traded for last 24 hours
```

Now the problem with baseVolume is that it's in base-currency - so what does it tell you if you compare 
300 BTC with 10_000 IOTA? 
USDT volume for these tow might correspond to `300 * 10_000 = 3_000_000` and `10_000 * 0.2 = 2_000` USDT (which will correspond to QuoteVolume) - and for that reason, using baseVolume does not make sense (it's never been possibile iirc). 

bidVolume / askVolume on the other hand uses the volume of the orderbook closest to the current spread (not rolling - just the "current" volume.

Now using that in VolumePairlist, which is refreshed every X hours is higly questionable to me - but it's been there as possibility from the beginning...



If i have well understood the unique criteria that have a functional sense for VolumePairList  is quoteVolume has the others are defined on their base currency and cannot not be compared to each others.

Plus event even if bidVolume and askVolume could be transform in a bidQuoteVolume and askQuoteVolume to be sure we can compare markets volume to each other than it drop in this case :

> It means like if Bitcoin have a bid with 30 dollar and a shitcoin have a bid at 3000 dollar it will update the pairlist and put before in the list shitcoin vs BTC ?

witch still don't have sense i think.

So how it can be handle ?

Have a nice day.




ah, yeah, they are from the orderbook, I do not know where I got the opposite info, maybe just invented it from the air, my bad

in this case seems these values are also in the base currency units, no?
you're probably right, and no other setting than `quoteVolume` does make sense in reality, as the others are in base-currency.


let's deprecate this parameter? to have this filter using quoteVolume only?

Resolving patch:
diff --git a/freqtrade/configuration/deprecated_settings.py b/freqtrade/configuration/deprecated_settings.py
--- a/freqtrade/configuration/deprecated_settings.py
+++ b/freqtrade/configuration/deprecated_settings.py
@@ -58,29 +58,6 @@ def process_temporary_deprecated_settings(config: Dict[str, Any]) -> None:
     process_deprecated_setting(config, 'ask_strategy', 'ignore_roi_if_buy_signal',
                                'experimental', 'ignore_roi_if_buy_signal')
 
-    if not config.get('pairlists') and not config.get('pairlists'):
-        config['pairlists'] = [{'method': 'StaticPairList'}]
-        logger.warning(
-            "DEPRECATED: "
-            "Pairlists must be defined explicitly in the future."
-            "Defaulting to StaticPairList for now.")
-
-    if config.get('pairlist', {}).get("method") == 'VolumePairList':
-        logger.warning(
-            "DEPRECATED: "
-            f"Using VolumePairList in pairlist is deprecated and must be moved to pairlists. "
-            "Please refer to the docs on configuration details")
-        pl = {'method': 'VolumePairList'}
-        pl.update(config.get('pairlist', {}).get('config'))
-        config['pairlists'].append(pl)
-
-    if config.get('pairlist', {}).get('config', {}).get('precision_filter'):
-        logger.warning(
-            "DEPRECATED: "
-            f"Using precision_filter setting is deprecated and has been replaced by"
-            "PrecisionFilter. Please refer to the docs on configuration details")
-        config['pairlists'].append({'method': 'PrecisionFilter'})
-
     if (config.get('edge', {}).get('enabled', False)
        and 'capital_available_percentage' in config.get('edge', {})):
         logger.warning(
diff --git a/freqtrade/pairlist/VolumePairList.py b/freqtrade/pairlist/VolumePairList.py
--- a/freqtrade/pairlist/VolumePairList.py
+++ b/freqtrade/pairlist/VolumePairList.py
@@ -39,6 +39,10 @@ def __init__(self, exchange, pairlistmanager, config: Dict[str, Any], pairlistco
         if not self._validate_keys(self._sort_key):
             raise OperationalException(
                 f'key {self._sort_key} not in {SORT_VALUES}')
+        if self._sort_key != 'quoteVolume':
+            logger.warning(
+                "DEPRECATED: using any key other than quoteVolume for VolumePairList is deprecated."
+                )
 
     @property
     def needstickers(self) -> bool:


Test patch:
diff --git a/tests/pairlist/test_pairlist.py b/tests/pairlist/test_pairlist.py
--- a/tests/pairlist/test_pairlist.py
+++ b/tests/pairlist/test_pairlist.py
@@ -4,11 +4,11 @@
 
 import pytest
 
-from freqtrade.exceptions import OperationalException
 from freqtrade.constants import AVAILABLE_PAIRLISTS
-from freqtrade.resolvers import PairListResolver
+from freqtrade.exceptions import OperationalException
 from freqtrade.pairlist.pairlistmanager import PairListManager
-from tests.conftest import get_patched_freqtradebot, log_has_re
+from freqtrade.resolvers import PairListResolver
+from tests.conftest import get_patched_freqtradebot, log_has, log_has_re
 
 # whitelist, blacklist
 
@@ -228,6 +228,13 @@ def test_VolumePairList_whitelist_gen(mocker, whitelist_conf, shitcoinmarkets, t
             assert (log_has_re(r'^Removed .* from whitelist, because 1 unit is .*%$', caplog) or
                     log_has_re(r"^Removed .* from whitelist, because ticker\['last'\] is empty.*",
                                caplog))
+        if pairlist['method'] == 'VolumePairList':
+            logmsg = ("DEPRECATED: using any key other than quoteVolume for "
+                      "VolumePairList is deprecated.")
+            if pairlist['sort_key'] != 'quoteVolume':
+                assert log_has(logmsg, caplog)
+            else:
+                assert not log_has(logmsg, caplog)
 
 
 def test_gen_pair_whitelist_not_supported(mocker, default_conf, tickers) -> None:
diff --git a/tests/test_configuration.py b/tests/test_configuration.py
--- a/tests/test_configuration.py
+++ b/tests/test_configuration.py
@@ -1041,18 +1041,6 @@ def test_process_temporary_deprecated_settings(mocker, default_conf, setting, ca
     assert default_conf[setting[0]][setting[1]] == setting[5]
 
 
-def test_process_deprecated_setting_pairlists(mocker, default_conf, caplog):
-    patched_configuration_load_config_file(mocker, default_conf)
-    default_conf.update({'pairlist': {
-        'method': 'VolumePairList',
-        'config': {'precision_filter': True}
-    }})
-
-    process_temporary_deprecated_settings(default_conf)
-    assert log_has_re(r'DEPRECATED.*precision_filter.*', caplog)
-    assert log_has_re(r'DEPRECATED.*in pairlist is deprecated and must be moved*', caplog)
-
-
 def test_process_deprecated_setting_edge(mocker, edge_conf, caplog):
     patched_configuration_load_config_file(mocker, edge_conf)
     edge_conf.update({'edge': {