Repository: freqtrade/freqtrade

Issue:
Logs and Empty ticker history  when using dynamic pair list (binance)
Hi,

When using dynamic pair list and when binance introduce a new pair i got this type of message :
```
2020-02-07 07:45:53,047 - freqtrade.exchange.common - WARNING - _async_get_candle_history() returned exception: "Could not load ticker history due to BadSymbol. Message: binance does not have market symbol BNT/USDT"
2020-02-07 07:45:53,047 - freqtrade.exchange.common - WARNING - Giving up retrying: _async_get_candle_history()
```

then

`2020-02-07 07:54:53,306 - freqtrade.exchange.exchange - WARNING - Async code raised an exception: TemporaryError`

and then

`2020-02-07 07:54:56,265 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair BNT/USDT`


The problem is that warning comes a lot of time per minutes because of retries and after 3 weeks of run of my bot i have a beautiful log file of 2.3 Go :-)

I don't know what is the best to manage this maybe just the 
` WARNING - Empty ticker history for pair WRX/USDT` once per  `internals.heartbeat_interval`.

And i don't know if it should appear at warning level as it is not dangerous for trading money but just 'hello binance has find this market but have don't ticker yet'

Have a nice day.





Discussion:
what's your volumePairlist config look like so it even finds this pair (and at what timeframe / ticker_interval) ? 

I mean - VolumePairlist does filter per volume in the last 24 h - so unless you're using a large timeframe, the pair should have at least 1 point of history within "ticker_interval".

That said - once you have 1 point (candle) of history - it becomes similar to #1067 - where it does not show this message - but trading it may not work correctly depending on the strategy (not enough history ...).


the timeframe in 15 minutes and my strategy need a minimum of 50 candle of history.
my config is (i filter volume straight in strategy so 500 for number_assets is just like unlimited)

```
"pairlists": [{
        "method": "VolumePairList",
        "number_assets": 500,
        "sort_key": "quoteVolume",
        "refresh_period": 1800
    },
```
Kind of weird is that i this log  : But i make the test for WRX market is open since 3 days and CCXT gives me ticker and ohlc when i test

but freqtrade consider WRX/USDT with empty ticker.
 
```
2020-02-07 10:39:29,453 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair WRX/USDT
2020-02-07 10:39:29,809 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair BEAR/USDT
2020-02-07 10:39:29,821 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair ETHBEAR/USDT
2020-02-07 10:39:29,866 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair ETHBULL/USDT
2020-02-07 10:39:29,879 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair BULL/USDT
2020-02-07 10:39:29,991 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair TCT/USDT
2020-02-07 10:39:30,335 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair LSK/USDT
2020-02-07 10:39:30,430 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair BTS/USDT
2020-02-07 10:39:30,471 - freqtrade.strategy.interface - WARNING - Empty ticker history for pair BNT/USDT
```

```
import ccxt
binance   = ccxt.binance()

import time
if binance.has['fetchOHLCV']:
      time.sleep (binance.rateLimit / 1000) # time.sleep wants seconds
      print ('WRX/USDT', binance.fetch_ohlcv ('WRX/USDT', '15m')) 
```

Tks.

Well yes - ccxt returns values - which i can confirm.

Now one thing which i guess can (somehow) cause this for new pairs is if the markets are not reloaded (so this pair's metadata is not fully available).

Can you confirm this also after restarting the bot?
Restarting the bot resolve the problem. It is not the rôle of refresh_period to reload market ?

Many tks.
yes it should be ... 

Resolving patch:
diff --git a/freqtrade/exchange/exchange.py b/freqtrade/exchange/exchange.py
--- a/freqtrade/exchange/exchange.py
+++ b/freqtrade/exchange/exchange.py
@@ -285,6 +285,8 @@ def reload_markets(self) -> None:
         logger.debug("Performing scheduled market reload..")
         try:
             self._api.load_markets(reload=True)
+            # Also reload async markets to avoid issues with newly listed pairs
+            self._load_async_markets(reload=True)
             self._last_markets_refresh = arrow.utcnow().timestamp
         except ccxt.BaseError:
             logger.exception("Could not reload markets.")


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -56,6 +56,7 @@ def patched_configuration_load_config_file(mocker, config) -> None:
 
 
 def patch_exchange(mocker, api_mock=None, id='bittrex', mock_markets=True) -> None:
+    mocker.patch('freqtrade.exchange.Exchange._load_async_markets', MagicMock(return_value={}))
     mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
     mocker.patch('freqtrade.exchange.Exchange.validate_pairs', MagicMock())
     mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
diff --git a/tests/exchange/test_exchange.py b/tests/exchange/test_exchange.py
--- a/tests/exchange/test_exchange.py
+++ b/tests/exchange/test_exchange.py
@@ -319,7 +319,12 @@ def test_set_sandbox_exception(default_conf, mocker):
 
 
 def test__load_async_markets(default_conf, mocker, caplog):
-    exchange = get_patched_exchange(mocker, default_conf)
+    mocker.patch('freqtrade.exchange.Exchange._init_ccxt')
+    mocker.patch('freqtrade.exchange.Exchange.validate_pairs')
+    mocker.patch('freqtrade.exchange.Exchange.validate_timeframes')
+    mocker.patch('freqtrade.exchange.Exchange._load_markets')
+    mocker.patch('freqtrade.exchange.Exchange.validate_stakecurrency')
+    exchange = Exchange(default_conf)
     exchange._api_async.load_markets = get_mock_coro(None)
     exchange._load_async_markets()
     assert exchange._api_async.load_markets.call_count == 1
@@ -365,6 +370,7 @@ def load_markets(*args, **kwargs):
     default_conf['exchange']['markets_refresh_interval'] = 10
     exchange = get_patched_exchange(mocker, default_conf, api_mock, id="binance",
                                     mock_markets=False)
+    exchange._load_async_markets = MagicMock()
     exchange._last_markets_refresh = arrow.utcnow().timestamp
     updated_markets = {'ETH/BTC': {}, "LTC/BTC": {}}
 
@@ -373,11 +379,13 @@ def load_markets(*args, **kwargs):
     # less than 10 minutes have passed, no reload
     exchange.reload_markets()
     assert exchange.markets == initial_markets
+    assert exchange._load_async_markets.call_count == 0
 
     # more than 10 minutes have passed, reload is executed
     exchange._last_markets_refresh = arrow.utcnow().timestamp - 15 * 60
     exchange.reload_markets()
     assert exchange.markets == updated_markets
+    assert exchange._load_async_markets.call_count == 1
     assert log_has('Performing scheduled market reload..', caplog)