Repository: freqtrade/freqtrade

Issue:
`test-pairlist` fails if no markets loaded
`test-pairlist` fails in unexpected way if markets were not able to be loaded (due to 4xx error, for example);

This should be handled gracefully with a DependencyException (for example, or OperationalException) exception raised, an error message and exit code 1, it should not crash with traceback.

```
$ freqtrade test-pairlist -c config_kraken.json
2020-06-09 12:46:47,312 - freqtrade.loggers - INFO - Verbosity set to 0
2020-06-09 12:46:47,312 - freqtrade.configuration.configuration - INFO - Using user-data directory: /home/user/freqtrade-wrk/github-hroff-1902/freqtrade/user_data ...
2020-06-09 12:46:47,313 - freqtrade.configuration.configuration - INFO - Using data directory: /home/user/freqtrade-wrk/github-hroff-1902/freqtrade/user_data/data/kraken ...
2020-06-09 12:46:47,313 - freqtrade.configuration.check_exchange - INFO - Checking exchange...
2020-06-09 12:46:47,313 - freqtrade.configuration.check_exchange - INFO - Exchange "kraken" is officially supported by the Freqtrade development team.
2020-06-09 12:46:47,313 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration.
2020-06-09 12:46:47,313 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2020-06-09 12:46:47,315 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2020-06-09 12:46:47,315 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True}
2020-06-09 12:46:47,317 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True, 'rateLimit': 3100}
2020-06-09 12:46:47,320 - freqtrade.exchange.exchange - INFO - Using Exchange "Kraken"
2020-06-09 12:46:47,320 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Kraken'...
2020-06-09 12:46:47,324 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/pairlist/StaticPairList.py'...
2020-06-09 12:46:47,324 - freqtrade.exchange.exchange - WARNING - Markets were not loaded. Loading them now..
2020-06-09 12:46:53,817 - freqtrade.exchange.exchange - WARNING - Unable to initialize markets. Reason: GET https://kraken.zendesk.com/api/v2/help_center/en-us/articles/205893708 429 Too Many Requests Number of allowed API requests per minute exceeded
2020-06-09 12:46:53,818 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/main.py", line 36, in main
    return_code = args['func'](args)
  File "/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/commands/pairlist_commands.py", line 30, in start_test_pairlist
    pairlists.refresh_pairlist()
  File "/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/pairlist/pairlistmanager.py", line 91, in refresh_pairlist
    pairlist = self._pairlist_handlers[0].gen_pairlist(self._whitelist, tickers)
  File "/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/pairlist/StaticPairList.py", line 50, in gen_pairlist
    return self._whitelist_for_active_markets(self._config['exchange']['pair_whitelist'])
  File "/home/user/freqtrade-wrk/github-hroff-1902/freqtrade/freqtrade/pairlist/IPairList.py", line 157, in _whitelist_for_active_markets
    if pair not in markets:
TypeError: argument of type 'NoneType' is not iterable
```


Discussion:
Please make sure to use an uptodate version of ccxt...

the "zendesk" / kraken issue is supposed to be fixed since early May ... 
(not to say we should handle it gracefully anyway).
yeah, it does not appear again after update, but the fix here can be in the proper handling of empty markets loaded for some reasons...

Resolving patch:
diff --git a/freqtrade/pairlist/IPairList.py b/freqtrade/pairlist/IPairList.py
--- a/freqtrade/pairlist/IPairList.py
+++ b/freqtrade/pairlist/IPairList.py
@@ -150,6 +150,9 @@ def _whitelist_for_active_markets(self, pairlist: List[str]) -> List[str]:
         black_listed
         """
         markets = self._exchange.markets
+        if not markets:
+            raise OperationalException(
+                    'Markets not loaded. Make sure that exchange is initialized correctly.')
 
         sanitized_whitelist: List[str] = []
         for pair in pairlist:


Test patch:
diff --git a/tests/pairlist/test_pairlist.py b/tests/pairlist/test_pairlist.py
--- a/tests/pairlist/test_pairlist.py
+++ b/tests/pairlist/test_pairlist.py
@@ -421,6 +421,23 @@ def test__whitelist_for_active_markets(mocker, whitelist_conf, markets, pairlist
     assert log_message in caplog.text
 
 
+@pytest.mark.parametrize("pairlist", AVAILABLE_PAIRLISTS)
+def test__whitelist_for_active_markets_empty(mocker, whitelist_conf, markets, pairlist, tickers):
+    whitelist_conf['pairlists'][0]['method'] = pairlist
+
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', return_value=True)
+
+    freqtrade = get_patched_freqtradebot(mocker, whitelist_conf)
+    mocker.patch.multiple('freqtrade.exchange.Exchange',
+                          markets=PropertyMock(return_value=None),
+                          get_tickers=tickers
+                          )
+    # Assign starting whitelist
+    pairlist_handler = freqtrade.pairlists._pairlist_handlers[0]
+    with pytest.raises(OperationalException, match=r'Markets not loaded.*'):
+        pairlist_handler._whitelist_for_active_markets(['ETH/BTC'])
+
+
 def test_volumepairlist_invalid_sortvalue(mocker, markets, whitelist_conf):
     whitelist_conf['pairlists'][0].update({"sort_key": "asdf"})