Repository: freqtrade/freqtrade

Issue:
After a few days  of run : TypeError: cannot unpack non-iterable IndexError object
## Step 1: Have you search for this issue before posting it?

Yes

## Step 2: Describe your environment

  * Python Version: 3.7.2 (`python -V`)
  * CCXT version: 1.18.102 (`pip freeze | grep ccxt`)
  * Branch: Develop
  * Last Commit ID: id image docker image ebfd99410421 (`git log --format="%H" -n 1`)
  * Debian 9 64 bit
  * Docker version 17.05.0-ce, build 89658be

## Step 3: Describe the problem:

Got a fatal exception after 3-4 days of running.

### Steps to reproduce:

No idea has i don't know the event that generate the problem
  
### Observed Results:

  * What happened?
  Stop the running contener
  * What did you expect to happen?
 Nothing

### Relevant code exceptions or logs:

  ```
2019-01-15 18:05:13,808 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "/freqtrade/freqtrade/main.py", line 49, in main
    state = freqtrade.worker(old_state=state)
  File "/freqtrade/freqtrade/freqtradebot.py", line 122, in worker
    min_secs=min_secs)
  File "/freqtrade/freqtrade/freqtradebot.py", line 134, in _throttle
    result = func(*args, **kwargs)
  File "/freqtrade/freqtrade/freqtradebot.py", line 170, in _process
    self.exchange.refresh_tickers(self.active_pair_whitelist, self.strategy.ticker_interval)
  File "/freqtrade/freqtrade/exchange/__init__.py", line 536, in refresh_tickers
    self.async_get_candles_history(pair_list, ticker_interval))
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/freqtrade/freqtrade/exchange/__init__.py", line 556, in async_get_candles_history
    for pair, ticks in tickers:
TypeError: cannot unpack non-iterable IndexError object
2019-01-15 18:05:13,809 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'process died'}
  ```
Many thks and tell me if you need more infos sorry to don't have the commit id.


Discussion:
@gaugau3000 how many pairs were in your whitelist ? 
19 pairs
The IndexError almost certainly is raised in async code here:
https://github.com/freqtrade/freqtrade/blob/e8423d81550d0ba616237e530bd119c707749ec5/freqtrade/exchange/__init__.py#L581

and is then passed back as result, having the tickers-loop fail.
I'm quite certain that it's based on an odd result from the exchange/binance where the result was not a list of lists (but also not empty - as we check that with `if  data ...`").

I am tempted to check if "tickers" is an exception type before the loop, but I'm not certain that will solve the problem.

More likely, we will need to iterate without unzipping the tuple, check each element in the list - and then see if one of the pairs generated an exception or not.

something similar to the following:
``` python 
 # handle caching
        for res in tickers:
            if isinstance(res, Exception):
                logger.warning("Async code raised an exception %s", res)
            if len(res) <= 1:
                logger.warning("Did not find correct result")
                continue
            pair = res[0]
            ticks = res[1]
```
...
agreed, it solves the problem. but it would be much better if we could also log `data` in case it is not a list of lists 
For this error to happen, the following needs to be returned ( it's VERY Specific)

* wrong type(integer): TypeError - int not subscribtable
* empty list: IndexError
* empty list of list : IndexError

So the result was ok (but empty) for whatever reason (network, exchange, ...)
we'll catch that as it shall not impact regular operations


Resolving patch:
diff --git a/freqtrade/exchange/__init__.py b/freqtrade/exchange/__init__.py
--- a/freqtrade/exchange/__init__.py
+++ b/freqtrade/exchange/__init__.py
@@ -556,12 +556,17 @@ async def async_get_candles_history(self, pairs: List[str],
         tickers = await asyncio.gather(*input_coroutines, return_exceptions=True)
 
         # handle caching
-        for pair, ticks in tickers:
+        for res in tickers:
+            if isinstance(res, Exception):
+                logger.warning("Async code raised an exception: %s", res.__class__.__name__)
+                continue
+            pair = res[0]
+            ticks = res[1]
             # keeping last candle time as last refreshed time of the pair
             if ticks:
                 self._pairs_last_refresh_time[pair] = ticks[-1][0] // 1000
-            # keeping parsed dataframe in cache
-            self._klines[pair] = parse_ticker_dataframe(ticks, tick_interval, fill_missing=True)
+                # keeping parsed dataframe in cache
+                self._klines[pair] = parse_ticker_dataframe(ticks, tick_interval, fill_missing=True)
         return tickers
 
     @retrier_async
@@ -578,9 +583,12 @@ async def _async_get_candle_history(self, pair: str, tick_interval: str,
             # Ex: Bittrex returns a list of tickers ASC (oldest first, newest last)
             # when GDAX returns a list of tickers DESC (newest first, oldest last)
             # Only sort if necessary to save computing time
-            if data and data[0][0] > data[-1][0]:
-                data = sorted(data, key=lambda x: x[0])
-
+            try:
+                if data and data[0][0] > data[-1][0]:
+                    data = sorted(data, key=lambda x: x[0])
+            except IndexError:
+                logger.exception("Error loading %s. Result was %s.", pair, data)
+                return pair, []
             logger.debug("done fetching %s ...", pair)
             return pair, data
 


Test patch:
diff --git a/freqtrade/tests/exchange/test_exchange.py b/freqtrade/tests/exchange/test_exchange.py
--- a/freqtrade/tests/exchange/test_exchange.py
+++ b/freqtrade/tests/exchange/test_exchange.py
@@ -923,6 +923,30 @@ async def mock_get_candle_hist(pair, tick_interval, since_ms=None):
     assert exchange._async_get_candle_history.call_count == 2
 
 
+@pytest.mark.asyncio
+async def test_async_get_candles_history_inv_result(default_conf, mocker, caplog):
+
+    async def mock_get_candle_hist(pair, *args, **kwargs):
+        if pair == 'ETH/BTC':
+            return [[]]
+        else:
+            raise TypeError()
+
+    exchange = get_patched_exchange(mocker, default_conf)
+
+    # Monkey-patch async function with empty result
+    exchange._api_async.fetch_ohlcv = MagicMock(side_effect=mock_get_candle_hist)
+
+    pairs = ['ETH/BTC', 'XRP/BTC']
+    res = await exchange.async_get_candles_history(pairs, "5m")
+    assert type(res) is list
+    assert len(res) == 2
+    assert type(res[0]) is tuple
+    assert type(res[1]) is TypeError
+    assert log_has("Error loading ETH/BTC. Result was [[]].", caplog.record_tuples)
+    assert log_has("Async code raised an exception: TypeError", caplog.record_tuples)
+
+
 def test_get_order_book(default_conf, mocker, order_book_l2):
     default_conf['exchange']['name'] = 'binance'
     api_mock = MagicMock()