Repository: scrtlabs/catalyst

Issue #422: Algo crash in live mode cause of a timeout (no retry handling?)
Created: 2018-07-30T20:10:37Z
Use catalyst version 0.5.19
Use bitfinex as exchange
Here the exception
-->
Traceback (most recent call last):
  File "main.py", line 323, in <module>
    main()
  File "main.py", line 285, in main
    quote_currency='usd'
  File "/catalyst/utils/run_algo.py", line 596, in run_algorithm
    stats_output=stats_output
  File "catalyst/utils/run_algo.py", line 358, in _run
    overwrite_sim_params=False,
  File "catalyst/exchange/exchange_algorithm.py", line 1075, in run
    data, overwrite_sim_params
  File "/catalyst/exchange/exchange_algorithm.py", line 362, in run
    data, overwrite_sim_params
  File "/catalyst/algorithm.py", line 725, in run
    for perf in self.get_generator():
  File "/catalyst/gens/tradesimulation.py", line 224, in transform
    for capital_change_packet in every_bar(dt):
  File "catalyst/gens/tradesimulation.py", line 137, in every_bar
    handle_data(algo, current_data, dt_to_use)
  File "/catalyst/utils/events.py", line 216, in handle_data
    dt,
  File "catalyst/utils/events.py", line 235, in handle_data
    self.callback(context, data)
  File "/catalyst/exchange/exchange_algorithm.py", line 919, in handle_data
    cleanup=lambda: log.warn('Syncing portfolio again.')
  File "/opt/anaconda3/envs/catalyst_0_5_18/lib/python3.6/site-packages/redo/__init__.py", line 162, in retry
    return action(*args, **kwargs)
  File "/catalyst/exchange/exchange_algorithm.py", line 721, in synchronize_portfolio
    cash=required_cash,
  File "/catalyst/exchange/exchange.py", line 783, in sync_positions
    for position_on_exchange in self.api.private_post_positions():

----- CCXT PART ->
  File "/python/ccxt/base/exchange.py", line 305, in request
    return self.fetch2(path, api, method, params, headers, body)
  File "/python/ccxt/base/exchange.py", line 302, in fetch2
    return self.fetch(request['url'], request['method'], request['headers'], request['body'])
  File "/python/ccxt/base/exchange.py", line 366, in fetch
    self.raise_error(RequestTimeout, method, url, e)
  File "/python/ccxt/base/exchange.py", line 287, in raise_error
    raise exception_type(output)
ccxt.base.errors.RequestTimeout: bitfinex POST https://api.bitfinex.com/v1/positions HTTPSConnectionPool(host='api.bitfinex.com', port=443): Read timed out. (read timeout=10)

There is a hardcoded timeout in this version of ccxt without a retry(I think):
/Users/user/projects/forked-libs/ccxt-fork/python/ccxt/base/exchange.py

Duplicate issue #420: poloniex get request crash
Created: 2018-07-29T11:00:32Z
enigma-catalyst version 0.5.18
poloniex GET https://poloniex.com/public?period=900&start=1532679300&command=returnChartData&end=1532859300&currencyPair=BTC_DASH HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=10)
If poloniex give timeout, then it should be retried instead of crashed.

Closing comment (2018-08-05T12:29:23Z): Hi @ytkang,
I am not sure it is related to #422.
There is a retry mechanism for retrieving history data but perhaps it should be improved.
I am reopening this, we will investigate.
If you have the entire trace, it could be helpful.
Thanks,
Lena