Repository: scrtlabs/catalyst

Issue #183: Some trades which sell entire position leave a very small amount in the position amount variable, leading to a crash.
Created: 2018-01-23T03:27:55Z
Dear Catalyst Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment

Operating System: Windows 10
Python Version: Python 2.7.13 :: Anaconda, Inc.
Python Bitness: Command doesn't work.
How did you install Catalyst: Conda
Python packages:

alembic                   0.9.6                     <pip>
backports.functools-lru-cache 1.4                       <pip>
bcolz                     0.12.1                    <pip>
boto3                     1.5.19                    <pip>
botocore                  1.8.33                    <pip>
Bottleneck                1.2.1                     <pip>
ca-certificates           2017.08.26           h94faf87_0
ccxt                      1.10.774                  <pip>
certifi                   2016.2.28                py27_0
certifi                   2018.1.18                 <pip>
chardet                   3.0.4                     <pip>
click                     6.7                       <pip>
contextlib2               0.5.5                     <pip>
cycler                    0.10.0                    <pip>
cyordereddict             1.0.0                     <pip>
Cython                    0.27.1                    <pip>
decorator                 4.1.2                     <pip>
docutils                  0.14                      <pip>
empyrical                 0.2.1                     <pip>
enigma-catalyst           0.4.7                     <pip>
futures                   3.2.0                     <pip>
idna                      2.6                       <pip>
intervaltree              2.1.0                     <pip>
jmespath                  0.9.3                     <pip>
Logbook                   1.1.0                     <pip>
lru-dict                  1.1.6                     <pip>
Mako                      1.0.7                     <pip>
MarkupSafe                1.0                       <pip>
matplotlib                2.1.0                     <pip>
mkl                       2017.0.3                      0
multipledispatch          0.4.9                     <pip>
networkx                  2.0                       <pip>
numexpr                   2.6.4                     <pip>
numpy                     1.13.1                   py27_0
openssl                   1.0.2l            vc9hb274bc6_5  [vc9]
pandas                    0.19.2                    <pip>
pandas-datareader         0.5.0                     <pip>
patsy                     0.4.1                     <pip>
pip                       9.0.1                    py27_1
pyparsing                 2.2.0                     <pip>
python                    2.7.13              h1b6d89f_16
python-dateutil           2.6.1                     <pip>
python-editor             1.0.3                     <pip>
pytz                      2017.2                    <pip>
redo                      1.6                       <pip>
requests                  2.18.4                    <pip>
requests-file             1.4.2                     <pip>
requests-ftp              0.3.1                     <pip>
s3transfer                0.1.12                    <pip>
scikit-learn              0.19.1                    <pip>
scipy                     0.19.1              np113py27_0
setuptools                36.4.0                   py27_1
six                       1.11.0                    <pip>
sortedcontainers          1.5.7                     <pip>
SQLAlchemy                1.1.14                    <pip>
sqlite                    3.13.0                    vc9_1  [vc9]
statsmodels               0.8.0                     <pip>
subprocess32              3.2.7                     <pip>
TA-Lib                    0.4.16                    <pip>
tables                    3.4.2                     <pip>
tk                        8.5.18                    vc9_0  [vc9]
toolz                     0.8.2                     <pip>
urllib3                   1.22                      <pip>
vc                        9                    h7299396_1
vs2008_runtime            9.00.30729.1         hfaea7d5_1
wheel                     0.29.0                   py27_0
wincertstore              0.2              py27hf04cefb_0
zlib                      1.2.11               hbc2faf4_2

Now that you know a little about me, let me tell you about the issue I am
having:
Description of Issue
Occasionally, when live trading performs is asked to sell my entire position (either through doing a "order percent" of 0, or submitting an order for a negative amount equal to my position) , it leaves a very small "position amount". I have experienced a similar issue live trading on both Binance and Poloniex. The same code (and mine), perform a sell if I currently have any position, so this causes it to try to sell the very small fractional amount in my position, which leads to a crash. This seems to happen inconsistently for some reason (perhaps it's related to the amount I am trying to sell and a rounding error?). Today my algorithm traded for 5 hours straight with no issues before this happened.
Below is the particular sale, where you can see I sold the exact amount in my position (.10394), yet it left a very small amount in the position amount variable.
[2018-01-23 01:18:05.703000] INFO: mean_reversion_binance: Pricerise: False, Pricerisecount: 3, pos_amount: 0.10394, Minute Count: 1, Price: 995.0, Trade Happened: False
[2018-01-23 01:18:06.207000] INFO: mean_reversion_binance: 2018-01-23 01:18:00+00:00: selling - price: 995.0, rsi: 55.8221143398, reason: None
[2018-01-23 01:18:06.795000] INFO: exchange_algorithm: statistics for the last 1 minutes:
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           last_sale_price

[2018-01-23 01:19:00.887000] INFO: exchange_blotter: found open order: 18739543
[2018-01-23 01:19:01.048000] INFO: exchange_blotter: filled order 18739543 / eth_usdt: -0.10394, avg price: 994.278641524
[2018-01-23 01:19:02.019000] INFO: exchange_algorithm: portfolio balances, cash: 127.6148006, positions: 3.46261214146e-15
[2018-01-23 01:19:06.519000] INFO: mean_reversion_binance: Pricerise: False, Pricerisecount: 4, pos_amount: 3.46944695195e-18, Minute Count: 1, Price: 995.4, Trade Happened: True
[2018-01-23 01:19:06.613000] INFO: exchange_algorithm: statistics for the last 1 minutes:
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               last_sale_price

[2018-01-23 01:20:01.348000] INFO: exchange_algorithm: portfolio balances, cash: 127.6148006, positions: 3.46254275252e-15
[2018-01-23 01:20:05.848000] INFO: mean_reversion_binance: Pricerise: False, Pricerisecount: 5, pos_amount: 3.46944695195e-18, Minute Count: 2, Price: 998.01, Trade Happened: True
[2018-01-23 01:20:05.951000] INFO: exchange_algorithm: statistics for the last 1 minutes:

And below is the instance where my algo decided to sell the position since there was an amount in there, which caused an error.
[2018-01-23 01:32:05.820000] INFO: mean_reversion_binance: Pricerise: False, Pricerisecount: 17, pos_amount: 3.46944695195e-18, Minute Count: 14, Price: 1001.0, Trade Happened: True
                                                                                                                                                                                                                                                                                                                                                                                                                                                              last_sale_price
....
[2018-01-23 01:33:00.657000] INFO: exchange_algorithm: portfolio balances, cash: 127.6148006, positions: 3.47284700997e-15
[2018-01-23 01:33:05.158000] INFO: mean_reversion_binance: Pricerise: False, Pricerisecount: 18, pos_amount: 3.46944695195e-18, Minute Count: 15, Price: 1000.98, Trade Happened: True
[2018-01-23 01:33:05.658000] INFO: mean_reversion_binance: 2018-01-23 01:33:00+00:00: selling - price: 1000.98, rsi: 58.86642418, reason: None
[2018-01-23 01:33:05.658000] INFO: CCXT: adjusted order amount 3.46944695195e-18 to 0.0 based on lot size
Traceback (most recent call last):
  File "Z:/Users/Brian/Google Drive/Catalyst/mean_reversion_simple - Binance.py", line 625, in <module>
    stats_output=None,
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\run_algo.py", line 551, in run_algorithm
    stats_output=stats_output
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\run_algo.py", line 330, in _run
    overwrite_sim_params=False,
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange_algorithm.py", line 297, in run
    data, overwrite_sim_params
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\algorithm.py", line 724, in run
    for perf in self.get_generator():
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\gens\tradesimulation.py", line 224, in transform
    for capital_change_packet in every_bar(dt):
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\gens\tradesimulation.py", line 137, in every_bar
    handle_data(algo, current_data, dt_to_use)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\events.py", line 216, in handle_data
    dt,
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\events.py", line 235, in handle_data
    self.callback(context, data)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange_algorithm.py", line 751, in handle_data
    self._handle_data(self, data)
  File "Z:/Users/Brian/Google Drive/Catalyst/mean_reversion_simple - Binance.py", line 268, in handle_data
    context.market, pos_amount * -1
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\api_support.py", line 57, in wrapped
    return getattr(algo_instance, f.__name__)(*args, **kwargs)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\utils\api_support.py", line 126, in wrapped_method
    return method(self, *args, **kwargs)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\algorithm.py", line 1462, in order
    return self.blotter.order(asset, amount, style)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange_blotter.py", line 169, in order
    if amount == 0:
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange_blotter.py", line 185, in order
    args=(asset, amount, style),
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\redo\__init__.py", line 162, in retry
    return action(*args, **kwargs)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange_blotter.py", line 163, in exchange_order
    asset, amount, style
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\exchange.py", line 822, in order
    return self.create_order(asset, amount, is_buy, style)
  File "C:\Users\brian\Anaconda2\envs\catalyst\lib\site-packages\catalyst\exchange\ccxt\ccxt_exchange.py", line 779, in create_order
    e='order amount lower than the smallest lot: {}'.format(amount)
catalyst.exchange.exchange_errors.CreateOrderError: <exception str() failed>

If this is a difficult issue to track down, is there a way for me to just zero out my position if it's say less than 1?
Reproduction Steps

Hopefully the steps above will be sufficient.



...
What steps have you taken to resolve this already?
Explained above.
...
Anything else?
...
Sincerely,
$ whoami

Duplicate issue #211: info: exchange_blotter: order has not reached the trigger at current price
Created: 2018-02-06T03:56:24Z
Dear Catalyst Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment

Operating System: ubuntu 14.04
Python Version: 2.7
Python Bitness: ``
How did you install Catalyst:pip
Python packages: $ pip freeze

Now that you know a little about me, let me tell you about the issue I am
having:
Description of Issue
During backtest I noticed a lot of my buy orders are not executing and catalyst logs  ...order has not reached the trigger at current price. I suspect this is to do with my limit orders but I can't seem to get rid of it.  Can you explain this behavior and how to set my orders up so they execute?

What did you expect to happen?
Orders execute
What happened instead?

catalyst issues above warning and never executes the orders
Here is how you can reproduce this issue on your machine:
I have not idea what's the cause so can't tell you how to accurately reproduce other than:
Reproduction Steps

select a pair on bitterex
run a backtest over a period of time
use  order_target_percent(asset=asset,  target=1, limit_price=price * 1.05)
...

What steps have you taken to resolve this already?
increased limit_price
...
Anything else?
...
Sincerely,
demontamer

Closing comment (2018-02-06T04:29:58Z): I believe that this is a duplicate of #183. We fixed in the develop branch but did not release yet. We'll release in the next day or two.