Repository: freqtrade/freqtrade

Issue:
Coinbase Pro crash after first trade
  * Python Version:Python 3.7.2+ (`python -V`)
  * CCXT version: ccxt==1.18.725(`pip freeze | grep ccxt`)
  * Branch:  Develop
  * Last Commit ID: da5f77c96f6f16c7e58abba76a18ca1f2b39c371 (`git log --format="%H" -n 1`)
  
## Step 3: Describe the problem:

Start using the bot in live mode with Coinbase Pro after a couple of days dry run without issues. After the firs buy signal there is a trade, but bot crashes:
  ```
2019-06-23 16:51:44,611 - freqtrade.persistence - INFO - Found open trade: Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=2 hours ago)
2019-06-23 16:51:50,104 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=2 hours ago)
2019-06-23 16:51:50,620 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "freqtrade/main.py", line 52, in main
    worker.run()
  File "/home/martos/freqtrade/freqtrade/worker.py", line 72, in run
    state = self._worker(old_state=state)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 110, in _worker
    self._throttle(func=self._process, min_secs=throttle_secs)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 123, in _throttle
    result = func(*args, **kwargs)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 133, in _process
    state_changed = self.freqtrade.process()
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 136, in process
    state_changed |= self.process_maybe_execute_sell(trade)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 449, in process_maybe_execute_sell
    self.update_trade_state(trade)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 525, in update_trade_state
    new_amount = self.get_real_amount(trade, order)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 483, in get_real_amount
    if trade.pair.startswith(order['fee']['currency']):
TypeError: startswith first arg must be str or a tuple of str, not NoneType
2019-06-23 16:51:50,621 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'process died'}
2019-06-23 16:51:50,683 - freqtrade.freqtradebot - INFO - Cleaning up modules ...
2019-06-23 16:51:50,683 - freqtrade.rpc.rpc_manager - INFO - Cleaning up rpc modules ...
  ```

I commented out lines from 482 to 487 of freqtradebot.py and now bot runs but I am not sure it is ok. Is it normal to get this every process_throttle_secs from console:

  ```
2019-06-23 17:23:19,612 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=3 hours ago)
2019-06-23 17:23:21,404 - freqtrade.freqtradebot - INFO - Found no buy signals for whitelisted currencies. Trying again..
2019-06-23 17:23:22,791 - freqtrade.wallets - INFO - Wallets synced.
  ```
Any suggestions?

P.S.
In telegram when I check status I got this line:
```
Open Order: (market buy rem=0.00000000)
```
So is it possible bot does not mark the buy order as completed?
Coinbase Pro crash after first trade
  * Python Version:Python 3.7.2+ (`python -V`)
  * CCXT version: ccxt==1.18.725(`pip freeze | grep ccxt`)
  * Branch:  Develop
  * Last Commit ID: da5f77c96f6f16c7e58abba76a18ca1f2b39c371 (`git log --format="%H" -n 1`)
  
## Step 3: Describe the problem:

Start using the bot in live mode with Coinbase Pro after a couple of days dry run without issues. After the firs buy signal there is a trade, but bot crashes:
  ```
2019-06-23 16:51:44,611 - freqtrade.persistence - INFO - Found open trade: Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=2 hours ago)
2019-06-23 16:51:50,104 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=2 hours ago)
2019-06-23 16:51:50,620 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "freqtrade/main.py", line 52, in main
    worker.run()
  File "/home/martos/freqtrade/freqtrade/worker.py", line 72, in run
    state = self._worker(old_state=state)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 110, in _worker
    self._throttle(func=self._process, min_secs=throttle_secs)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 123, in _throttle
    result = func(*args, **kwargs)
  File "/home/martos/freqtrade/freqtrade/worker.py", line 133, in _process
    state_changed = self.freqtrade.process()
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 136, in process
    state_changed |= self.process_maybe_execute_sell(trade)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 449, in process_maybe_execute_sell
    self.update_trade_state(trade)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 525, in update_trade_state
    new_amount = self.get_real_amount(trade, order)
  File "/home/martos/freqtrade/freqtrade/freqtradebot.py", line 483, in get_real_amount
    if trade.pair.startswith(order['fee']['currency']):
TypeError: startswith first arg must be str or a tuple of str, not NoneType
2019-06-23 16:51:50,621 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'process died'}
2019-06-23 16:51:50,683 - freqtrade.freqtradebot - INFO - Cleaning up modules ...
2019-06-23 16:51:50,683 - freqtrade.rpc.rpc_manager - INFO - Cleaning up rpc modules ...
  ```

I commented out lines from 482 to 487 of freqtradebot.py and now bot runs but I am not sure it is ok. Is it normal to get this every process_throttle_secs from console:

  ```
2019-06-23 17:23:19,612 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/EUR, amount=0.37134688, open_rate=269.29000000, open_since=3 hours ago)
2019-06-23 17:23:21,404 - freqtrade.freqtradebot - INFO - Found no buy signals for whitelisted currencies. Trying again..
2019-06-23 17:23:22,791 - freqtrade.wallets - INFO - Wallets synced.
  ```
Any suggestions?

P.S.
In telegram when I check status I got this line:
```
Open Order: (market buy rem=0.00000000)
```
So is it possible bot does not mark the buy order as completed?


Discussion:
> Any suggestions?

Yes, I localized the problem and a corresponding more technical issue has then been opened in the ccxt github: https://github.com/ccxt/ccxt/issues/5343

I'll probably add (a bit later) some additional checks there in freqtrade as a protection measure against incorrectly filled fee structures from ccxt for this and other similar exchanges (ccxt also uses same code for gdax, for example).

For your telegram question, pls create a new issue, so that this won't become a mess of different questions...

For telegram, best check the [SQL Cheatsheet](https://www.freqtrade.io/en/latest/sql_cheatsheet/) and verify the trade ... i would assume that it didn't properly close the one you had the issue with. If you have additional questions about that -> new issue, thanks 
> Any suggestions?

Yes, I localized the problem and a corresponding more technical issue has then been opened in the ccxt github: https://github.com/ccxt/ccxt/issues/5343

I'll probably add (a bit later) some additional checks there in freqtrade as a protection measure against incorrectly filled fee structures from ccxt for this and other similar exchanges (ccxt also uses same code for gdax, for example).

For your telegram question, pls create a new issue, so that this won't become a mess of different questions...

For telegram, best check the [SQL Cheatsheet](https://www.freqtrade.io/en/latest/sql_cheatsheet/) and verify the trade ... i would assume that it didn't properly close the one you had the issue with. If you have additional questions about that -> new issue, thanks 

Resolving patch:
diff --git a/freqtrade/freqtradebot.py b/freqtrade/freqtradebot.py
--- a/freqtrade/freqtradebot.py
+++ b/freqtrade/freqtradebot.py
@@ -478,8 +478,11 @@ def get_real_amount(self, trade: Trade, order: Dict) -> float:
             return order_amount
 
         # use fee from order-dict if possible
-        if 'fee' in order and order['fee'] and (order['fee'].keys() >= {'currency', 'cost'}):
-            if trade.pair.startswith(order['fee']['currency']):
+        if ('fee' in order and order['fee'] is not None and
+                (order['fee'].keys() >= {'currency', 'cost'})):
+            if (order['fee']['currency'] is not None and
+                    order['fee']['cost'] is not None and
+                    trade.pair.startswith(order['fee']['currency'])):
                 new_amount = order_amount - order['fee']['cost']
                 logger.info("Applying fee on amount for %s (from %s to %s) from Order",
                             trade, order['amount'], new_amount)
@@ -496,9 +499,12 @@ def get_real_amount(self, trade: Trade, order: Dict) -> float:
         fee_abs = 0
         for exectrade in trades:
             amount += exectrade['amount']
-            if "fee" in exectrade and (exectrade['fee'].keys() >= {'currency', 'cost'}):
+            if ("fee" in exectrade and exectrade['fee'] is not None and
+                    (exectrade['fee'].keys() >= {'currency', 'cost'})):
                 # only applies if fee is in quote currency!
-                if trade.pair.startswith(exectrade['fee']['currency']):
+                if (exectrade['fee']['currency'] is not None and
+                        exectrade['fee']['cost'] is not None and
+                        trade.pair.startswith(exectrade['fee']['currency'])):
                     fee_abs += exectrade['fee']['cost']
 
         if amount != order_amount:


Test patch:
diff --git a/freqtrade/tests/test_freqtradebot.py b/freqtrade/tests/test_freqtradebot.py
--- a/freqtrade/tests/test_freqtradebot.py
+++ b/freqtrade/tests/test_freqtradebot.py
@@ -2886,6 +2886,30 @@ def test_get_real_amount_stake(default_conf, trades_for_order, buy_order_fee, mo
     assert freqtrade.get_real_amount(trade, buy_order_fee) == amount
 
 
+def test_get_real_amount_no_currency_in_fee(default_conf, trades_for_order, buy_order_fee, mocker):
+
+    limit_buy_order = deepcopy(buy_order_fee)
+    limit_buy_order['fee'] = {'cost': 0.004, 'currency': None}
+    trades_for_order[0]['fee']['currency'] = None
+
+    patch_RPCManager(mocker)
+    patch_exchange(mocker)
+    mocker.patch('freqtrade.exchange.Exchange.get_trades_for_order', return_value=trades_for_order)
+    amount = sum(x['amount'] for x in trades_for_order)
+    trade = Trade(
+        pair='LTC/ETH',
+        amount=amount,
+        exchange='binance',
+        open_rate=0.245441,
+        open_order_id="123456"
+    )
+    freqtrade = FreqtradeBot(default_conf)
+    patch_get_signal(freqtrade)
+
+    # Amount does not change
+    assert freqtrade.get_real_amount(trade, limit_buy_order) == amount
+
+
 def test_get_real_amount_BNB(default_conf, trades_for_order, buy_order_fee, mocker):
     trades_for_order[0]['fee']['currency'] = 'BNB'
     trades_for_order[0]['fee']['cost'] = 0.00094518