Repository: rotki/rotki

Issue:
Failed to deserialize a Bitstamp asset movement - notification
## Problem Definition

I often get tis notification in the top right corner:

Backend messages:
Failed to deserialize a Bitstamp asset movement. Check logs for details. Ignoring it.



## Logs

the log is empty


### System Description


Operating system: Win 10 Pro 64bit
Rotki version: 1.13.0



Discussion:
Hello @maravac we need logs for this. Check my comment on your other issue to figure out how to give us logs with the offending asset movement.
I tried to open it in a debug mode via UI as well as via manual adding the .json file, as described, but still the rotkehlchen.log file is 0 bytes (empty).
While going through another user's logs for another problem I found a bitstamp asset movements deserialization error so I am going to assume it's the same cause.

Log entry amounts/values/date changed to preserve user's privacy
```
[DATE] ERROR rotkehlchen.exchanges.bitstamp: Error processing a Bitstamp asset movement. raw_result={'fee': FVal(0.80), 'btc_usd': FVal(0.00), 'datetime': '2018-03-21 06:46:06.559877', 'usd': FVal(0.0), 'btc': FVal(0.0), 'type': 1, 'id': 123, 'eur': FVal(-100)}, error=Could not deserialize Bitstamp asset movement from user transaction. Unexpected asset amount combination found in: {'fee': FVal(0.80), 'btc_usd': FVal(0.00), 'datetime': '2018-03-21 06:46:06.559877', 'usd': FVal(0.0), 'btc': FVal(0.0), 'type': 1, 'id': 123, 'eur': FVal(-100)}.
```

Going to look at the code and their docs to see what is happening.
Hi,
few last days I had no time to open the Rotki app. But today I started it again and it displays another errors, including problem with Kraken exchange API key. (Ofcourse first I downloaded and installed the most recent version 1.13.1 of Rotki):

![obrazek](https://user-images.githubusercontent.com/37882327/107125395-10c4e400-68aa-11eb-9b48-28ffcc75e9a1.png)

OK, so I simply removed the Kraken API and tried to setup it again. But it is impossible not to even add the API:

![obrazek](https://user-images.githubusercontent.com/37882327/107125541-e7f11e80-68aa-11eb-93a7-e24043b067ba.png)


This is really pain. First, My Binance exchange transactions are not synchronized, then problem with Bitstamp deserialization and now even problem with Kraken API...   :)

As I mentioned in the other issue your system clock seems to be completely out of sync. That will result in such errors querying exchange APIS and more. You can't use rotki or other applications that query the internet without a synchronized system clock. Please synchronize your system clock. Also make sure your Kraken API keys are valid.
No, this one is my home computer which is OK in terms och correct time synchronization. That other PC, where I had the time problem was my office PC, where I tried Rotki as well, because in my home PC I have problem to switch into debug mode. 
Btw. in my office PC I am able to switch the debug mode (even that there is tha t time sync problem), but also there the rotkehlchen.log file is empty.

My API keys are 100% valid, I did not change anything with them.
> My API keys are 100% valid, I did not change anything with them.

It's either your keys are invalid, your clock is out of sync or Kraken is having problems. And without logs it's virtually impossible to provide any further assistance.

I found the problem of the bitstamp asset movement issue thanks to the other user's logs and will fix this issue now.

---

Now as to the reason why your logs are not there, I have no idea. We tried to help you in the other issue where @kelsos also wrote an explanation as to what may be wrong. No other Windows user has ever had problems generating logs. Let's just use https://github.com/rotki/rotki/issues/2230 to see if there is anything more we can do to figure out why your logs are not there.
OK, thankc. I will then create another new API key in Kraken and will try with this one.
My PC time should be OK - I synced it manually (Control Panels) before I started Rotki app just to be sure that there is no time problem.

Resolving patch:
diff --git a/rotkehlchen/exchanges/bitstamp.py b/rotkehlchen/exchanges/bitstamp.py
--- a/rotkehlchen/exchanges/bitstamp.py
+++ b/rotkehlchen/exchanges/bitstamp.py
@@ -90,6 +90,23 @@
 # Asset movement type int: 0 - deposit, 1 - withdrawal
 USER_TRANSACTION_ASSET_MOVEMENT_TYPE = {0, 1}
 
+# from https://www.bitstamp.net/api/#user-transactions
+BITSTAMP_ASSET_MOVEMENT_SYMBOLS = (
+    'usd',
+    'eur',
+    'btc',
+    'xrp',
+    'gbp',
+    'ltc',
+    'eth',
+    'bch',
+    'xlm',
+    'pax',
+    'link',
+    'omg',
+    'usdc',
+)
+
 
 class TradePairData(NamedTuple):
     pair: str
@@ -494,8 +511,8 @@ def _check_for_system_clock_not_synced_error(
                 remote_server='Bitstamp',
             )
 
+    @staticmethod
     def _deserialize_asset_movement(
-            self,
             raw_movement: Dict[str, Any],
     ) -> AssetMovement:
         """Process a deposit/withdrawal user transaction from Bitstamp and
@@ -507,18 +524,8 @@ def _deserialize_asset_movement(
         (the amount is expected to be in the currency involved)
         https://www.bitstamp.net/fee-schedule/
 
-        Bitstamp support confirmed the following withdrawal JSON:
-        {
-            "fee": "0.00050000",
-            "btc_usd": "0.00",
-            "datetime": "2020-12-04 09:30:00.000000",
-            "usd": "0.0",
-            "btc": "-0.50000000",
-            "type": "1",
-            "id": 123456789,
-            "eur": "0.0"
-        }
-        NB: any asset key not related with the pair is discarded (e.g. 'eur').
+        Endpoint docs:
+        https://www.bitstamp.net/api/#user-transactions
         """
         type_ = raw_movement['type']
         category: AssetMovementCategory
@@ -530,22 +537,15 @@ def _deserialize_asset_movement(
             raise AssertionError(f'Unexpected Bitstamp asset movement case: {type_}.')
 
         timestamp = deserialize_timestamp_from_bitstamp_date(raw_movement['datetime'])
-        trade_pair_data = self._get_trade_pair_data_from_transaction(raw_movement)
-        base_asset_amount = deserialize_asset_amount(
-            raw_movement[trade_pair_data.base_asset_symbol],
-        )
-        quote_asset_amount = deserialize_asset_amount(
-            raw_movement[trade_pair_data.quote_asset_symbol],
-        )
         amount: FVal
         fee_asset: Asset
-        if base_asset_amount != ZERO and quote_asset_amount == ZERO:
-            amount = base_asset_amount
-            fee_asset = trade_pair_data.base_asset
-        elif base_asset_amount == ZERO and quote_asset_amount != ZERO:
-            amount = quote_asset_amount
-            fee_asset = trade_pair_data.quote_asset
-        else:
+        for symbol in BITSTAMP_ASSET_MOVEMENT_SYMBOLS:
+            amount = deserialize_asset_amount(raw_movement.get(symbol, '0'))
+            if amount != ZERO:
+                fee_asset = Asset(symbol)
+                break
+
+        if amount == ZERO:
             raise DeserializationError(
                 'Could not deserialize Bitstamp asset movement from user transaction. '
                 f'Unexpected asset amount combination found in: {raw_movement}.',


Test patch:
diff --git a/rotkehlchen/tests/exchanges/test_bitstamp.py b/rotkehlchen/tests/exchanges/test_bitstamp.py
--- a/rotkehlchen/tests/exchanges/test_bitstamp.py
+++ b/rotkehlchen/tests/exchanges/test_bitstamp.py
@@ -798,6 +798,33 @@ def test_deserialize_asset_movement_deposit(mock_bitstamp):
     expected_movement = mock_bitstamp._deserialize_asset_movement(raw_movement)
     assert movement == expected_movement
 
+    raw_movement = {
+        'id': 3,
+        'type': 0,
+        'datetime': '2018-03-21 06:46:06.559877',
+        'btc': '0',
+        'usd': '0.00000000',
+        'btc_usd': '0.00',
+        'fee': '0.1',
+        'order_id': 2,
+        'gbp': '1000.51',
+    }
+    asset = Asset('GBP')
+    movement = AssetMovement(
+        timestamp=1521614766,
+        location=Location.BITSTAMP,
+        category=AssetMovementCategory.DEPOSIT,
+        address=None,
+        transaction_id=None,
+        asset=asset,
+        amount=FVal('1000.51'),
+        fee_asset=asset,
+        fee=Fee(FVal('0.1')),
+        link='3',
+    )
+    expected_movement = mock_bitstamp._deserialize_asset_movement(raw_movement)
+    assert movement == expected_movement
+
 
 def test_deserialize_asset_movement_withdrawal(mock_bitstamp):
     raw_movement = {
@@ -827,6 +854,33 @@ def test_deserialize_asset_movement_withdrawal(mock_bitstamp):
     expected_movement = mock_bitstamp._deserialize_asset_movement(raw_movement)
     assert movement == expected_movement
 
+    raw_movement = {
+        'id': 5,
+        'type': 1,
+        'datetime': '2018-03-21 06:46:06.559877',
+        'btc': '0',
+        'usd': '0',
+        'btc_usd': '0.00',
+        'fee': '0.1',
+        'order_id': 2,
+        'eur': '500',
+    }
+    asset = Asset('EUR')
+    movement = AssetMovement(
+        timestamp=1521614766,
+        location=Location.BITSTAMP,
+        category=AssetMovementCategory.WITHDRAWAL,
+        address=None,
+        transaction_id=None,
+        asset=asset,
+        amount=FVal('500'),
+        fee_asset=asset,
+        fee=Fee(FVal('0.1')),
+        link='5',
+    )
+    expected_movement = mock_bitstamp._deserialize_asset_movement(raw_movement)
+    assert movement == expected_movement
+
 
 @pytest.mark.parametrize('start_ts, since_id', [(0, 1), (1, 6)])
 def test_query_online_deposits_withdrawals(mock_bitstamp, start_ts, since_id):