Repository: rotki/rotki

Issue:
Some coinbase balances are missing
## Problem Definition

I have some Link and BTC on my coinbase account but rotki isn't reporting them although access to Link and BTC balances are checked on the API setting.

I didn't had this before, it has happened during the week.
I've tried to set some new API key but the problem is still here.

## Logs

No specific logs

### System Description

Operating system: Mac OS Catalina 10.15.7 
Rotki version: 15.2



Discussion:
Hello @bitboy99 thanks for the issue and thanks for using rotki!

Let's try to figure this out.

When you say, some coinbase balances, I assume then that some are displayed? Also from what you wrote I understand that all your coinbase balances were displaying properly before but now some of them are not?

Those that are not must have something in common together. Did you by any chance move them to a specific part of coinbase? A different type of wallet perhaps? Or did you just recently deposit them and have not restarted rotki or refreshed balances?
Hello @LefterisJP thanks for your quick answer.

Your understanding is good, my BTC and Link balances aren't displayed by Rotki even after refreshing the balances (by clicking on the circle arrow near "exchange balances"). 

Also, my Link and BTC trades appear in the history provided by rotki. 
I haven't done anything on Coinbase concerning those assets and they are here since more than two months.
Finally, I have some Link and BTC on other exchanges supported by Rotki and their balance appear on my Rotki balance (they are displayed). 

This issue wasn't here before V.1.15.1 I guess, before (from February 21) I was using V1.14.1 and even when I go back to this version the issue is still here. I cannot go back to an older version (V1.13.1, Rotki doesn't allow me to install it).
So @bitboy99 let me understand. The issue persists no matter what rotki version you are in, right?

It sounds to me like a problem with coinbase's API and not rotki as we have really not changed anything at all in coinbase's implementation for many months. And if you are going back to a version that used to work for you (v1.14.1) and it still happens that all seems to point at a change in what coinbase itself returns via its API.

To help you further we would need:

1. Debug logs. Even though I don't think they will show anything since this is about missing balances and not an error. To see how to activate debug logs check [here](https://rotki.readthedocs.io/en/latest/contribute.html#id2).
2. A read only version of your coinbase api key to try and debug this at its source.

You should provide those in private and not here in the issue. We can chat about it in DMs in discord: https://discord.gg/aGCxHG7 or you can email us at support@rotki.com. Discord is preferred.
I have the same error and added a logfile here: https://github.com/rotki/rotki/issues/2678

Edit by Lefteris. Let's keep things in one place:

## Logs

I added `len(final_data)` and a quick print iteration to `coinbase.py` to dig into the error, but it looks like the API just no longer returns ETH and BTC (for me at least).

```
2021-04-06 11:45:13 DEBUG https://api.coinbase.com:443 "GET /v2/accounts HTTP/1.1" 200 None
final data 24
 * Curve DAO Token
 * Storj
 * Ankr                                                                                                            ,
 * Cardano
 * Polygon
 * SKALE
 * SushiSwap
 * Synthetix Network Token
 * Bancor Network Token
 * The Graph
 * Aave
 * Filecoin
 * NuCypher
 * Civic
 * Decentraland
 * district0x
 * Ren
 * Wrapped Bitcoin
 * Balancer
 * Uniswap
 * yearn.finance
 * UMA
 * Loopring
 * Celo
/v2/accounts?starting_after=01898657-5c90-53fa-8b5a-38abbdc67585
2021-04-06 11:45:13 DEBUG Coinbase API query request_url=/v2/accounts?starting_after=01898657-5c90-53fa-8b5a-38abbdc67585
2021-04-06 11:45:13 DEBUG https://api.coinbase.com:443 "GET /v2/accounts?starting_after=01898657-5c90-53fa-8b5a-38abbdc67585 HTTP/1.1" 200 None
final data 25
 * Numeraire
 * REPv2
 * Band Protocol
 * Algorand
 * Compound
 * Maker
 * OMG Network
 * Orchid
 * Cosmos
 * EOS
 * Dash
 * Stellar Lumens
 * XRP
 * Zcash                                                                                                           ,
 * USD Coin
 * Basic Attention Token
 * 0x
 * Kyber Network
 * Augur
 * Chainlink
 * Tezos
 * Dai
 * Ethereum Classic
 * Bitcoin Cash
 * Litecoin
```

Nothing has changed in rotki at that part of the code. And if coinbase returned the balances of those assets before,  and now suddenly it simply does not include them in the response of the "accounts" api call then I think you should open a support request with them.

Perhaps they moved them to another endpoint, or the api key needs some extra permissions. Nothing I can see with a quick look in their docs: https://developers.coinbase.com/api/v2#list-accounts
This also leads to the fact that the trades of those missing accounts/balances are not imported
@LefterisJP yea looks like it. however what is striking is that the first call returns 24 accounts and the second 25. Thats weird.
I see the second call has a pagination identifier. Perhaps it's also an error in the way we do pagination, that started showing only recently due to something.
Yea, I found the error, its the `not pagination_next_url` that is wrong
So it will only paginate once, not multiple times
Hmm isn't it paginating with recursive calls to the function below? I don't have data to confirm the problem and fix it. Can you make a PR and test it with your account? Or alternatively give me an api key to test myself and tell me what balances you expect to see.
Sorry too busy for an PR :-)

If I remove `not pagination_next_url` in https://github.com/rotki/rotki/blob/develop/rotkehlchen/exchanges/coinbase.py#L353, balances and trades work again for me.

Resolving patch:
diff --git a/rotkehlchen/exchanges/coinbase.py b/rotkehlchen/exchanges/coinbase.py
--- a/rotkehlchen/exchanges/coinbase.py
+++ b/rotkehlchen/exchanges/coinbase.py
@@ -349,8 +349,8 @@ def _api_query(
 
         final_data = json_ret['data']
 
-        # If we got pagination and this is the first query, gather all the subsequent queries
-        if 'pagination' in json_ret and not pagination_next_uri and not ignore_pagination:
+        # If we got pagination recursively gather all the subsequent queries
+        if 'pagination' in json_ret and not ignore_pagination:
             if 'next_uri' not in json_ret['pagination']:
                 raise RemoteError('Coinbase json response contained no "next_uri" key')
 


Test patch: