Repository: rotki/rotki

Issue:
Cryptocompare query failed with "You are over your rate limit, please upgrade your account"
## Problem Definition

This happened in the tests here: https://github.com/rotki/rotki/pull/872/checks?check_run_id=581306979

```
>                   raise RemoteError(error_message)
E                   rotkehlchen.errors.RemoteError: Failed to query cryptocompare for: "https://min-api.cryptocompare.com/data/v2/histohour?fsym=DASH&tsym=EUR&limit=2000&toTs=1434470400". Error: You are over your rate limit please upgrade your account!
```

## Task

Find out what's this new error from cryptocompare and handle accordingly.

Edit: What happened is that all 4 retries during the backing off were used up and then Rotki gave up querying cryptocompare. Perhaps ... increase the number of retries?


Discussion:
Okay more information. This seems to happen only in Github actions. In Travis we cache the location where the price information are queried `$HOME/.cache/.rotkehlchen-test-dir`

https://github.com/rotki/rotki/blob/19154b14ef2f6225b0a4f1b7744ac85d64363fae/.travis.yml#L79-L87

and it's also given as a test fixture:
https://github.com/rotki/rotki/blob/19154b14ef2f6225b0a4f1b7744ac85d64363fae/rotkehlchen/tests/fixtures/accounting.py#L28-L29

In the github-actions workflow I don't see this directory cached. Is this observation correct @kelsos ?

Then the solution is twofold:

1. Increase the number of retries ... don't want this to ever happen in reality
2. Cache that directory so that for tests we have reduced cryptocompare queries.

Resolving patch:
diff --git a/rotkehlchen/externalapis/cryptocompare.py b/rotkehlchen/externalapis/cryptocompare.py
--- a/rotkehlchen/externalapis/cryptocompare.py
+++ b/rotkehlchen/externalapis/cryptocompare.py
@@ -12,7 +12,6 @@
 from rotkehlchen.constants import ZERO
 from rotkehlchen.constants.assets import A_BTC, A_USD
 from rotkehlchen.constants.cryptocompare import KNOWN_TO_MISS_FROM_CRYPTOCOMPARE
-from rotkehlchen.constants.timing import QUERY_RETRY_TIMES
 from rotkehlchen.db.dbhandler import DBHandler
 from rotkehlchen.errors import NoPriceForGivenTimestamp, PriceQueryUnknownFromAsset, RemoteError
 from rotkehlchen.externalapis.interface import ExternalServiceWithApiKey
@@ -36,6 +35,7 @@
 PairCacheKey = NewType('PairCacheKey', T_PairCacheKey)
 
 RATE_LIMIT_MSG = 'You are over your rate limit please upgrade your account!'
+CRYPTOCOMPARE_QUERY_RETRY_TIMES = 10
 
 
 class PriceHistoryEntry(NamedTuple):
@@ -147,7 +147,7 @@ def _api_query(self, path: str) -> Dict[str, Any]:
         if api_key:
             querystr += f'&api_key={api_key}'
 
-        tries = QUERY_RETRY_TIMES
+        tries = CRYPTOCOMPARE_QUERY_RETRY_TIMES
         while tries >= 0:
             try:
                 response = self.session.get(querystr)
@@ -173,7 +173,7 @@ def _api_query(self, path: str) -> Dict[str, Any]:
                     else:
                         log.debug(
                             f'Got rate limited by cryptocompare and did not manage to get a '
-                            f'request through even after {QUERY_RETRY_TIMES} '
+                            f'request through even after {CRYPTOCOMPARE_QUERY_RETRY_TIMES} '
                             f'incremental backoff retries',
                         )
 


Test patch: