Repository: rotki/rotki

Issue:
Delisted coinbase pro assets break rotki due to bad CoinbasePro API design
## Problem Definition

A user reported a failed coinbase pro query: 

`Failed query for Coinbase Pro: Error: Coinbase Pro GET query at https://api.pro.coinbase.com/fills?product_id=XRP-EUR&limit=100 responded with error status code: 400 and text: {"message":"product_id is not a valid product"}`

After investigation it turns out that coinbase/coinbasepro has delisted XRP.

https://help.coinbase.com/en/coinbase/trading-and-funding/cryptocurrency-trading-pairs/ripple-sec
![2021-09-21_21-43](https://user-images.githubusercontent.com/1658405/134238691-102bfdfd-05c2-4c36-8620-0eb235f3c03a.png)

The problem with this is that the delisted `product_id` exists for historical completed orders but when you try to query the fills endpoint with the `product_id` this error appears.

## Task

1. Fix it by getting all valid products at the beginning of the coinbase pro session and if the given product is not in there then DO NOT query for it.
2. Report this to coinbase pro. This means history of trades for delisted assets is no longer retrievable.
Delisted coinbase pro assets break rotki due to bad CoinbasePro API design
## Problem Definition

A user reported a failed coinbase pro query: 

`Failed query for Coinbase Pro: Error: Coinbase Pro GET query at https://api.pro.coinbase.com/fills?product_id=XRP-EUR&limit=100 responded with error status code: 400 and text: {"message":"product_id is not a valid product"}`

After investigation it turns out that coinbase/coinbasepro has delisted XRP.

https://help.coinbase.com/en/coinbase/trading-and-funding/cryptocurrency-trading-pairs/ripple-sec
![2021-09-21_21-43](https://user-images.githubusercontent.com/1658405/134238691-102bfdfd-05c2-4c36-8620-0eb235f3c03a.png)

The problem with this is that the delisted `product_id` exists for historical completed orders but when you try to query the fills endpoint with the `product_id` this error appears.

## Task

1. Fix it by getting all valid products at the beginning of the coinbase pro session and if the given product is not in there then DO NOT query for it.
2. Report this to coinbase pro. This means history of trades for delisted assets is no longer retrievable.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/exchanges/coinbasepro.py b/rotkehlchen/exchanges/coinbasepro.py
--- a/rotkehlchen/exchanges/coinbasepro.py
+++ b/rotkehlchen/exchanges/coinbasepro.py
@@ -113,6 +113,7 @@ def __init__(
         self.base_uri = 'https://api.pro.coinbase.com'
         self.msg_aggregator = msg_aggregator
         self.account_to_currency: Optional[Dict[str, Asset]] = None
+        self.available_products = {0}
 
         self.session.headers.update({
             'Content-Type': 'Application/JSON',
@@ -165,6 +166,14 @@ def validate_api_key(self) -> Tuple[bool, str]:
 
         return True, ''
 
+    def first_connection(self) -> None:
+        if self.first_connection_made:
+            return
+
+        products_response, _ = self._api_query('products')
+        self.available_products = {x['id'] for x in products_response}
+        self.first_connection_made = True
+
     def _api_query(
             self,
             endpoint: str,
@@ -487,6 +496,7 @@ def query_online_trade_history(
     ) -> List[Trade]:
         """Queries coinbase pro for trades"""
         log.debug('Query coinbasepro trade history', start_ts=start_ts, end_ts=end_ts)
+        self.first_connection()
 
         trades = []
         # first get all orders, to see which product ids we need to query fills for
@@ -514,8 +524,8 @@ def query_online_trade_history(
                 )
                 continue
 
-            if product_id in queried_product_ids:
-                continue  # already queried this product id
+            if product_id in queried_product_ids or product_id not in self.available_products:
+                continue  # already queried this product id or delisted product id
 
             # Now let's get all fills for this product id
             queried_product_ids.add(product_id)


Test patch: