internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: rotki/rotki

Issue:
Bitcoin legacy xpub with too many used address may fail to query balances after restart
## Problem Definition

If a user adds a legacy xpub that generates too many used addresses rotki will properly add the xpub and all its addresses. But then after a restart it will fail to query all balances again.

Error is:

```
Error at querying blockchain balances: bitcoin external API request for balances failed due to https://blockchain.info/multiaddr?active=XXXXXX
```

The failure is due to attempting to query all addresses from blockchain.info using the multiaddr endpoint which seems to have a non-documented limit.

blockchain.info errors with a 502 if too many bitcoin addresses are sent.

## Task

Fix it by experimenting and figuring out a safe limit.
Then apply that limit where we query for multiple legacy bitcoin addresses.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/bitcoin/__init__.py b/rotkehlchen/chain/bitcoin/__init__.py
--- a/rotkehlchen/chain/bitcoin/__init__.py
+++ b/rotkehlchen/chain/bitcoin/__init__.py
@@ -33,16 +33,20 @@ def get_bitcoin_addresses_balances(accounts: List[BTCAddress]) -> Dict[BTCAddres
                 balance = int(stats['funded_txo_sum']) - int(stats['spent_txo_sum'])
                 balances[account] = satoshis_to_btc(balance)
         else:
-            params = '|'.join(accounts)
-            btc_resp = request_get_dict(
-                url=f'https://blockchain.info/multiaddr?active={params}',
-                handle_429=True,
-                # If we get a 429 then their docs suggest 10 seconds
-                # https://blockchain.info/q
-                backoff_in_seconds=10,
-            )
-            for entry in btc_resp['addresses']:
-                balances[entry['address']] = satoshis_to_btc(FVal(entry['final_balance']))
+            # split the list of accounts into sublists of 80 addresses per list to overcome:
+            # https://github.com/rotki/rotki/issues/3037
+            accounts_chunks = [accounts[x:x + 80] for x in range(0, len(accounts), 80)]
+            for accounts_chunk in accounts_chunks:
+                params = '|'.join(accounts_chunk)
+                btc_resp = request_get_dict(
+                    url=f'https://blockchain.info/multiaddr?active={params}',
+                    handle_429=True,
+                    # If we get a 429 then their docs suggest 10 seconds
+                    # https://blockchain.info/q
+                    backoff_in_seconds=10,
+                )
+                for entry in btc_resp['addresses']:
+                    balances[entry['address']] = satoshis_to_btc(FVal(entry['final_balance']))
     except (
             requests.exceptions.RequestException,
             UnableToDecryptRemoteData,


Test patch:
Repository: rotki/rotki

Issue:
Editing an ethereum token fails with 400 error in 1.18.0
## Problem Definition

Trying to edit a token via the frontend returns a 400 Error when trying to change some attributes via the frontend.

## Task

Fix it. Also check why the tests did not catch this. This should not be happening.




Discussion:
Also as part of this issue make sure to add newly detected tokens with a `""` cryptocompare symbol so that no matching by symbol in cryptocompare is attempted.

Resolving patch:
diff --git a/rotkehlchen/assets/asset.py b/rotkehlchen/assets/asset.py
--- a/rotkehlchen/assets/asset.py
+++ b/rotkehlchen/assets/asset.py
@@ -596,7 +596,8 @@ def initialize(
             started: Optional[Timestamp] = None,
             swapped_for: Optional[Asset] = None,
             coingecko: Optional[str] = None,
-            cryptocompare: Optional[str] = None,
+            # add the token with inactive cryptocompare so querying is not attempted by symbol
+            cryptocompare: Optional[str] = '',
             protocol: Optional[str] = None,
             underlying_tokens: Optional[List[UnderlyingToken]] = None,
     ) -> Y:
diff --git a/rotkehlchen/globaldb/handler.py b/rotkehlchen/globaldb/handler.py
--- a/rotkehlchen/globaldb/handler.py
+++ b/rotkehlchen/globaldb/handler.py
@@ -186,7 +186,7 @@ def add_asset(
             swapped_for_asset = data.get('swapped_for', None)
             swapped_for = swapped_for_asset.identifier if swapped_for_asset else None
             coingecko = data.get('coingecko', None)
-            cryptocompare = data.get('cryptocompare', None)
+            cryptocompare = data.get('cryptocompare', '')
 
         try:
             cursor.execute(
@@ -420,7 +420,7 @@ def _add_underlying_tokens(
                         started, swapped_for, coingecko, cryptocompare, details_reference)
                         VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?)""",
                         (asset_id, 'C', None, None, None,
-                         None, None, None, underlying_token.address),
+                         None, None, '', underlying_token.address),
                     )
                 except sqlite3.IntegrityError as e:
                     connection.rollback()
@@ -764,7 +764,7 @@ def edit_custom_asset(data: Dict[str, Any]) -> None:
                     data.get('started'),
                     swapped_for,
                     data.get('coingecko'),
-                    data.get('cryptocompare'),
+                    data.get('cryptocompare', ''),
                     identifier,
                 ),
             )


Test patch:
Repository: rotki/rotki

Issue:
Please support UNIUP, UNIDOWN on Binance
## Abstract

Please support UNIUP, UNIDOWN on Binance


Discussion:
Sorry, we added this one in last version but forgot to remove it from the ignore list. Will fix for the next patch release :pray: 

Resolving patch:
diff --git a/rotkehlchen/assets/converters.py b/rotkehlchen/assets/converters.py
--- a/rotkehlchen/assets/converters.py
+++ b/rotkehlchen/assets/converters.py
@@ -486,8 +486,6 @@
     '456',  # https://twitter.com/rotkiapp/status/1161977327078838272
     '1INCHDOWN',  # no cryptocompare/coingecko data
     '1INCHUP',  # no cryptocompare/coingecko data
-    'UNIDOWN',  # no cryptocompare/coingecko data
-    'UNIUP',  # no cryptocompare/coingecko data
     'SXPDOWN',  # no cryptocompare/coingecko data
     'SXPUP',  # no cryptocompare/coingecko data
     'AAVEDOWN',  # no cryptocompare/coingecko data


Test patch:
Repository: rotki/rotki

Issue:
Fix test_get_events_history_filtering_by_timestamp for sushiswap
## Problem Definition

This test suddently started failing with no events found in the given time range. Need to figure out why (possibly subgraph problem?) and fix it.

The test was skipped here: https://github.com/rotki/rotki/commit/cb91b877e99d9096f91a9a53751ce3be0e2052ae

## Task

Do it


Discussion:

Resolving patch:
diff --git a/rotkehlchen/db/dbhandler.py b/rotkehlchen/db/dbhandler.py
--- a/rotkehlchen/db/dbhandler.py
+++ b/rotkehlchen/db/dbhandler.py
@@ -1132,7 +1132,7 @@ def get_amm_events(
         and address
         """
         cursor = self.conn.cursor()
-        events_sql_str = ", ".join([f'"{str(event)}"' for event in events])
+        events_sql_str = ", ".join([f'"{EventType.serialize_for_db(event)}"' for event in events])
         query = f'SELECT * FROM uniswap_events WHERE uniswap_events.type IN ({events_sql_str}) '
 
         # Timestamp filters are omitted, done via `form_query_to_filter_timestamps`
@@ -1223,7 +1223,9 @@ def delete_uniswap_trades_data(self) -> None:
     def delete_uniswap_events_data(self) -> None:
         """Delete all historical Uniswap events data"""
         cursor = self.conn.cursor()
-        uniswap_types = f'"{EventType.MINT_UNISWAP}", "{EventType.BURN_UNISWAP}"'
+        mint_uniswap = EventType.serialize_for_db(EventType.MINT_UNISWAP)
+        burn_uniswap = EventType.serialize_for_db(EventType.BURN_UNISWAP)
+        uniswap_types = f'"{mint_uniswap}", "{burn_uniswap}"'
         cursor.execute(
             f'DELETE FROM uniswap_events WHERE uniswap_events.type IN ({uniswap_types});',
         )
@@ -1248,7 +1250,9 @@ def delete_sushiswap_trades_data(self) -> None:
     def delete_sushiswap_events_data(self) -> None:
         """Delete all historical Sushiswap events data"""
         cursor = self.conn.cursor()
-        sushiswap_types = f'"{EventType.MINT_SUSHISWAP}", "{EventType.BURN_SUSHISWAP}"'
+        mint_sushiswap = EventType.serialize_for_db(EventType.MINT_SUSHISWAP)
+        burn_sushiswap = EventType.serialize_for_db(EventType.BURN_SUSHISWAP)
+        sushiswap_types = f'"{mint_sushiswap}", "{burn_sushiswap}"'
         cursor.execute(
             f'DELETE FROM uniswap_events WHERE uniswap_events.type IN ({sushiswap_types});',
         )


Test patch:
diff --git a/rotkehlchen/tests/api/test_sushiswap.py b/rotkehlchen/tests/api/test_sushiswap.py
--- a/rotkehlchen/tests/api/test_sushiswap.py
+++ b/rotkehlchen/tests/api/test_sushiswap.py
@@ -422,7 +422,6 @@ def test_get_sushiswap_trades_history(
 ]
 
 
-@pytest.mark.skip('Fails with no event found anymore in the given time range')
 @pytest.mark.parametrize('ethereum_accounts', [[TEST_EVENTS_ADDRESS_1]])
 @pytest.mark.parametrize('ethereum_modules', [['sushiswap']])
 @pytest.mark.parametrize('start_with_valid_premium', [True])
Repository: rotki/rotki

Issue:
Change current price API to POST instead of get
## Problem Definition

Due to the new format of the asset identifier, the query string becomes really huge when the user has a lot of assets.

Update both the backend and frontend to do a post instead of get when fetching the current price for multiple assets

### Task
- change get to post

Change current price API to POST instead of get
## Problem Definition

Due to the new format of the asset identifier, the query string becomes really huge when the user has a lot of assets.

Update both the backend and frontend to do a post instead of get when fetching the current price for multiple assets

### Task
- change get to post



Discussion:

Resolving patch:
diff --git a/rotkehlchen/api/v1/resources.py b/rotkehlchen/api/v1/resources.py
--- a/rotkehlchen/api/v1/resources.py
+++ b/rotkehlchen/api/v1/resources.py
@@ -1595,10 +1595,10 @@ def post(self, asset: Asset, file: FileStorage) -> Response:
 
 class CurrentAssetsPriceResource(BaseResource):
 
-    get_schema = CurrentAssetsPriceSchema()
+    post_schema = CurrentAssetsPriceSchema()
 
-    @use_kwargs(get_schema, location='json_and_query')
-    def get(
+    @use_kwargs(post_schema, location='json')
+    def post(
             self,
             assets: List[Asset],
             target_asset: Asset,


Test patch:
diff --git a/rotkehlchen/tests/api/test_current_assets_price.py b/rotkehlchen/tests/api/test_current_assets_price.py
--- a/rotkehlchen/tests/api/test_current_assets_price.py
+++ b/rotkehlchen/tests/api/test_current_assets_price.py
@@ -18,10 +18,10 @@
 }])
 def test_get_current_assets_price_in_usd(rotkehlchen_api_server):
     async_query = random.choice([False, True])
-    response = requests.get(
+    response = requests.post(
         api_url_for(
             rotkehlchen_api_server,
-            "currentassetspriceresource",
+            'currentassetspriceresource',
         ),
         json={
             'assets': ['BTC', 'USD', 'GBP'],
@@ -49,10 +49,10 @@ def test_get_current_assets_price_in_usd(rotkehlchen_api_server):
 def test_get_current_assets_price_in_btc(rotkehlchen_api_server):
 
     async_query = random.choice([False, True])
-    response = requests.get(
+    response = requests.post(
         api_url_for(
             rotkehlchen_api_server,
-            "currentassetspriceresource",
+            'currentassetspriceresource',
         ),
         json={
             'assets': ['BTC', 'USD', 'GBP'],
Repository: rotki/rotki

Issue:
Sushiswap trades are not exported in the PnL report
## Problem Definition

Running a PnL report for a premium rotki account that has addresses with sushiswap trades does not include them in the PnL report even though they are show in DeFi trades.

Rotki version: 1.20.1

## Task

Solve it.



Discussion:
I know who to blame. I believe I already know what is missing, will try to have it ready #soon

Resolving patch:
diff --git a/rotkehlchen/chain/ethereum/trades.py b/rotkehlchen/chain/ethereum/trades.py
--- a/rotkehlchen/chain/ethereum/trades.py
+++ b/rotkehlchen/chain/ethereum/trades.py
@@ -22,8 +22,8 @@
 )
 
 # Quick lookup for AMMTrade locations
-AMMTradeLocations = {Location.BALANCER, Location.UNISWAP}
-AMMTRADE_LOCATION_NAMES = Literal['balancer', 'uniswap']
+AMMTradeLocations = {Location.BALANCER, Location.UNISWAP, Location.SUSHISWAP}
+AMMTRADE_LOCATION_NAMES = Literal['balancer', 'uniswap', 'sushiswap']
 AMMSwapDBTuple = (
     Tuple[
         str,  # tx_hash


Test patch:
Repository: rotki/rotki

Issue:
xsushi shows double amount in balances
## Problem Definition

xsushi is queried by zerion sdk (sushibar) and queries to token balances resulting in a double calculation of the asset amount.



Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/ethereum/tokens.py b/rotkehlchen/chain/ethereum/tokens.py
--- a/rotkehlchen/chain/ethereum/tokens.py
+++ b/rotkehlchen/chain/ethereum/tokens.py
@@ -124,6 +124,10 @@ def query_tokens_for_addresses(
             # to approach it as there is no way to assign a price to 1 veCRV. It
             # can be 1 CRV locked for 4 years or 4 CRV locked for 1 year etc.
             string_to_ethereum_address('0x5f3b5DfEb7B28CDbD7FAba78963EE202a494e2A2'),
+            # Ignore for now xsushi since is queried by defi SDK. We'll do it for now
+            # since the SDK entry might return other tokens from sushi and we don't
+            # fully support sushi now.
+            string_to_ethereum_address('0x8798249c2E607446EfB7Ad49eC89dD1865Ff4272'),
         ]
         for asset in ignored_assets:  # don't query for the ignored tokens
             if asset.is_eth_token():  # type ignore since we know asset is a token


Test patch:
Repository: rotki/rotki

Issue:
Barnbirdge appears as unknown asset in coinbase pro
## Problem Definition

When opening v1.20.1 rotki I got this message:

"Found coinbase pro account result with unknown asset BOND. Ignoring it."

## Task

Fix it.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/assets/asset.py b/rotkehlchen/assets/asset.py
--- a/rotkehlchen/assets/asset.py
+++ b/rotkehlchen/assets/asset.py
@@ -401,6 +401,7 @@ def deserialize_from_db(cls, entry: UnderlyingTokenDBTuple) -> 'UnderlyingToken'
     strethaddress_to_identifier('0xaea46A60368A7bD060eec7DF8CBa43b7EF41Ad85'): 'FET',
     strethaddress_to_identifier('0xd2877702675e6cEb975b4A1dFf9fb7BAF4C91ea9'): 'WLUNA',
     strethaddress_to_identifier('0xBB0E17EF65F82Ab018d8EDd776e8DD940327B28b'): 'AXS',
+    strethaddress_to_identifier('0x0391D2021f89DC339F60Fff84546EA23E337750f'): 'BOND',
 }
 
 WORLD_TO_COINBASE = {


Test patch:
Repository: rotki/rotki

Issue:
ShapeShift incorrect rate causing losses in P&L
## Problem Definition

The development ShapeShift external exchange location registers trades with the rate in the quote_asset.  Rotki expects rate in base_asset.  To convert divide 1 by the rate given in the CSV artifact.


### System Description


Operating system: Mac OSX 11.6
Rotki version:  1.20.2.DEV



Discussion:
Hey thanks for opening an issue for this but there is no reason to open issues for development features :smile: 
Sure I wasn't sure what to name the branch for the PR, in the future I will just make it topical.

Resolving patch:
diff --git a/rotkehlchen/data/importer.py b/rotkehlchen/data/importer.py
--- a/rotkehlchen/data/importer.py
+++ b/rotkehlchen/data/importer.py
@@ -1016,6 +1016,10 @@ def _consume_shapeshift_trade(self, csv_row: Dict[str, Any]) -> None:
             buy_asset = A_SAI
         if sold_asset == A_DAI and timestamp <= SAI_TIMESTAMP:
             sold_asset = A_SAI
+        if rate <= ZERO:
+            log.warning(f'shapeshift csv entry has negative or zero rate. Ignoring. {csv_row}')
+            return
+
         trade = Trade(
             timestamp=timestamp,
             location=Location.SHAPESHIFT,
@@ -1023,7 +1027,7 @@ def _consume_shapeshift_trade(self, csv_row: Dict[str, Any]) -> None:
             quote_asset=sold_asset,
             trade_type=TradeType.BUY,
             amount=buy_amount,
-            rate=Price(rate),
+            rate=Price(1 / rate),
             fee=Fee(fee),
             fee_currency=buy_asset,  # Assumption that minerFee is denominated in outputCurrency
             link='',


Test patch:
diff --git a/rotkehlchen/tests/utils/dataimport.py b/rotkehlchen/tests/utils/dataimport.py
--- a/rotkehlchen/tests/utils/dataimport.py
+++ b/rotkehlchen/tests/utils/dataimport.py
@@ -643,7 +643,7 @@ def assert_shapeshift_trades_import_results(rotki: Rotkehlchen):
             quote_asset=A_SAI,
             trade_type=TradeType.BUY,
             amount=AssetAmount(FVal('0.59420343')),
-            rate=Price(FVal('0.00578758')),
+            rate=Price(1 / FVal('0.00578758')),
             fee=Fee(FVal('0.002')),
             fee_currency=symbol_to_asset_or_token('DASH'),
             link='',
@@ -656,7 +656,7 @@ def assert_shapeshift_trades_import_results(rotki: Rotkehlchen):
             quote_asset=A_USDC,
             trade_type=TradeType.BUY,
             amount=AssetAmount(FVal('0.06198721')),
-            rate=Price(FVal('0.00065004')),
+            rate=Price(1 / FVal('0.00065004')),
             fee=Fee(FVal('0.0042')),
             fee_currency=A_ETH,
             link='',
Repository: rotki/rotki

Issue:
Account already exists error after changing selected modules
## Problem Definition

If an user changes the selected modules while an account is being added the backend complains because the dictionary of active modules has been modified during the addition and process fails. If you try to add it again the frontend says that the account is already added. 

This gets solved if you logout and login again

### logs
```
[30/09/2021 12:12:55 CEST] ERROR rotkehlchen.api.rest Greenlet with id 139943860655696: Greenlet for task 65 dies with exception: dictionary changed size during iteration.
Exception Name: <class 'RuntimeError'>
Exception Info: dictionary changed size during iteration
Traceback:
   File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "rotkehlchen/api/rest.py", line 280, in _do_query_async
  File "rotkehlchen/api/rest.py", line 1753, in _add_blockchain_accounts
  File "rotkehlchen/rotkehlchen.py", line 466, in add_blockchain_accounts
  File "rotkehlchen/chain/manager.py", line 998, in add_blockchain_accounts
  File "rotkehlchen/chain/manager.py", line 1133, in modify_blockchain_accounts
  File "rotkehlchen/chain/manager.py", line 398, in iterate_modules
```

### System Description
Operating system:  Linux
Rotki version: dev



Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/manager.py b/rotkehlchen/chain/manager.py
--- a/rotkehlchen/chain/manager.py
+++ b/rotkehlchen/chain/manager.py
@@ -389,10 +389,11 @@ def process_new_modules_list(self, module_names: List[ModuleName]) -> None:
         modules_to_remove = existing_names.difference(given_modules_set)
         modules_to_add = given_modules_set.difference(existing_names)
 
-        for name in modules_to_remove:
-            self.deactivate_module(name)
-        for name in modules_to_add:
-            self.activate_module(name)
+        with self.eth_lock:
+            for name in modules_to_remove:
+                self.deactivate_module(name)
+            for name in modules_to_add:
+                self.activate_module(name)
 
     def iterate_modules(self) -> Iterator[Tuple[str, EthereumModule]]:
         for name, module in self.eth_modules.items():


Test patch:
Repository: rotki/rotki

Issue:
Uphold has an issue with "Exchanged amount" vs "Received amount"
## Problem Definition

The CSV provides information to determine the fee and gives the received amount (which is the amount after the fee).

Rotki expects the exchanged amount, which is the amount before the fee was subtracted.  When the fee is entered into rotki, it applies it by reducing the base asset amount (action buy) accordingly.  With the current uphold implementation the received amount in P&L is less than what was actually received resulting in not enough acquisitions at a later date when the full balance is sold.  

To determine this we have to determine the amount of the fee and add it back into the received amount to get to the exchanged amount.  Then when the fee is subtracted by rotki it will correctly show the received amount as the base amount.


### System Description


Operating system: OS X
Rotki version: 1.20.2



Discussion:

Resolving patch:
diff --git a/rotkehlchen/data/importer.py b/rotkehlchen/data/importer.py
--- a/rotkehlchen/data/importer.py
+++ b/rotkehlchen/data/importer.py
@@ -1136,6 +1136,9 @@ def _consume_uphold_transaction(self, csv_row: Dict[str, Any]) -> None:
                 )
                 self.db_ledger.add_ledger_action(action)
             else:  # Assets or amounts differ (Trades)
+                # in uphold UI the exchanged amount includes the fee.
+                if fee_asset == destination_asset:
+                    destination_amount = AssetAmount(destination_amount + fee)
                 if destination_amount > 0:
                     trade = Trade(
                         timestamp=timestamp,


Test patch:
Repository: rotki/rotki

Issue:
Uniswap V3 is missing trades to ETH
## Problem Definition

Uniswap V3 subgraph doesn't recognize trades from XXX -> ETH like https://etherscan.io/tx/0x1a4f28d3f661e5f1a64b37820867c65102f2be57dae206e4272226e87bf8d8b7

The reason in that we filter by the `recipient` field in the subgraph but this one doesn't seem to store the right address in case like this one. 

## Solution

Something similar happened with the `V2` subgraph. In this case @croco-finance has as covered and deployed a subgraph with the same fix as in V2

https://thegraph.com/hosted-service/subgraph/benesjan/uniswap-v3-subgraph

https://github.com/croco-finance/uniswap-v3-subgraph


Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/ethereum/modules/uniswap/graph.py b/rotkehlchen/chain/ethereum/modules/uniswap/graph.py
--- a/rotkehlchen/chain/ethereum/modules/uniswap/graph.py
+++ b/rotkehlchen/chain/ethereum/modules/uniswap/graph.py
@@ -8,7 +8,7 @@
         first: $limit,
         skip: $offset,
         where: {{
-            recipient: $address,
+            origin: $address,
             timestamp_gte: $start_ts,
             timestamp_lte: $end_ts,
         }}
diff --git a/rotkehlchen/chain/ethereum/modules/uniswap/uniswap.py b/rotkehlchen/chain/ethereum/modules/uniswap/uniswap.py
--- a/rotkehlchen/chain/ethereum/modules/uniswap/uniswap.py
+++ b/rotkehlchen/chain/ethereum/modules/uniswap/uniswap.py
@@ -51,7 +51,7 @@ class Uniswap(AMMSwapPlatform, EthereumModule):
 
     * Uniswap subgraph:
     https://github.com/Uniswap/uniswap-v2-subgraph
-    https://github.com/Uniswap/uniswap-v3-subgraph
+    https://github.com/croco-finance/uniswap-v3-subgraph
     """
     def __init__(
             self,
@@ -65,7 +65,7 @@ def __init__(
                 'https://api.thegraph.com/subgraphs/name/benesjan/uniswap-v2',
             )
             self.graph_v3 = Graph(
-                'https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3',
+                'https://api.thegraph.com/subgraphs/name/benesjan/uniswap-v3-subgraph',
             )
         except RemoteError as e:
             self.msg_aggregator.add_error(


Test patch: