internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: rotki/rotki

Issue:
Detected vault with collateral_type ETH-C. That is not yet supported by rotki. Skipping...
## Problem Definition

Please add ETH-C MakerDAO vault to Rotki.

## Logs

Backend messages:
```
Detected vault with collateral_type ETH-C. That is not yet supported by rotki. Skipping...
```


Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/ethereum/modules/makerdao/vaults.py b/rotkehlchen/chain/ethereum/modules/makerdao/vaults.py
--- a/rotkehlchen/chain/ethereum/modules/makerdao/vaults.py
+++ b/rotkehlchen/chain/ethereum/modules/makerdao/vaults.py
@@ -47,6 +47,7 @@
     MAKERDAO_DAI_JOIN,
     MAKERDAO_ETH_A_JOIN,
     MAKERDAO_ETH_B_JOIN,
+    MAKERDAO_ETH_C_JOIN,
     MAKERDAO_GET_CDPS,
     MAKERDAO_GUSD_A_JOIN,
     MAKERDAO_JUG,
@@ -92,6 +93,7 @@
     'BAT-A': MAKERDAO_BAT_A_JOIN,
     'ETH-A': MAKERDAO_ETH_A_JOIN,
     'ETH-B': MAKERDAO_ETH_B_JOIN,
+    'ETH-C': MAKERDAO_ETH_C_JOIN,
     'KNC-A': MAKERDAO_KNC_A_JOIN,
     'TUSD-A': MAKERDAO_TUSD_A_JOIN,
     'USDC-A': MAKERDAO_USDC_A_JOIN,
@@ -115,6 +117,7 @@
     'BAT-A': A_BAT,
     'ETH-A': A_ETH,
     'ETH-B': A_ETH,
+    'ETH-C': A_ETH,
     'KNC-A': A_KNC,
     'TUSD-A': A_TUSD,
     'USDC-A': A_USDC,
diff --git a/rotkehlchen/constants/ethereum.py b/rotkehlchen/constants/ethereum.py
--- a/rotkehlchen/constants/ethereum.py
+++ b/rotkehlchen/constants/ethereum.py
@@ -91,6 +91,7 @@ def abi(name: str) -> List[Dict[str, Any]]:
 MAKERDAO_VAT = EthereumConstants().contract('MAKERDAO_VAT')
 MAKERDAO_ETH_A_JOIN = EthereumConstants().contract('MAKERDAO_ETH_A_JOIN')
 MAKERDAO_ETH_B_JOIN = EthereumConstants().contract('MAKERDAO_ETH_B_JOIN')
+MAKERDAO_ETH_C_JOIN = EthereumConstants().contract('MAKERDAO_ETH_C_JOIN')
 MAKERDAO_BAT_A_JOIN = EthereumConstants().contract('MAKERDAO_BAT_A_JOIN')
 MAKERDAO_USDC_A_JOIN = EthereumConstants().contract('MAKERDAO_USDC_A_JOIN')
 MAKERDAO_USDC_B_JOIN = EthereumConstants().contract('MAKERDAO_USDC_B_JOIN')


Test patch:
Repository: rotki/rotki

Issue:
PnL CSV export cost basis info error splits entry in two lines
## Problem Definition

When exporting in CSV and specifically for the cost_basis column there is a bug that will split one CSV entry into two making rendering them both incorrect.

It's actually quite a simple bug. In the `to_string()` function of the cost_basis:

https://github.com/rotki/rotki/blob/c20564602f9608e10e300d94930e86f56c9f4d14/rotkehlchen/accounting/cost_basis.py#L131-L141

If a sell is matched with multiple buys, then they are separated by a comma due to the `",".join()` :sweat_smile: 

Which is bad, since comma is the default delimiting character in our CSV export

## Task

Fix it!


Discussion:

Resolving patch:
diff --git a/rotkehlchen/accounting/cost_basis.py b/rotkehlchen/accounting/cost_basis.py
--- a/rotkehlchen/accounting/cost_basis.py
+++ b/rotkehlchen/accounting/cost_basis.py
@@ -137,7 +137,7 @@ def to_string(self, converter: Callable[[Timestamp], str]) -> str:
         if len(self.matched_acquisitions) == 0:
             return value
 
-        value += f'Used: {",".join([x.to_string(converter) for x in self.matched_acquisitions])}'
+        value += f'Used: {"|".join([x.to_string(converter) for x in self.matched_acquisitions])}'
         return value
 
 


Test patch:
Repository: rotki/rotki

Issue:
Premium stats: Unused location loopring in value distribution by location
## Problem Definition

![2021-04-16_09-53](https://user-images.githubusercontent.com/1658405/114997574-48e73500-9ea0-11eb-83cd-93094f13febf.png)


## Task

Fix it



Discussion:

Resolving patch:
diff --git a/rotkehlchen/db/dbhandler.py b/rotkehlchen/db/dbhandler.py
--- a/rotkehlchen/db/dbhandler.py
+++ b/rotkehlchen/db/dbhandler.py
@@ -3133,7 +3133,7 @@ def get_latest_location_value_distribution(self) -> List[LocationData]:
         cursor = self.conn.cursor()
         results = cursor.execute(
             'SELECT time, location, usd_value FROM timed_location_data WHERE '
-            'time=(SELECT MAX(time) FROM timed_location_data);',
+            'time=(SELECT MAX(time) FROM timed_location_data) AND usd_value!=0;',
         )
         results = results.fetchall()
 
diff --git a/rotkehlchen/rotkehlchen.py b/rotkehlchen/rotkehlchen.py
--- a/rotkehlchen/rotkehlchen.py
+++ b/rotkehlchen/rotkehlchen.py
@@ -797,7 +797,8 @@ def query_balances(
                 force_token_detection=ignore_cache,
                 ignore_cache=ignore_cache,
             )
-            balances[str(Location.BLOCKCHAIN)] = blockchain_result.totals.assets
+            if len(blockchain_result.totals.assets) != 0:
+                balances[str(Location.BLOCKCHAIN)] = blockchain_result.totals.assets
             liabilities = blockchain_result.totals.liabilities
         except (RemoteError, EthSyncError) as e:
             problem_free = False
@@ -806,7 +807,9 @@ def query_balances(
 
         # retrieve loopring balances if module is activated
         if self.chain_manager.get_module('loopring'):
-            balances[str(Location.LOOPRING)] = self.chain_manager.get_loopring_balances()
+            loopring_balances = self.chain_manager.get_loopring_balances()
+            if len(loopring_balances) != 0:
+                balances[str(Location.LOOPRING)] = loopring_balances
 
         balances = account_for_manually_tracked_balances(db=self.data.db, balances=balances)
 


Test patch:
Repository: rotki/rotki

Issue:
FTX doesn't detect assets properly
## Problem Definition
A discord user complained about FTT not being detected. This is the only asset that doesn't have a direct mapping to our db. The issue comes from [this line](https://github.com/rotki/rotki/blob/develop/rotkehlchen/exchanges/ftx.py#L311) that uses `symbol_to_asset_or_token` instead of `asset_from_ftx`



Discussion:

Resolving patch:
diff --git a/rotkehlchen/exchanges/ftx.py b/rotkehlchen/exchanges/ftx.py
--- a/rotkehlchen/exchanges/ftx.py
+++ b/rotkehlchen/exchanges/ftx.py
@@ -14,7 +14,6 @@
 from rotkehlchen.accounting.structures import Balance
 from rotkehlchen.assets.asset import Asset
 from rotkehlchen.assets.converters import asset_from_ftx
-from rotkehlchen.assets.utils import symbol_to_asset_or_token
 from rotkehlchen.constants.misc import ZERO
 from rotkehlchen.errors import DeserializationError, RemoteError, UnknownAsset, UnsupportedAsset
 from rotkehlchen.exchanges.data_structures import AssetMovement, MarginPosition, Trade
@@ -308,7 +307,7 @@ def query_balances(self) -> ExchangeQueryBalances:
                 if amount == ZERO:
                     continue
 
-                asset = symbol_to_asset_or_token(balance_info['coin'])
+                asset = asset_from_ftx(balance_info['coin'])
                 try:
                     usd_price = Inquirer().find_usd_price(asset=asset)
                 except RemoteError as e:


Test patch:
Repository: rotki/rotki

Issue:
Add a ui for the users to manage the json config options
## Problem Definition

While the config works  (on macos I can't locate the directory). It is quite cumbersome to edit the file and find the proper location to put it in different OSes.

We should provide a UI where the user can edit the configuration before logging in.




Discussion:
- the AppImage works fine. 
- With the installer for windows, the file has to go to `%LOCALAPPDATA%\Programs\rotki\
> Properly escape paths C:\\rotki\\data and make sure the directory exists

on macos I cannot locate the directory, where the file has to be if you have the application installed

Resolving patch:
diff --git a/rotkehlchen/rotkehlchen.py b/rotkehlchen/rotkehlchen.py
--- a/rotkehlchen/rotkehlchen.py
+++ b/rotkehlchen/rotkehlchen.py
@@ -128,6 +128,7 @@ def __init__(self, args: argparse.Namespace) -> None:
             self.data_dir = default_data_directory()
         else:
             self.data_dir = Path(args.data_dir)
+            self.data_dir.mkdir(parents=True, exist_ok=True)
 
         if not os.access(self.data_dir, os.W_OK | os.R_OK):
             raise SystemPermissionError(


Test patch:
Repository: rotki/rotki

Issue:
Add convex airdrop
Convex finance will make an airdrop according to this rules https://convexfinance.medium.com/cvx-airdrop-details-a20ff7448075

The list of tentative addresses to receive the airdrop is generated by them and available here https://github.com/convex-eth/platform/blob/main/airdrop/userconvex_final_formatted.json

# Task

Add the airdrop to the airdrops sections


Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/ethereum/airdrops.py b/rotkehlchen/chain/ethereum/airdrops.py
--- a/rotkehlchen/chain/ethereum/airdrops.py
+++ b/rotkehlchen/chain/ethereum/airdrops.py
@@ -6,6 +6,7 @@
 
 import requests
 
+from rotkehlchen.assets.asset import EthereumToken
 from rotkehlchen.chain.ethereum.utils import token_normalized_value_decimals
 from rotkehlchen.constants.assets import (
     A_1INCH,
@@ -70,6 +71,11 @@
         A_CRV,
         'https://www.curve.fi/',
     ),
+    'convex': (
+        'https://gist.githubusercontent.com/LefterisJP/fd0ebccbc645f7de2b142907bd207363/raw/0613689dd5212b81788ed1a108c751b29b2ce93a/convex_airdrop.csv',  # noqa: E501
+        EthereumToken('0x4e3FBD56CD56c3e72c1403e103b45Db9da5B9D2B'),
+        'https://www.convexfinance.com/',
+    ),
 }
 
 POAP_AIRDROPS = {


Test patch:
Repository: rotki/rotki

Issue:
v1.17.0 will not work for MacOS premium users who are syncing the DB with the rotki server.
## Problem Definition

As usual MacOS has problems!

Upgrading pycryptodome lead to this: https://github.com/Legrandin/pycryptodome/issues/281 being hit for MacOS users.

And it was not seen in our smoke tests due to not testing with a premium account with DB sync.

## Task

Fix it by downgrading the package and making a new release.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/rotkehlchen.py b/rotkehlchen/rotkehlchen.py
--- a/rotkehlchen/rotkehlchen.py
+++ b/rotkehlchen/rotkehlchen.py
@@ -165,7 +165,7 @@ def reset_after_failed_account_creation_or_login(self) -> None:
         """If the account creation or login failed make sure that the Rotki instance is clear
 
         Tricky instances are when after either failed premium credentials or user refusal
-        to sync premium databases we relogged in.
+        to sync premium databases we relogged in
         """
         self.cryptocompare.db = None
 


Test patch:
Repository: rotki/rotki

Issue:
FTX fails to query information in 1.17
## Problem Definition

FTX shows an error on querying information in 1.17. DOesn't happen in 1.16

## Logs
```
[26/05/2021 12:47:12 CEST] DEBUG rotkehlchen.exchanges.ftx: FTX API query request_url=/api/wallet/all_balances?limit=100
[26/05/2021 12:47:12 CEST] ERROR rotkehlchen.exchanges.ftx: FTX API request failed. Could not reach FTX due to FTX query https://ftx.com/api/wallet/all_balances?limit=100 responded with error status code: 401 and text: {"success":false,"error":"Not logged in"}
[26/05/2021 12:47:14 CEST] DEBUG rotkehlchen.api.rest: Request successful response={"result": {"status": "completed", "outcome": {"result": null, "message": "FTX API request failed. Could not reach FTX due to FTX query https://ftx.com/api/wallet/all_balances?limit=100 responded with error status code: 401 and text: {\"success\":false,\"error\":\"Not logged in\"}\n"}, "status_code": 409}, "message": ""}, status_code=200
[26/05/2021 12:48:04 CEST] DEBUG rotkehlchen.exchanges.ftx: FTX API query request_url=/api/fills?limit=100&start_time=1619269215&end_time=1622026084
[26/05/2021 12:48:04 CEST] ERROR rotkehlchen.tasks.manager: FTX query https://ftx.com/api/fills?limit=100&start_time=1619269215&end_time=1622026084 responded with error status code: 401 and text: {"success":false,"error":"Not logged in"}
```

### System Description


Operating system: Linux
Rotki version: 1.17



Discussion:

Resolving patch:
diff --git a/rotkehlchen/exchanges/ftx.py b/rotkehlchen/exchanges/ftx.py
--- a/rotkehlchen/exchanges/ftx.py
+++ b/rotkehlchen/exchanges/ftx.py
@@ -102,6 +102,7 @@ def __init__(
         self.apiversion = 'v2'
         self.base_uri = 'https://ftx.com'
         self.msg_aggregator = msg_aggregator
+        self.session.headers.update({'FTX-KEY': self.api_key})
 
     def first_connection(self) -> None:
         self.first_connection_made = True


Test patch:
Repository: rotki/rotki

Issue:
Binance withdrawal id instead of txId
## Problem Definition

Auserreported that it seems that now binance's response of the withdrawals endpoint can contain either a `txId` or an `id` field for the transaction hash.

```
ERROR rotkehlchen.exchanges.binance: Error processing a binance deposit/withdrawal asset_movement={'amount': 50, 'transactionFee': 0.65, 'address': '0x0', 'withdrawOrderId': '', 'addressTag': '', 'id': 'x', 'asset': 'MCO', 'applyTime': 1520624790000, 'status': 1}, error=Missing key entry for 'txId'.
```

This is confusing and does not seem to agree with their api docs: https://binance-docs.github.io/apidocs/spot/en/#withdraw-history-user_data

It seems as if the response is from this endpoint: https://binance-docs.github.io/apidocs/spot/en/#withdraw-history-supporting-network-user_data  which we don't call.

## Task

This is a puzzling result. The only thing we can do is just search either for `id` or `txId` in the response and be done with this particular problem ... but if they start mixing responses from different endpoints without any warning or api update we will have more problems.




Discussion:

Resolving patch:
diff --git a/rotkehlchen/exchanges/binance.py b/rotkehlchen/exchanges/binance.py
--- a/rotkehlchen/exchanges/binance.py
+++ b/rotkehlchen/exchanges/binance.py
@@ -939,17 +939,20 @@ def _deserialize_asset_movement(self, raw_data: Dict[str, Any]) -> Optional[Asse
 
             timestamp = deserialize_timestamp_from_binance(raw_data[time_key])
             asset = asset_from_binance(raw_data['asset'])
+            tx_id = get_key_if_has_val(raw_data, 'txId')
+            internal_id = get_key_if_has_val(raw_data, 'id')
+            link_str = str(internal_id) if internal_id else str(tx_id) if tx_id else ''
             return AssetMovement(
                 location=self.location,
                 category=category,
                 address=deserialize_asset_movement_address(raw_data, 'address', asset),
-                transaction_id=get_key_if_has_val(raw_data, 'txId'),
+                transaction_id=tx_id,
                 timestamp=timestamp,
                 asset=asset,
                 amount=deserialize_asset_amount_force_positive(raw_data['amount']),
                 fee_asset=asset,
                 fee=fee,
-                link=str(raw_data['txId']),
+                link=link_str,
             )
 
         except UnknownAsset as e:


Test patch:
Repository: rotki/rotki

Issue:
flaky 502 on kusama node request
## Problem Definition

This happened on v1.17.1 on a kusama node request. Doesn't happen always.


```
[30/05/2021 00:28:12 CEST] ERROR rotkehlchen.greenlets: Kusama manager connection to onfinality node died with exception: RPC request failed with HTTP status code 502.
Exception Name: <class 'substrateinterface.exceptions.SubstrateRequestException'>
Exception Info: RPC request failed with HTTP status code 502
Traceback:
   File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "rotkehlchen/chain/substrate/manager.py", line 248, in _connect_node
  File "rotkehlchen/chain/substrate/manager.py", line 434, in _get_node_interface
  File "site-packages/substrateinterface/base.py", line 450, in __init__
  File "site-packages/substrateinterface/base.py", line 638, in ss58_format
  File "site-packages/substrateinterface/base.py", line 595, in properties
  File "site-packages/substrateinterface/base.py", line 576, in rpc_request
 
[30/05/2021 00:28:12 CEST] ERROR rotkehlchen.user_messages: Kusama manager connection to onfinality node died with exception: RPC request failed with HTTP status code 502. Check the logs for more details

```
## Task

Figure out why this happened and fix it.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/chain/substrate/manager.py b/rotkehlchen/chain/substrate/manager.py
--- a/rotkehlchen/chain/substrate/manager.py
+++ b/rotkehlchen/chain/substrate/manager.py
@@ -433,7 +433,7 @@ def _get_node_interface(self, endpoint: str) -> SubstrateInterface:
                 type_registry_preset=si_attributes.type_registry_preset,
                 use_remote_preset=True,
             )
-        except (requests.exceptions.RequestException, WebSocketException) as e:
+        except (requests.exceptions.RequestException, WebSocketException, SubstrateRequestException) as e:  # noqa: E501
             message = (
                 f'{self.chain} could not connect to node at endpoint: {endpoint}. '
                 f'Connection error: {str(e)}.'


Test patch:
Repository: rotki/rotki

Issue:
Invalidate token cache at token addition and assets update
## Problem Definition

This should go to bugfixes.

When a user adds a new token the token detection cache is not invalidated which makes it possible to miss balances for this token if the user also owns it.

## Task

- Invalidate token detection cache when user adds a new ethereum token in the asset manager
- Invalidate token detection cache when user updates the asset list.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/api/rest.py b/rotkehlchen/api/rest.py
--- a/rotkehlchen/api/rest.py
+++ b/rotkehlchen/api/rest.py
@@ -1417,6 +1417,9 @@ def add_custom_ethereum_token(self, token: CustomEthereumToken) -> Response:
         except InputError as e:
             return api_response(wrap_in_fail_result(str(e)), status_code=HTTPStatus.CONFLICT)
 
+        cursor = self.rotkehlchen.data.db.conn.cursor()
+        # clean token detection cache. Next function will commit to the DB
+        cursor.execute('DELETE from ethereum_accounts_details;')
         self.rotkehlchen.data.db.add_asset_identifiers([identifier])
         return api_response(
             _wrap_in_ok_result({'identifier': identifier}),
@@ -2974,6 +2977,9 @@ def _perform_assets_updates(
             return {'result': None, 'message': str(e), 'status_code': HTTPStatus.BAD_GATEWAY}
 
         if result is None:
+            cursor = self.rotkehlchen.data.db.conn.cursor()
+            # clean token detection cache. Next function will commit to the DB
+            cursor.execute('DELETE from ethereum_accounts_details;')
             self.rotkehlchen.data.db.add_globaldb_assetids()
             return OK_RESULT
 
diff --git a/rotkehlchen/db/dbhandler.py b/rotkehlchen/db/dbhandler.py
--- a/rotkehlchen/db/dbhandler.py
+++ b/rotkehlchen/db/dbhandler.py
@@ -1763,9 +1763,8 @@ def remove_blockchain_accounts(
                 f'{blockchain.value} accounts that do not exist',
             )
 
-        # Also remove all ethereum address details saved in the D
+        # Also remove all ethereum address details saved in the DB
         if blockchain == SupportedBlockchain.ETHEREUM:
-
             for address in accounts:
                 self.delete_data_for_ethereum_address(address)  # type: ignore
 
diff --git a/rotkehlchen/globaldb/updates.py b/rotkehlchen/globaldb/updates.py
--- a/rotkehlchen/globaldb/updates.py
+++ b/rotkehlchen/globaldb/updates.py
@@ -355,7 +355,10 @@ def perform_update(
         """Performs an asset update by downloading new changes from the remote
 
         If `up_to_version` is given then changes up to and including that version are made.
-        If not all possible changes are applied
+        If not all possible changes are applied.
+
+        For success returns None. If there is conflicts a list of conflicting
+        assets identifiers is going to be returned.
 
         May raise:
             - RemoteError if there is a problem querying Github


Test patch:
Repository: rotki/rotki

Issue:
Can't setup new Bitfinex api key since v1.17.1
## Problem Definition

A user reported that they can not set up a new bitfinex api key since v1.17.1

Reported error is:
```
ERROR rotkehlchen.exchanges.manager: Failed to validate API key for bitfinex exchange bitfinex 1 due to Provided API key/secret is invalid or does not have enough permissions. Make sure it has get/read permission of both "Balances" and "Account
```

## Task

Investigate why and if a bug fix.


Discussion:

Resolving patch:
diff --git a/rotkehlchen/exchanges/bitfinex.py b/rotkehlchen/exchanges/bitfinex.py
--- a/rotkehlchen/exchanges/bitfinex.py
+++ b/rotkehlchen/exchanges/bitfinex.py
@@ -167,7 +167,7 @@ def _api_query(
         """Request a Bitfinex API v2 endpoint (from `endpoint`).
         """
         call_options = options.copy() if options else {}
-        for header in ('Content-Type', 'bfx-apikey', 'bfx-nonce', 'bfx-signature'):
+        for header in ('Content-Type', 'bfx-nonce', 'bfx-signature'):
             self.session.headers.pop(header, None)
 
         if endpoint == 'configs_list_currency':


Test patch:
Repository: rotki/rotki

Issue:
Binance US can't fetch balances `API-key format invalid`
Retried a few time to regenerate and re-import api key and secret, but still fails to fetch balances:

**Error:**
`Binance US account API request failed. Could not reach binance due to Binance US API request https://api.binance.us/sapi/v1/lending/union/account?recvWindow=10000&timestamp=1622602107180&signature=MY_SIGNATURE for lending/union/account failed with HTTP status code: 500, error code: -1000 and error message: An unknown error occurred while processing the request.`

`Response`
```json
{"code":-2014,"msg":"API-key format invalid."}
```

App: 1.17.1


Discussion:
Hey @CruiseMan that's strange. Can you hop in discord and share a read only version of the key? We don't have any binance.us key to test with and the binance keys we have work fine.

Resolving patch:
diff --git a/rotkehlchen/exchanges/binance.py b/rotkehlchen/exchanges/binance.py
--- a/rotkehlchen/exchanges/binance.py
+++ b/rotkehlchen/exchanges/binance.py
@@ -800,7 +800,7 @@ def query_balances(self) -> ExchangeQueryBalances:
             self.first_connection()
             returned_balances: DefaultDict[Asset, Balance] = defaultdict(Balance)
             returned_balances = self._query_spot_balances(returned_balances)
-            if self.name != str(Location.BINANCEUS):
+            if self.location != Location.BINANCEUS:
                 returned_balances = self._query_lending_balances(returned_balances)
                 returned_balances = self._query_cross_collateral_futures_balances(returned_balances)  # noqa: E501
                 returned_balances = self._query_margined_futures_balances('fapi', returned_balances)  # noqa: E501


Test patch: