Repository: rotki/rotki

Issue:
Cannot delete an XPUB key that has no coins
## Problem Definition

After adding an XPUB key that has balance on its derived address, I can not delete that XPUB from my database anymore. I assume this is because the XPUB address does not show up in the list BTC per account balances since there's no balance. 

### System Description

Here add a detailed description of your system, e.g. output of the following script:

```
uname -a
command -v geth && geth version
command -v parity && parity --version
command -v rotki && rotki version
[ -d .git ] && git rev-parse HEAD
```

Darwin ChocoBookPro.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64

rotki is version 18.1



Discussion:
Hey @coblee let me see if I understand. You added an account xpub. Did you see any addresses or not? If not then are you sure you added an account xpub? This is the only xpubs we can derive from at the moment. For example if you take the xpub of an electrum wallet the addresses will be properly derived.

And your problem is that with an xpub that has never had any derived addresses, it does not show in the UI but is still in the DB and you can't delete it from there. Right?

That last part is indeed a bug. We should handle it.
It is indeed an account xpub. The issue is that my derivation path is m/49'/0'/0', but I added it with no derivation path specified. I assume because of that it probably defaulted to m/44 or something. So it found no coins.

Afterwards, I tried adding the same xpub with the m/49'/0'/0' derivation path, but that failed also. I think that's another bug, but I can't be sure until I can delete this entry.
This is our first attempt at derivation of addresses from xpubs so it's still experimental but in its basic form it should work. We tried it with 2 electrum wallets and the test xpub from ledger for its accounts: `xpub6DCi5iJ57ZPd5qPzvTm5hUt6X23TJdh9H4NjNsNbt7t7UuTMJfawQWsdWRFhfLwkiMkB1rQ4ZJWLB9YBnzR7kbs9N8b2PsKZgKUHQm1X4or`
https://github.com/LedgerHQ/bitcoin-keychain-svc/blob/744736af1819cdab0a46ea7faf834008aeade6b1/integration/p2pkh_keychain_test.go#L40-L95


The intended function of the xpub derivation for now is for simple usage. Just a simple account xpub. You can also specify a derivation path that will lead from the given xpub to the final account xpub. That's why we have the extra derivation path now.

Once we get the account xpub we start the derivation in the way most BTC wallets do it.

So we do (pseudocode):

```python
receiving_xpub = account_xpub.derive_child(0)
addresses.extend(derived_used_addresses(receiving_xpub))
change_xpub = account_xpub.derive_child(1)
addresses.extend(derived_used_addresses(change_xpub))
```


@coblee I suspect what you are experiencing is due to our current implementation. It has some limitations which are explained here: https://github.com/rotki/rotki/issues/1562. We only support legacy xpubs and native segwit ones starting with zpub at the moment.

(Bitcoin side of things is still new to me)

Next release by implementing what #1562 says I hope to provide a more complete functionality. As a more seasoned dev in the bitcoin side of things can you check if that would make sense?
Yup, that must be it. I don't know BIP44/BIP49 well, but yes, my addresses are P2SH-P2WPKH and derivation path is m/49'/0'/0'.

Anyways, this bug is about deleting an existing XPUB account. I will track the other bug separately.

Resolving patch:
diff --git a/rotkehlchen/api/rest.py b/rotkehlchen/api/rest.py
--- a/rotkehlchen/api/rest.py
+++ b/rotkehlchen/api/rest.py
@@ -1153,33 +1153,7 @@ def delete_xpub(self, xpub_data: 'XpubData', async_query: bool) -> Response:
 
     @require_loggedin_user()
     def get_blockchain_accounts(self, blockchain: SupportedBlockchain) -> Response:
-        account_data = self.rotkehlchen.data.db.get_blockchain_account_data(blockchain)
-        if blockchain == SupportedBlockchain.BITCOIN:
-            xpub_data = self.rotkehlchen.data.db.get_bitcoin_xpub_data()
-            addresses_to_account_data = {x.address: x for x in account_data}
-            address_to_xpub_mappings = self.rotkehlchen.data.db.get_addresses_to_xpub_mapping(
-                list(addresses_to_account_data.keys()),  # type: ignore
-            )
-
-            xpub_mappings: Dict[XpubData, List[BlockchainAccountData]] = {}
-            for address, xpub_entry in address_to_xpub_mappings.items():
-                if xpub_entry not in xpub_mappings:
-                    xpub_mappings[xpub_entry] = []
-                xpub_mappings[xpub_entry].append(addresses_to_account_data[address])
-
-            data: Dict[str, Any] = {'standalone': [], 'xpubs': []}
-            # Add xpub data
-            for xpub_entry in xpub_data:
-                data_entry = xpub_entry.serialize()
-                addresses = xpub_mappings.get(xpub_entry, None)
-                data_entry['addresses'] = addresses if addresses and len(addresses) != 0 else None
-                data['xpubs'].append(data_entry)
-            # Add standalone addresses
-            for account in account_data:
-                if account.address not in address_to_xpub_mappings:
-                    data['standalone'].append(account)
-        else:
-            data = account_data  # type: ignore
+        data = self.rotkehlchen.get_blockchain_account_data(blockchain)
         return api_response(process_result(_wrap_in_result(data, '')), status_code=HTTPStatus.OK)
 
     def _add_blockchain_accounts(
@@ -1250,9 +1224,8 @@ def edit_blockchain_accounts(
             return api_response(wrap_in_fail_result(str(e)), status_code=HTTPStatus.BAD_REQUEST)
 
         # success
-        data = self.rotkehlchen.data.db.get_blockchain_account_data(blockchain)
-        result_dict = _wrap_in_result([entry._asdict() for entry in data], '')
-        return api_response(process_result(result_dict), status_code=HTTPStatus.OK)
+        data = self.rotkehlchen.get_blockchain_account_data(blockchain)
+        return api_response(process_result(_wrap_in_result(data, '')), status_code=HTTPStatus.OK)
 
     def _remove_blockchain_accounts(
             self,
diff --git a/rotkehlchen/rotkehlchen.py b/rotkehlchen/rotkehlchen.py
--- a/rotkehlchen/rotkehlchen.py
+++ b/rotkehlchen/rotkehlchen.py
@@ -6,7 +6,7 @@
 import time
 from collections import defaultdict
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Tuple, Union, overload
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, Union, overload
 
 import gevent
 from gevent.lock import Semaphore
@@ -66,6 +66,9 @@
 from rotkehlchen.user_messages import MessagesAggregator
 from rotkehlchen.utils.misc import combine_stat_dicts, dict_get_sumof, merge_dicts
 
+if TYPE_CHECKING:
+    from rotkehlchen.chain.bitcoin.xpub import XpubData
+
 logger = logging.getLogger(__name__)
 log = RotkehlchenLogsAdapter(logger)
 
@@ -357,6 +360,40 @@ def main_loop(self) -> None:
                     xpub_derivation_scheduled = True
                 log.debug('Main loop end')
 
+    def get_blockchain_account_data(
+            self,
+            blockchain: SupportedBlockchain,
+    ) -> Union[List[BlockchainAccountData], Dict[str, Any]]:
+        account_data = self.data.db.get_blockchain_account_data(blockchain)
+        if blockchain != SupportedBlockchain.BITCOIN:
+            return account_data
+
+        xpub_data = self.data.db.get_bitcoin_xpub_data()
+        addresses_to_account_data = {x.address: x for x in account_data}
+        address_to_xpub_mappings = self.data.db.get_addresses_to_xpub_mapping(
+            list(addresses_to_account_data.keys()),  # type: ignore
+        )
+
+        xpub_mappings: Dict['XpubData', List[BlockchainAccountData]] = {}
+        for address, xpub_entry in address_to_xpub_mappings.items():
+            if xpub_entry not in xpub_mappings:
+                xpub_mappings[xpub_entry] = []
+            xpub_mappings[xpub_entry].append(addresses_to_account_data[address])
+
+        data: Dict[str, Any] = {'standalone': [], 'xpubs': []}
+        # Add xpub data
+        for xpub_entry in xpub_data:
+            data_entry = xpub_entry.serialize()
+            addresses = xpub_mappings.get(xpub_entry, None)
+            data_entry['addresses'] = addresses if addresses and len(addresses) != 0 else None
+            data['xpubs'].append(data_entry)
+        # Add standalone addresses
+        for account in account_data:
+            if account.address not in address_to_xpub_mappings:
+                data['standalone'].append(account)
+
+        return data
+
     def add_blockchain_accounts(
             self,
             blockchain: SupportedBlockchain,


Test patch:
diff --git a/rotkehlchen/tests/api/test_blockchain.py b/rotkehlchen/tests/api/test_blockchain.py
--- a/rotkehlchen/tests/api/test_blockchain.py
+++ b/rotkehlchen/tests/api/test_blockchain.py
@@ -937,6 +937,10 @@ def test_add_blockchain_accounts_with_tags_and_label_and_querying_them(rotkehlch
 
 
 @pytest.mark.parametrize('number_of_eth_accounts', [3])
+@pytest.mark.parametrize('btc_accounts', [[
+    UNIT_BTC_ADDRESS1,
+    UNIT_BTC_ADDRESS2,
+]])
 def test_edit_blockchain_accounts(
         rotkehlchen_api_server,
         ethereum_accounts,
@@ -1004,6 +1008,34 @@ def test_edit_blockchain_accounts(
     json_data = response.json()
     compare_account_data(json_data['result'], expected_result)
 
+    # Edit a BTC account
+    request_data = {'accounts': [{
+        'address': UNIT_BTC_ADDRESS1,
+        'label': 'BTC account label',
+        'tags': ['public'],
+    }]}
+    response = requests.patch(api_url_for(
+        rotkehlchen_api_server,
+        "blockchainsaccountsresource",
+        blockchain='BTC',
+    ), json=request_data)
+    result = assert_proper_response_with_result(response)
+    assert len(result) == 2
+    # Assert the result is in the expected format and is edited
+    standalone = result['standalone']
+    assert len(standalone) == 2
+    assert standalone[0] == {
+        'address': UNIT_BTC_ADDRESS1,
+        'label': 'BTC account label',
+        'tags': ['public'],
+    }
+    assert standalone[1] == {
+        'address': UNIT_BTC_ADDRESS2,
+        'label': None,
+        'tags': None,
+    }
+    assert len(result['xpubs']) == 0
+
 
 @pytest.mark.parametrize('number_of_eth_accounts', [2])
 def test_edit_blockchain_account_errors(